End-to-end automatic driving system based on electroencephalogram intention modeling and machine mental reasoning

Abstract

The invention relates to the technical field of automatic driving, and discloses an end-to-end automatic driving system based on electroencephalogram intention modeling and machine mental reasoning, and the system comprises an electroencephalogram signal collection module which is used for collecting an electroencephalogram signal of a driver in real time and carrying out the preprocessing of the electroencephalogram signal; the electroencephalogram intention decoding module is used for extracting driving intention features and outputting structured intention vectors; the machine mental reasoning module is used for fusing the intention vector, the vehicle state and the environment perception information, deducing the mental state of the driver and outputting a mental state vector; and the end-to-end automatic driving control module is used for performing multi-modal fusion on the intention vector, the mental state vector, the environment perception characteristics and the vehicle dynamic state to generate a continuous vehicle control instruction. According to the method, the neurocognitive activity of the driver can be converted into decision-making elements which can be understood and utilized by a machine in real time, so that man-machine cognitive alignment and closed-loop cooperative control are realized.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving technology, specifically to an end-to-end autonomous driving system based on EEG intention modeling and machine mental reasoning. Background Technology

[0002] Current advanced driver assistance systems (ADAS) primarily rely on external environmental perception information (such as cameras, millimeter-wave radar, and lidar) and vehicle state parameters (such as speed, acceleration, and steering angle) at the decision-making and control levels. Their design assumptions are typically based on the premise that driver behavior is predictable or that the driver can take over promptly and rationally when necessary. However, in real-world driving, when faced with highly uncertain scenarios such as complex urban road conditions, sudden traffic incidents, or mixed pedestrian and vehicle traffic, the system's decision-making mechanism often appears rigid and lagging because it cannot understand and predict the inherent intentions and potential behaviors of human drivers or other road users. To compensate for the shortcomings of this assumption in real-world driving, some existing technologies introduce driver state monitoring modules to assist in assessing driver fatigue, distraction, or attention levels; however, these technologies still have significant limitations.

[0003] First, at the level of modeling driver intent and decision-making motivation, existing technologies mostly rely on explicit behavioral signals or post-event response signals, such as steering wheel operation, pedal input, gaze deviation, or takeover actions. These signals often lag behind the process of generating the driver's true intent, making it difficult to reflect the driver's subjective judgment of risk, level of readiness for takeover, or strategic preferences in advance. This results in autonomous driving systems only being able to respond passively after a conflict has already occurred, posing significant safety hazards.

[0004] Secondly, some studies have attempted to introduce physiological signals such as electroencephalograms (EEGs) for driver state recognition, but these are mostly limited to classifying and judging single cognitive states such as fatigue and alertness. Furthermore, they are typically presented as independent modules, used only for alarm or prompt control, and are not deeply coupled with the autonomous driving decision-making and control process. Because EEG signals are susceptible to motion artifacts, individual differences, and interference from the in-vehicle electromagnetic environment, without a systematic intent modeling and semantic mapping mechanism, their output cannot directly participate in autonomous driving control decisions, limiting their engineering usability and stability.

[0005] Secondly, regarding human-machine collaborative decision-making mechanisms, existing autonomous driving systems generally lack explicit modeling of the driver's internal cognitive state, failing to characterize key "mental" factors such as the driver's cognitive load level, emotional state, risk perception ability, and level of trust in the autonomous driving system. Existing human-machine co-driving modes mostly use fixed rules or threshold triggering methods for control switching, failing to combine real-time cognitive abilities and changes in intent for dynamic reasoning, easily leading to takeover conflicts, trust mismatches, or excessive intervention in complex scenarios. Furthermore, existing technologies typically employ a separate design at the control architecture level, meaning driver state perception, decision-making and planning, and vehicle control are independent, with multiple layers of information flow and significant latency, making it difficult to achieve real-time response to human cognitive states and end-to-end collaborative optimization. This fragmented architecture limits the practical value of the driver's internal cognitive information in autonomous driving control.

[0006] In summary, the current autonomous driving architecture is still essentially "vehicle-centric," and driver status information is not organically integrated into the control loop. It cannot achieve intelligent collaboration that dynamically adjusts autonomous driving strategies based on the driver's real-time cognitive abilities and plans for smooth transitions of responsibilities in advance. This limits the overall safety and user experience of the system in dealing with complex scenarios. Summary of the Invention

[0007] The present invention aims to provide an end-to-end autonomous driving system based on EEG intention modeling and machine mental reasoning, which can transform the driver's neurocognitive activities into decision elements that can be understood and utilized by the machine in real time, thereby achieving human-machine cognitive alignment and closed-loop collaborative control.

[0008] The basic solution provided by this invention is: an end-to-end autonomous driving system based on EEG intention modeling and machine mental reasoning, comprising: The EEG signal acquisition module is used to acquire and preprocess the driver's EEG signals in real time. The EEG intention decoding module is connected to the EEG signal acquisition module and is used to extract driving intention features based on the preprocessed EEG signals and output a structured intention vector. The machine mind reasoning module is connected to the EEG intention decoding module and is used to fuse the intention vector, vehicle status and environmental perception information to infer at least one of the driver's cognitive load, attention state, emotional state and trust level, and output a mind state vector. An end-to-end autonomous driving control module is used to perform multimodal fusion of the intent vector, the mental state vector, environmental perception features and vehicle dynamics state to generate continuous vehicle control commands. The system achieves closed-loop coordinated control from the driver's neural intention perception to the vehicle control output.

[0009] The working principle and advantages of this invention are as follows: This invention relates to an end-to-end autonomous driving system based on EEG intention modeling and machine mental reasoning. It can transform the driver's neurocognitive activities into machine-understandable and usable decision-making elements in real time, thereby achieving human-machine cognitive alignment and closed-loop collaborative control. The key points are: This solution directly interprets the brain electrical activity characteristics related to motor preparation and decision-making, enabling it to identify key driving intentions such as takeover intent, risk aversion, and route selection preferences at the early stages when the brain forms clear operational commands, even before the driver is fully aware of their intentions. These intentions are then transformed into structured probability distribution vectors, giving the autonomous driving system predictive capabilities and allowing it to incorporate the future behavior of this key variable—the driver—into current decision-making and planning.

[0010] More importantly, this solution further uses a machine mental reasoning module to model these intentions in relation to deeper mental factors such as the driver's real-time cognitive load, attention resource allocation, emotional state, and level of trust in the system. This allows the system to not only know what the driver wants to do, but also assess whether the driver can do it well in the current state. Based on a comprehensive and continuous profile of the driver's cognitive ability, the system can dynamically adjust the allocation strategy of control and the intensity of intervention in autonomous driving behavior. This represents a fundamental shift from taking over after a conflict occurs to anticipating and adjusting the load, which helps improve the smoothness, safety, and user trust in the human-machine co-driving process.

[0011] Furthermore, this solution overcomes the information barriers and decision-making delays inherent in traditional modular designs. Innovatively, it unifies and jointly models the decoded driver intent vector, inferred mental state vector, environmental perception features, and vehicle dynamics state at the raw or feature level within the end-to-end autonomous driving control module. Through training based on methods such as deep reinforcement learning, the control module can directly learn to generate optimal continuous control commands (such as steering, throttle, and braking) from this fused multimodal state representation. Its advantage lies in allowing the system to comprehensively consider the complex interactions between the driver's cognitive state and external environmental constraints at the decision-making stage. For example, the system can learn to choose a more conservative but less restrictive trajectory even in the same traffic scenario when the driver's cognitive load is high; while when the driver's intent is clear and attention is focused, it may execute more efficient but slightly more complex operations. This eliminates the distortion and loss of information transmitted between modules in traditional architectures, achieving global optimization of the control strategy. The resulting autonomous driving behavior is not only safe and compliant but also highly adapted to the driver's current cognitive characteristics, achieving a human-like and personalized driving experience. Attached Figure Description

[0012] Figure 1 This is a schematic diagram illustrating the module operation principle of an embodiment of the end-to-end autonomous driving system based on EEG intention modeling and machine mental reasoning of the present invention. Figure 2 This is a schematic diagram illustrating the application process of an end-to-end autonomous driving system based on EEG intention modeling and machine mental reasoning, according to an embodiment of the present invention. Detailed Implementation

[0013] The following detailed description illustrates the specific implementation methods: The basic implementation examples are as follows: Figure 1 As shown: An end-to-end autonomous driving system based on EEG intention modeling and machine mental reasoning includes an EEG signal acquisition module, an EEG intention decoding module, a machine mental reasoning module, an end-to-end autonomous driving control module, and an online learning and adaptive optimization module.

[0014] The system achieves closed-loop coordinated control from driver's neural intention perception to vehicle control output through the above modules. Furthermore, the system supports Level 2 or Level 3 autonomous driving scenarios and is suitable for at least one of the following conditions: highways, urban roads, and unguided left turns and extreme U-turns.

[0015] The EEG signal acquisition module is used to acquire the driver's EEG signals in real time and perform preprocessing.

[0016] The EEG signal acquisition module includes a wearable EEG cap, which supports simultaneous acquisition of multi-channel signals and connects to the vehicle controller via wireless communication. In this embodiment, a 32-channel wireless EEG headset conforming to international standards can be selected.

[0017] Specifically, the EEG signal acquisition module collects EEG signals through EEG electrodes attached to the driver's scalp using a wearable EEG cap, obtaining its time-domain and frequency-domain characteristics, including changes in EEG power at different frequency bands and related neural activity indicators. After preprocessing operations such as noise reduction, artifact removal, and feature extraction, the EEG features are sent to the EEG intention decoding module for driving intention recognition and cognitive state assessment.

[0018] In this embodiment, the EEG signal acquisition module acquires multi-channel EEG signals in real time using a 32-channel or higher wireless EEG headset worn on the driver's scalp, conforming to international standards. The acquired signals primarily include: event-related desynchronization / synchronization signals of the sensorimotor cortex μ and β rhythms, which are closely related to motor preparation and execution; steady-state visual evoked potentials and parieto-occipital α rhythms, reflecting attention allocation and visual processing; event-related potentials such as P300, characterizing cognitive assessment and unexpected event detection; and the power of theta rhythms, which are related to prefrontal cognitive load and decision-making monitoring.

[0019] The preprocessing operations sequentially include hardware and power frequency filtering, physiological artifact removal based on independent component analysis, spatial reference normalization, and event-related segmentation. After preprocessing, a set of multi-dimensional, stable neural features related to driving intention is extracted and output, including: time-frequency power characteristics of key brain regions in different frequency bands (δ, θ, α, β, γ), event-related spectral perturbation characteristics, waveform characteristics of specific evoked potential components, brain functional network connectivity characteristics, and spatial distribution characteristics of brain power activity. These features collectively constitute the neural representation of the driver's real-time cognitive state (i.e., EEG characteristics), providing high-information-density input data for subsequent intention decoding.

[0020] The EEG intention decoding module is connected to the EEG signal acquisition module and is used to extract driving intention features based on the preprocessed EEG signals and output a structured intention vector.

[0021] The EEG intention decoding module is implemented using a deep neural network, including a spatiotemporal convolutional network and an attention mechanism, and is used to identify at least one of takeover intention, risk avoidance intention, and path selection intention.

[0022] Specifically, the deep neural network includes a spatial feature extraction layer, a temporal feature extraction layer, an attention mechanism layer, and an output layer. In this embodiment, the spatial feature extraction layer may use a GCN network (Graph Convolutional Network) to model the spatial topological relationships between electrodes in the EEG signal acquisition module and extract brain region coordination patterns; the temporal feature extraction layer may use a bidirectional LSTM network to capture the dynamic temporal dependence of intention formation. The attention mechanism layer employs a multi-head self-attention layer to focus on the most important time points and EEG features for the current driving task.

[0023] The output layer may contain two parallel fully connected layers. One fully connected layer is followed by a Softmax activation function, which outputs the probability distribution of the basic driving intention. The other fully connected layer is followed by a Sigmoid activation function, which outputs the overall intention confidence of the current decoding.

[0024] The probability distribution of the basic driving intention may include: an intention probability distribution vector consisting of takeover probability, left turn probability, right turn probability, acceleration probability, deceleration probability, lane change probability, and lane holding probability.

[0025] The intent probability distribution vector and the overall intent confidence level together form the structured intent vector that is finally output by the EEG intent decoding module.

[0026] The deep neural network is pre-trained using the following method: The driving history data of multiple testers in diverse scenarios, including urban areas, highways, and sudden dangers, was collected as a training dataset. This driving history data should include synchronously recorded EEG signals, vehicle control inputs (such as steering wheel angle and pedal signals), and scenario labels (such as "vehicle cutting in front").

[0027] Based on vehicle control inputs, truth intention labels are generated using rule-based and threshold methods. For example, when the absolute value of the steering wheel angle continuously exceeds 15 degrees, it is labeled as "left turn" or "right turn"; when the brake pedal pressure exceeds a certain threshold without accompanying steering, it is labeled as "decelerate". During the system-requested takeover period, the takeover intention label is assigned based on whether and when the driver responds.

[0028] The deep neural network is trained in a supervised manner using a training dataset with truth intention labels, and the loss function is a weighted sum of the cross-entropy loss of the classification task and the mean squared error loss of the confidence estimation.

[0029] The machine mental reasoning module is connected to the EEG intention decoding module and is used to fuse the intention vector, vehicle status and environmental perception information to infer at least one mental state of the driver, including cognitive load, attention state, emotional state and trust level, and output a mental state vector.

[0030] The environmental perception information includes traffic participant, road structure, and traffic signal information collected by at least one of the following sensors: cameras, millimeter-wave radar, and lidar. The traffic participant information includes participant location, speed, and participant type (e.g., vehicle / pedestrian / obstacle). The road structure includes lane markings, etc. The traffic signal information includes traffic light status, etc.

[0031] The vehicle status includes vehicle position, speed, acceleration, etc.

[0032] The system is also equipped with a unified timestamp mechanism for time synchronization of EEG data streams, vehicle status data streams, and environmental perception data streams.

[0033] The machine mental reasoning module constructs a dynamic heterogeneous graph model based on graph neural networks, and integrates multi-source data through message passing and attention mechanisms to output a quantitative assessment of the driver's mental state.

[0034] Specifically, in the dynamic heterogeneous graph model built on graph neural networks, the node types of its graph structure include: The driver node has the attribute of a structured intent vector output by the EEG intent decoding module.

[0035] The vehicle node has the attribute of vehicle status.

[0036] Key traffic participant nodes, whose attributes are perceived traffic participant information.

[0037] Key road element nodes, whose attributes are road structure and traffic signal information.

[0038] The edge types in its graph structure are defined based on rich semantic relationships, mainly including: directed edges representing the driver's active cognition and intention projection, such as attention, intention exertion, and expectation; affected edges representing the influence of the environment on the driver; physical interaction edges describing the spatial proximity and interactive conflicts between the vehicle and traffic participants and road elements; and environmental internal relationship edges characterizing the mutual influence of traffic participants and the topological connections of road elements. These edges with clear semantics enable the graph neural network to simulate the flow and integration of information in the coupled system of "physical scene-driver's mind" through message passing and attention mechanisms, thereby inferring a quantitative assessment of the driver's cognitive load, risk perception, and intention stability.

[0039] Furthermore, the fusion of multi-source data through message passing and attention mechanisms refers to: A relational graph attention network is employed. At each time step, the graph structure is updated based on the latest perception results, and then the relational graph attention network performs multi-round message passing. The driver node updates its representation by aggregating information from its own vehicle, key traffic participants, etc.

[0040] Then, from the updated driver node representation, specific mental state dimensions are decoded to form a dynamic profile of the driver's cognitive ability, which includes: cognitive load, attention distribution, hazard perception level (i.e., the driver's perception of key risks (such as vehicles cutting in), and trust level. The attention distribution refers to the probability distribution of the driver's attention in areas such as [the road ahead, the left rearview mirror, the right rearview mirror, and the cabin].

[0041] The dynamic heterogeneous graph model is pre-trained using the following method: Driving history data from multiple testers in diverse scenarios, including urban areas, highways, and sudden hazards, was collected as a training dataset. This driving history data needed to include synchronously recorded EEG signals, vehicle control inputs (such as steering wheel angle and pedal signals), and scenario labels (such as "vehicle cutting in front"). Furthermore, ground truth values ​​for mental states were annotated or calculated. For example: the ground truth for cognitive load could be obtained through the driver's subjective load rating (such as the NASA-TLX scale); the ground truth for attention distribution was obtained by mapping the gaze coordinates recorded by eye trackers to different regions of interest; the ground truth for hazard perception level was obtained by asking the driver, "Did you notice the XX hazard at the time?" while playing back the test video, and then performing binary annotation based on their answers; the ground truth for trust level could be obtained by designing a trust questionnaire.

[0042] The dynamic heterogeneous graph model is trained using a multi-task loss function with truth labels for mental states.

[0043] The end-to-end autonomous driving control module is used to perform multimodal fusion of the intent vector, the mental state vector, environmental perception features and vehicle dynamics state to generate continuous vehicle control commands.

[0044] Specifically, the end-to-end autonomous driving control module adopts a policy network structure, and uses a composite reward function to evaluate safety, comfort, and human-machine interaction. Figure 1 Consistency is optimized end-to-end, outputting continuous control quantities for throttle, braking, and steering. .

[0045] In this embodiment, the intent vector, mental state vector, environmental perception features, and vehicle dynamics state are jointly encoded into a unified state representation. An optimal control strategy is then learned through a policy network within a given cognitive context. This policy network is designed to incorporate safety, comfort, and human-machine interaction considerations. Figure 1 A consistent composite reward function is optimized end-to-end to ultimately output continuous and smooth throttle, braking, and steering control values, achieving human-like autonomous driving behavior that conforms to traffic rules and adapts to driver cognitive characteristics. Furthermore, test cases are run through a unified test interface to collect system operation logs, control outputs, and driver behavior data, supporting data playback, performance analysis, and safety verification, providing a basis for system optimization.

[0046] The composite reward function is set according to the following formula: .

[0047] In the formula, These are the weighting coefficients.

[0048] As a safety reward, it is based on the real-time collision time between the vehicle and the obstacle. Calculations are performed to account for values ​​below the safety threshold. Punishment is imposed in the following circumstances, namely .in, Based on the current relative speed and distance, the smaller the value, the higher the risk of collision. This is the penalty coefficient; the larger the value, the stronger the system's willingness to avoid dangerous behavior.

[0049] The danger threshold (e.g., 2.0 seconds); when If the value is below this threshold, a severe penalty is triggered. I is an indicator function, which is 1 when the condition is met and 0 otherwise.

[0050] For comfort rewards, the penalty is increased speed. . Jerk (the derivative of acceleration) is a direct indicator of impact. By penalizing drastic control actions, this parameter guides the system to output smooth throttle, brake, and steering commands, preventing passengers from experiencing jerking or dizziness.

[0051] People with cunning Figure 1 Consistent rewards, including controlled actions With decoding intent Deviation punishment And control fluctuation penalty weighted by cognitive load .

[0052] in, This is the control action vector (such as throttle, brake, steering angle) output by the policy network at time step t.

[0053] This is the driver's desired action vector decoded from the intent vector. For example, the probability of changing lanes or turning left in the intent vector can be mapped to a desired steering angle.

[0054] To control the penalty coefficient for deviations between actions and intentions, this item directly drives "human-machine collaboration".

[0055] This refers to the real-time cognitive load value extracted from the mental state vector (normalized to [0,1]).

[0056] The variance of the control action within a short time window represents the "hesitation" or "fluctuation" of the control.

[0057] This is the cognitive load amplification factor. When the driver's cognitive load is high, the system control should be more stable and decisive; therefore, the penalty for control variability (Var) will be proportional to the load. This amplification enables the system to provide smoother, more dominant control when the driver is "overloaded".

[0058] Specifically, the available control strategies are shown in Table 1, including: (1) Autonomous driving-led control: when or or When this occurs, the control strategy is triggered—the autonomous driving system takes the lead in controlling the vehicle, limiting the maximum speed, increasing the following distance, and the system takes the lead in avoiding hazards. This is the final continuous control output. And limit the speed of the vehicles involved (e.g., by multiplying by 0.7).

[0059] in, It is a policy network that outputs continuous control quantities based on real-time state representation.

[0060] (2) Cooperative control: when and and When this occurs, the control strategy is triggered—the autonomous driving system and the driver share control, and the final control command is a weighted fusion of the inputs from the autonomous driving system and the driver. That is, the final output is a continuous control quantity. . For collaborative weights, follow Increase as it rises. Input for the driver's current control operation.

[0061] (3) Individual-led control: when and and When this occurs, the control strategy is triggered—if the driver has no intention to actively intervene and is in a relaxed state, the intensity of the autonomous driving control is smoothly relaxed to provide auxiliary control. This results in the final continuous control output. and gradually reduce To smoothly relax the control intensity of autonomous driving.

[0062] In this embodiment, The strength of the takeover intent can be directly derived from the takeover probability component in the probability vector output by the EEG intent decoding module.

[0063] Cognitive load level refers to the cognitive load component in the mental state output by the machine's mental reasoning module.

[0064] The environmental risk level, derived from environmental perception information, can be obtained in this embodiment by weighting environmental information such as traffic density, road complexity, and weather visibility using a rule-based multi-feature weighted model. Its value ranges from 0 to 1. In practical applications, this factor may not be considered separately when selecting a control strategy.

[0065] Table 1 Examples of End-to-End Control Strategies

[0066] The online learning and adaptive optimization module is used to dynamically update the parameters of the EEG intention decoding module or the machine mind reasoning module based on the driver's actual takeover behavior and operational feedback.

[0067] Preferably, the system is configured with a mind-intention collaborative decision-making mechanism. When the driver's intention confidence is detected to be higher than a first threshold and the driver's mental state is unstable, it can be determined that the driver wants to take over but lacks the ability. In this case, the weight of autonomous driving control is enhanced, including increasing the proportion of autonomous driving control commands in the fused control, limiting the vehicle's maximum speed, and increasing the steering system damping. When the intention is stable (intention confidence is lower than the first threshold and remains stable) and the cognitive load is lower than a second threshold, it can be determined that the driver has no intention to actively intervene and is in a relaxed state. In this case, the intensity of autonomous driving control is gradually relaxed (e.g., smoothly relaxing the constraints on the vehicle's lateral and longitudinal control, allowing for greater flexibility in the vehicle trajectory to improve comfort). In practical applications, the first and second thresholds can be calibrated according to the actual vehicle conditions. In this embodiment, they can be set to 0.7 and 0.3, respectively.

[0068] In specific applications, such as Figure 2 As shown, the complete workflow of this system includes the following steps: S1, Device startup and signal initialization.

[0069] Test personnel entered the vehicle equipped with autonomous driving capabilities and reset and initialized the key sub-modules of the system. A wearable EEG acquisition device initiated signal acquisition and established a communication connection with the vehicle via Bluetooth or WiFi. The system automatically detected the quality of the EEG signals, performing baseline drift correction, power frequency noise suppression, and preliminary removal of motion artifacts. Simultaneously, the vehicle environmental perception system and vehicle status acquisition module were activated, monitoring vehicle speed, steering wheel angle, longitudinal acceleration, and other vehicle operating states in real time. The system employs a unified timestamp mechanism to synchronize the EEG data stream, vehicle status data stream, and environmental perception data stream, ensuring consistency of multi-source information in subsequent intention reasoning and control decisions.

[0070] S2, global configuration and driver cognitive baseline establishment.

[0071] The system inputs driver identity information and collects baseline EEG data of the driver at rest or under low load to establish an individualized neurocognitive baseline model. It configures the current driving task type, including autonomous or manual driving, and sets corresponding risk assessment parameters (specifically referring to those involved in control strategy judgment). , , (Parameters, etc.). During the system parameter configuration phase, key feature parameters for intent modeling and mental reasoning are set, including EEG frequency band energy change indicators, cognitive load change trend indicators, and intent confidence thresholds. The process proceeds only if the system status is detected as normal; if a device or communication anomaly is detected, the tester is notified via a prompt module, and the current process is terminated.

[0072] S3, Intent Modeling and Mental Reasoning Parameter Configuration.

[0073] After completing the global configuration, the system loads the EEG intention decoding module and the machine mind reasoning module, outputting the driver's current driving intention and its confidence level, and generating the corresponding mental state. Based on the above output results, the system configures the key parameters for end-to-end autonomous driving control. Proceeding to the next step is only permitted when the system is running normally.

[0074] S4, end-to-end autonomous driving control execution.

[0075] Testers initiate the test script, and the system updates the EEG intent, mental state, environmental perception information, and vehicle status at set intervals (e.g., 200ms). Based on these inputs, the end-to-end autonomous driving control module dynamically generates vehicle control commands. During control execution, when the system detects an increase in the driver's confidence in taking over but an unstable mental state, it automatically increases the autonomous driving control weights; when the driver's intent is stable and the cognitive load is within a controllable range, the system allows for gradual adjustments to the human-machine collaborative control ratio. The system synchronously records changes in intent, evolution of mental state, and control output until the vehicle completes the predetermined driving objective or reaches the maximum duration of the test case.

[0076] S5, Test End and Data Analysis After the test, the system resets the vehicle's status, stops data acquisition, and saves the EEG data, intent reasoning results, control commands, and vehicle CAN bus data. Test personnel generate a test report based on the recorded data, including assessments of intent recognition stability, mental state change trends, and the smoothness of autonomous driving control. System parameters are then adjusted and tests are repeated as needed.

[0077] To more specifically illustrate the workflow and effects of the method of the present invention in actual driving scenarios, two typical application cases are provided below.

[0078] Application Case 1: Driving control in unguided left turn scenarios.

[0079] This application case simulates a common "unguided left turn" scenario in urban roads, where there are no dedicated left-turn arrow lights or road markings at intersections, and vehicles must find a gap in oncoming traffic to complete the left turn. The specific implementation steps are as follows: (1) System startup and operating condition configuration Test staff followed Figure 2The system architecture shown demonstrates the reset and initialization configuration of the vehicle, EEG acquisition device, cockpit-driver fusion domain controller, and host computer system. The vehicle under test is placed in an approved unguided left-turn test scenario, which is an urban road intersection without dedicated left-turn signals or ground guide lines. The wearable EEG acquisition device initiates signal acquisition and transmits the driver's EEG characteristic data to the cockpit-driver fusion domain controller via the CAN bus at 20 ms intervals, enabling the system to perceive that the vehicle is currently in an unguided left-turn level road condition.

[0080] (2) Establishment of driving status and perception of intent To trigger the unguided left-turn decision-making process, testers guided the driver to operate the accelerator pedal, accelerating the vehicle to 30 km / h in autonomous driving mode (as specified in the test case). The vehicle then began to decelerate to 5–6 km / h within 5–10 m before the stop line, and made a lateral offset of 0.5–1.0 m to approach the edge of the left lane, reserving space for subsequent turning. During this phase, the EEG intention decoding module continuously analyzed the driver's EEG signals, outputting the intention vector and corresponding confidence level; the machine mental reasoning module simultaneously assessed the driver's mental state.

[0081] (3) End-to-end control decision and left turn execution Testers set up different traffic scenarios based on test cases to verify the system's control and decision-making capabilities under human-machine collaboration conditions. Typical scenarios and corresponding strategies are shown in Table 2.

[0082] Table 2 Examples of cooperative control strategies in unguided left turn scenarios

[0083] When the system detects a decrease in oncoming traffic and a decline in the driver's confidence in risk avoidance and a more stable mental state, the end-to-end autonomous driving control module smoothly executes a left turn without triggering a sudden switch of control.

[0084] (4) Results observation and data recording After the vehicle successfully entered the target lane and gradually accelerated to the speed limit indicated by the road sign (e.g., 40 km / h), the testers maintained a stable speed for approximately 10 seconds and saved the test data via a host computer system. The focus was on observing whether the vehicle could smoothly complete a left turn under changes in driver intent and mental state, and whether it met the expected requirements of road traffic regulations.

[0085] Application Case 2: Driving Control in Extreme U-Turn Scenarios.

[0086] This application case is used to verify the driving control mechanism under the long-tail condition of making an extreme U-turn in a narrow space.

[0087] The specific implementation steps are as follows: (1) System startup and scene settings Test personnel reset all key submodules of the system and placed the vehicle in a pre-approved extreme U-turn test scenario via the host computer system. This scenario involves a low-speed, high-curvature road environment with a limited U-turn radius, demanding high precision in vehicle control and strong human-machine collaboration. EEG acquisition equipment transmits the driver's brainwave characteristics to the controller at 20 ms intervals, enabling the system to perceive that a U-turn is currently in progress.

[0088] (2) Preparation for turning around and status assessment To observe the system's control capabilities under extreme conditions, testers ensured that the number of oncoming lanes met the safety requirements for U-turns. Generally, at least one oncoming lane was required, with at least two lanes preferred to provide a safety margin. The vehicle accelerated to 20 km / h in autonomous driving mode (as specified in the use case) and gradually decelerated to 5–6 km / h before entering the U-turn area. During this process, the EEG intention decoding module detected fluctuations in the driver's intention to take over, while the machine cognitive reasoning module simultaneously assessed the driver's cognitive load level and attention stability.

[0089] (3) Turnaround execution and exception handling Under the premise of meeting traffic rules and road conditions, the system comprehensively considers the gap between oncoming vehicles, the risk of pedestrians crossing, and the obstruction of non-motorized vehicle lanes to determine the feasibility of the U-turn trajectory. When the driver's cognitive load is high or their intention to take over is unstable, the end-to-end autonomous driving control module increases the weight of autonomous driving control and improves lane-keeping sensitivity. If a sudden change in environment or an infeasible trajectory occurs during the U-turn, the system can choose to wait for an opportunity to continue execution midway through the U-turn or directly abort the operation and safely stop the vehicle. When the system downgrades from L3 to L2 mode, it prompts the driver "Please take over immediately" through the HUD interface.

[0090] (4) Results evaluation and report generation After the test, the testers generated a test report based on the data saved in the host computer system. They analyzed the trajectory simulation to determine if the vehicle was at risk of crossing the boundary or colliding, and verified this information using the minimum turning radius formula. (L: wheelbase, : Maximum steering angle, d: safety margin).

[0091] This embodiment provides an end-to-end autonomous driving system based on EEG intention modeling and machine mental reasoning, which can transform the driver's neurocognitive activities into decision elements that can be understood and utilized by the machine in real time, thereby achieving human-machine cognitive alignment and closed-loop collaborative control.

[0092] The above descriptions are merely embodiments of the present invention. Commonly known structures and characteristics of the solutions are not described in detail here. Those skilled in the art are aware of all common technical knowledge in the field prior to the application date or priority date, are aware of all existing technologies in that field, and have the ability to apply conventional experimental methods prior to that date. Those skilled in the art can, under the guidance of this application, improve and implement this solution in combination with their own capabilities. Some typical known structures or methods should not be obstacles for those skilled in the art to implement this application. It should be noted that those skilled in the art can make several modifications and improvements without departing from the structure of the present invention. These should also be considered within the scope of protection of the present invention, and will not affect the effectiveness of the implementation of the present invention or the practicality of the patent.

Claims

1. An end-to-end autonomous driving system based on electroencephalography intention modeling and machine mind reasoning, characterized in that, The system comprises: a brain electrical signal acquisition module for real-time acquisition and preprocessing of the brain electrical signals of the driver; a brain electrical signal intention decoding module connected to the brain electrical signal acquisition module for extracting driving intention features based on the preprocessed brain electrical signals and outputting a structured intention vector; a machine mind reasoning module connected to the brain electrical signal intention decoding module for fusing the intention vector, vehicle state and environmental perception information, inferring at least one of the cognitive load, attention state, emotional state and trust level of the driver, and outputting a mind state vector; an end-to-end automatic driving control module for multi-modal fusion of the intention vector, the mind state vector, environmental perception features and vehicle dynamics state to generate continuous vehicle control instructions; The system realizes closed-loop cooperative control from driver neural intention perception to vehicle control output.

2. The end-to-end autonomous driving system based on electroencephalography intention modeling and machine mind reasoning according to claim 1, wherein, The brain electrical signal intention decoding module is implemented using a deep neural network, including a spatio-temporal convolution network and an attention mechanism, for identifying at least one of the takeover intention, risk avoidance intention and path selection intention.

3. The end-to-end autonomous driving system based on electroencephalography intention modeling and machine mind reasoning according to claim 2, characterized in that, The machine mind reasoning module constructs a dynamic heterogeneous graph model based on a graph neural network, fuses multi-source data through message passing and attention mechanism, and outputs a quantitative evaluation of the driver's mind state.

4. The end-to-end autonomous driving system based on electroencephalography intention modeling and machine mind reasoning according to claim 1, wherein, The end-to-end automatic driving control module uses a policy network structure to perform end-to-end optimization of safety, comfort and consistency of human-machine intention through a composite reward function, and outputs continuous control quantities of throttle, brake and steering.

5. The end-to-end autonomous driving system based on electroencephalography intention modeling and machine mind reasoning according to claim 1, wherein, It also includes an online learning and adaptive optimization module for dynamically updating the parameters of the brain electrical signal intention decoding module or the machine mind reasoning module based on the actual takeover behavior of the driver and operation feedback.

6. The end-to-end autonomous driving system based on electroencephalography intention modeling and machine mind reasoning according to claim 1, characterized in that, The system is configured with a mind-intention cooperative decision mechanism that enhances the weight of automatic driving control when the driver's intention confidence is higher than a first threshold and the mind state is unstable; when the intention is stable and the cognitive load is lower than a second threshold, gradually relax the intensity of automatic driving control.

7. The end-to-end autonomous driving system based on electroencephalography intention modeling and machine mind reasoning according to claim 1, wherein, The brain electrical signal acquisition module includes a wearable EEG cap that supports multi-channel signal synchronous acquisition and is connected to the vehicle controller through wireless communication. 8.The end-to-end autonomous driving system based on electroencephalic intention modeling and machine mind reasoning of claim 1, wherein, The system is also configured with a unified timestamp mechanism for time synchronization of brain electrical data stream, vehicle state data stream and environmental perception data stream. 9.The end-to-end autonomous driving system based on electroencephalic intention modeling and machine mind reasoning of claim 1, wherein, The environmental perception information includes traffic participant, road structure and traffic signal information collected by at least one sensor among car-road cooperative equipment, camera, millimeter wave radar and laser radar.

10. The end-to-end autonomous driving system based on electroencephalography intention modeling and machine mind reasoning according to claim 1, wherein, The system supports L2 or L3 automatic driving scenarios and is suitable for at least one of the following working conditions: highway, urban road, no guidance left turn and extreme U-turn.

Images