Intelligent driving method and system based on biofeedback of driver

Abstract

The invention relates to the technical field of intelligent driving, and provides an intelligent driving method and system based on driver biofeedback. The method comprises the following steps: firstly, acquiring multi-mode biological signals such as heart rate, electrodermal response and eye movement track of a driver and a vehicle movement signal in real time through an in-vehicle sensor; carrying out cross-modal feature fusion and analysis on the signals by utilizing a driver state analysis model, carrying out parallel calculation on tensity, motion sickness and attention level indexes of the driver, and generating a quantized comfort level score; furthermore, in a trajectory planning stage, a comfort level score corresponding to each candidate trajectory is predicted through a personalized comfort level prediction model, the comfort level score is used as an optimization target parallel to safety and efficiency for comprehensive decision making, and finally an optimal trajectory is selected for execution. According to the invention, the automatic driving system can sense and respond to the subjective state of the driver in real time, the tension and motion sickness of passengers can be effectively relieved, and the riding comfort and personalized experience are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent driving, and more particularly, to an intelligent driving driving method and system based on driver biological feedback. BACKGROUND

[0002] With the development of automatic driving technology, the decision planning system of the vehicle mainly makes logical judgments and behavior generation based on objective constraints such as safety, efficiency and traffic regulations. For example, in path planning, the system usually takes the shortest path or the shortest travel time as the goal; in the specific behavior decision of following, changing lanes, overtaking, etc., a preset and relatively conservative strategy is generally adopted to primarily guarantee driving safety. This "vehicle-centered" planning mode centered on machine logic and relying on fixed rules has become the mainstream technical paradigm in the current industry.

[0003] However, the above technical route has defects. First, it ignores the subjective feelings of the driver or passenger, a key factor. The same automatic driving behavior, such as more aggressive lane cut-in, higher speed through a curve, or emergency braking, will trigger completely different physiological and psychological reactions for passengers of different characteristics. Some users may think it is efficient, but others will feel nervous, uncomfortable or even car sickness, and the existing system cannot effectively perceive and quantify this. Second, the system lacks individualized adaptation ability and cannot distinguish the different preferences of experienced drivers and novice drivers, nor can it adapt to the state differences of the same passenger at different times, such as when tired and when mentally alert, resulting in a rigid and single automatic driving experience that is difficult to gain the trust and acceptance of users. In addition, some existing improvement schemes aimed at improving comfort, such as setting a fixed "comfort mode", are static adjustments that cannot dynamically optimize according to the real-time physiological state of the passenger, and the adjustment effect is limited, which is a kind of blind adjustment.

[0004] Therefore, there is an urgent need in the art for an intelligent driving scheme that can accurately perceive the state of the passenger in real time and feed back this state as the core decision basis to the planning system, thereby ultimately realizing truly personalized, humanized and dynamically optimized intelligent driving. SUMMARY

[0005] The present application provides an intelligent driving driving method and system based on driver biological feedback, which takes the real-time state of the driver as the core basis for the decision of the intelligent driving system, improving the individualization level of intelligent driving control and the comfort of automatic driving.

[0006] According to a first aspect of the present application, an intelligent driving driving method based on driver biological feedback is provided, comprising: S1, through a preset driver state analysis model, analyzes the real-time collected vehicle motion signals and the driver's multimodal biological signals to obtain a comfort score representing the driver's subjective feelings. S2, based on the current environmental perception results, plan multiple candidate trajectories, and based on the mapping relationship between vehicle motion state parameters and comfort scores in the personalized comfort prediction model, calculate the comfort prediction value corresponding to each candidate trajectory; S3, using the comfort prediction value to participate in the path cost estimation, and deciding the optimal trajectory from the multiple candidate trajectories.

[0007] Based on the above technical solution, the present invention can also be improved as follows.

[0008] Optionally, step S1 may further include preprocessing the vehicle motion signal and the driver's multimodal biosignals, wherein the multimodal biosignals include at least physiological signals, head motion signals and eye movement trajectory signals. The preprocessing includes: The collected vehicle motion signals were smoothed using Kalman filtering, and the collected multimodal biological signals were processed using the 3σ criterion to remove outliers. After removing outliers, the vehicle motion signals and multimodal biological signals were normalized, data augmented, and time-aligned.

[0009] Optionally, the driver state analysis model includes a multi-branch feature extraction network, a cross-modal feature fusion layer, and a multi-task regression output layer; step S1 includes: S101, vehicle motion signals and multimodal biological signals are input into a parallel multi-branch feature extraction network to extract modality-specific features and obtain the temporal features of each modality; S102, through the cross-modal feature fusion layer, calculates the correlation weight between temporal features of different modalities based on the multi-head self-attention mechanism, and weights and fuses the temporal features of each modality to generate a cross-modal fusion feature vector; S103, input the cross-modal fusion feature vector into the multi-task regression output layer, and calculate the driver's stress index, motion sickness index and attention level index in parallel. S104. Based on the tension index, motion sickness index and attention level index, a comfort score is generated by weighted fusion.

[0010] Optionally, step S103 includes: The cross-modal fusion feature vector is input into a multi-task regression output layer, which includes three parallel network branches corresponding to the calculation of stress index, motion sickness index, and attention level index, respectively; wherein: The network branch that calculates the stress index is configured to output weights that depend on the feature dimensions related to heart rate and skin conductance in the cross-modal fusion feature vector; The network branch that calculates motion sickness indices is configured to have its inputs fused with a coupling coefficient determined by the temporal correlation between head motion signals and vehicle motion signals; The network branch that calculates the attention level index is configured to have an attention weight matrix fused to its input, which is generated based on the key regions of the driver's field of vision where the eye-tracking gaze point is located. The three parallel network branches are configured to share the cross-modal fusion feature vector as the basic input and are trained collaboratively using a joint loss function to simultaneously output physiologically related stress, motion sickness and attention levels.

[0011] Optionally, in step S2, planning multiple candidate trajectories based on the current environmental perception results includes: S201, based on the current environmental perception results, generates multiple feasible candidate trajectories with differences in speed, acceleration and / or path curvature, under the premise of meeting preset safety conditions, traffic regulations and vehicle dynamics constraints.

[0012] Optionally, in step S2, the calculation of the predicted comfort value for each candidate trajectory based on the mapping relationship between vehicle motion state parameters and comfort scores in the personalized comfort prediction model includes: S202, calculate the predicted vehicle motion state parameters corresponding to each candidate trajectory, the predicted vehicle motion state parameters include: acceleration, jerkiness, lateral acceleration, yaw rate, curvature and rate of change of curvature; S203, input the predicted vehicle motion state parameters corresponding to the current candidate trajectory into the personalized comfort prediction model, output the comfort prediction sequence of the current candidate trajectory execution process according to the mapping relationship between the vehicle motion state parameters and the comfort score, and use the weighted average or minimum value of the comfort prediction sequence as the comfort prediction value of the current candidate trajectory. S204, loop through S203, and traverse all candidate trajectories.

[0013] Optionally, step S3 includes: S301, Construct a multi-objective decision function, which is defined as a weighted sum of safety cost, efficiency cost, and comfort cost; S302, for each candidate trajectory, its comfort prediction value is mapped to a comfort cost, and its safety cost and efficiency cost are calculated at the same time; S303, Substitute the safety cost, efficiency cost and comfort cost into the multi-objective decision function to calculate the total decision cost for each candidate trajectory; S304: Select the trajectory with the minimum total decision cost among all candidate trajectories as the optimal trajectory.

[0014] Optionally, the method also includes: The actual comfort score is calculated based on the vehicle operation signals collected when the vehicle executes the optimal trajectory and the multimodal biological signals. The actual comfort score is compared with the predicted comfort value of the current trajectory, and the personalized comfort prediction model is adaptively optimized based on the comparison result.

[0015] According to a second aspect of the present invention, an intelligent driving system based on driver biofeedback is provided, comprising: The acquisition module is configured to acquire multimodal biosignals of the driver, vehicle motion signals, and current environmental information around the vehicle in real time. The analysis module, which is communicatively connected to the acquisition module, is configured to analyze the multimodal biosignals and vehicle motion signals through a preset driver state analysis model to obtain a comfort score representing the driver's subjective feelings. The decision-making module, which is communicatively connected to the acquisition module and the analysis module, is configured to: plan multiple candidate trajectories based on the current environmental information; calculate the comfort prediction value corresponding to each candidate trajectory based on the mapping relationship between vehicle motion state parameters and comfort scores in the personalized comfort prediction model; and use the comfort prediction value to participate in path cost estimation to determine the optimal trajectory from the multiple candidate trajectories. The execution module, which is communicatively connected to the decision module, is configured to control the vehicle to travel along the optimal trajectory.

[0016] Optionally, the system may also include: The update module, which communicates with the analysis module and the decision module, is configured as follows: The analysis module calculates the actual comfort score when the vehicle executes the optimal trajectory. The actual comfort score is compared with the corresponding comfort prediction value used by the decision module when making the decision, and a difference signal is generated; Based on the difference signal, the personalized comfort prediction model is adaptively fine-tuned.

[0017] According to a third aspect of the present invention, an electronic device is provided, including a memory and a processor, wherein the processor is configured to implement the steps of the above-described intelligent driving method based on driver biofeedback when executing a computer management program stored in the memory.

[0018] According to a fourth aspect of the present invention, a computer-readable storage medium is provided having a computer management program stored thereon, which, when executed by a processor, implements the steps of the above-described intelligent driving method based on driver biofeedback.

[0019] This invention provides an intelligent driving method, system, electronic device, and storage medium based on driver biofeedback. It fuses real-time collected multimodal biosignals with vehicle motion signals through a driver state analysis model to directly quantify the driver's subjective comfort. Furthermore, it utilizes a personalized comfort prediction model to predict the potential impact of different planned trajectories on driver comfort. Finally, at the decision-making level, the predicted comfort value is used as a core cost function alongside safety and efficiency to optimize trajectory selection for intelligent driving. This invention transforms autonomous driving systems from relying solely on objective rules into collaborative decision-making systems that proactively perceive and adapt to the driver's subjective state, thereby effectively enhancing the personalized comfort experience of passengers while ensuring safety. Attached Figure Description

[0020] Figure 1 A flowchart of an intelligent driving method based on driver biofeedback is provided for one embodiment; Figure 2 A flowchart of an intelligent driving method based on driver biofeedback is provided as another embodiment; Figure 3 A block diagram of an intelligent driving system based on driver biofeedback provided for one embodiment; Figure 4 A block diagram of an intelligent driving system based on driver biofeedback is provided for yet another embodiment; Figure 5 A schematic diagram of the hardware structure of a possible electronic device provided by the present invention; Figure 6 This is a schematic diagram of the hardware structure of a possible computer-readable storage medium provided by the present invention. Detailed Implementation

[0021] The specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples. The following examples are for illustrative purposes only and are not intended to limit the scope of the invention.

[0022] In this embodiment of the invention, when collecting, processing, and storing user personal information (such as images, behavioral characteristics, etc.), the implementation of the technical solution strictly adheres to the principles of legality, legitimacy, and necessity, as well as the core rule of "notification-consent." Specifically, before information collection, the system clearly informs the user of the purpose, method, scope, and usage rules of information collection through an interactive interface, and requires the user's active authorization and consent. The entire information processing process employs data encryption, access control, and other technical measures to ensure information security, and establishes mechanisms to facilitate users' exercise of their rights (such as querying, correcting, withdrawing consent, and deleting information). For exceptions stipulated by law (such as those necessary for fulfilling statutory duties or responding to public health emergencies), their application is strictly limited to the scope and limits authorized by law, ensuring that the technical solution does not contain any content that violates the law, social morality, or harms the public interest.

[0023] Figure 1 A flowchart of an intelligent driving method based on driver biofeedback provided by this invention is shown below. Figure 1 As shown, the method includes steps S1 to S3: S1 analyzes real-time vehicle motion signals and driver multimodal biosignals using a preset driver state analysis model to obtain a comfort score representing the driver's subjective feelings.

[0024] This step establishes a sensory model capable of real-time perception of the driver's internal state by fusing vehicle motion signals (cause) and driver biosignals (effect). It transforms the driver's subjective and vague physiological and behavioral responses into an objective and quantifiable comfort score through a driver state analysis model. This step addresses the blind spot of existing technologies in recognizing driver emotions, enabling autonomous driving systems to expand their decision-making from purely focusing on the external environment to simultaneously considering the internal occupant state.

[0025] S2: Based on the current environmental perception results, plan multiple candidate trajectories. Based on the mapping relationship between vehicle motion state parameters and comfort scores in the personalized comfort prediction model, calculate the comfort prediction value corresponding to each candidate trajectory.

[0026] This step analyzes the vehicle motion state (such as acceleration and curvature) that each candidate trajectory will result in, predicting the potential comfort impact on the driver. By transforming comfort assessment from a reactive response to a proactive prediction, the system can pre-screen and avoid aggressive or uncomfortable trajectories that may cause driver stress or motion sickness before making a decision, thus achieving predictability in planning.

[0027] S3, using the comfort prediction value to participate in the path cost estimation, and deciding the optimal trajectory from the multiple candidate trajectories.

[0028] This step incorporates comfort as a decision factor of equal importance to safety and efficiency, embedding it into the cost function of path selection for multi-objective optimization. This step does not consider comfort in isolation, but rather seeks the optimal balance between safety, efficiency, and comfort while ensuring safety and compliance. This step enables personalized trajectory decision-making, generating the most suitable driving style for drivers with different preferences (such as those seeking smoothness or efficiency), thus making autonomous driving behavior closer to the personalized choices of human drivers, improving the riding experience and acceptance.

[0029] Understandably, given the shortcomings in the background technology, this invention proposes an intelligent driving method based on driver biofeedback. This method fuses and analyzes the driver's real-time biosignals with vehicle motion signals, transforming subjective comfort into quantifiable decision parameters. This allows the autonomous driving system to predict and optimize the impact of different driving behaviors on passenger comfort during the planning phase. Thus, while ensuring safety and efficiency, it enables personalized trajectory selection, effectively alleviating passenger anxiety and motion sickness, and improving the naturalness and acceptability of the driving experience.

[0030] Based on the above technical solutions, the embodiments of the present invention can be further improved as follows.

[0031] In one possible embodiment, step S1 further includes: real-time acquisition and preprocessing of vehicle motion signals and driver's multimodal biosignals, wherein the multimodal biosignals include at least physiological signals, head motion signals and eye movement trajectory signals.

[0032] Specifically, physiological signals include: heart rate, heart rate variability, skin conductance response, and respiratory rate; Behavioral cues include: facial expressions (especially eye and mouth muscles), head posture, and body posture; Visual signals include: eye movement trajectory and changes in pupil diameter.

[0033] For example, vehicle motion signals are acquired via the CAN bus to obtain real-time vehicle speed, longitudinal / lateral acceleration, and yaw rate, while millimeter-wave radar monitors the relative motion of surrounding vehicles; driver biosignals are measured via the capacitive steering wheel to measure heart rate and skin conductance, captured by the DMS camera to capture facial expressions, eye opening and closing, and head posture, and tracked by infrared sensors to track eye movement trajectories and pupil diameter changes, thereby achieving synchronous real-time acquisition of multimodal data.

[0034] The acquired signals are preprocessed, including: The collected vehicle motion signals were smoothed using Kalman filtering, and the collected multimodal biological signals were processed using the 3σ criterion to remove outliers. After removing outliers, the vehicle motion signals and multimodal biological signals were normalized, data augmented, and time-aligned.

[0035] For example, firstly, Kalman filtering is used to smooth motion signals such as vehicle acceleration collected from the CAN bus to eliminate measurement noise; simultaneously, the 3σ criterion is used to remove instantaneous outliers caused by accidental limb movements from physiological signals such as heart rate collected from the capacitive steering wheel; then, Z-score normalization is performed on all cleaned time-series signals to eliminate dimensional differences, and Gaussian noise is added for data enhancement; finally, interpolation algorithms are used to uniformly align signals with different sampling rates (such as high-frequency IMU data and mid-frequency heart rate data) to the same timestamp, providing a regular input for subsequent model analysis.

[0036] In one possible embodiment, the driver state analysis model includes a multi-branch feature extraction network, a cross-modal feature fusion layer, and a multi-task regression output layer; step S1 includes sub-steps S101~S104: S101, vehicle motion signals and multimodal biological signals are input into a parallel multi-branch feature extraction network to extract modality-specific features and obtain temporal features of each modality.

[0037] This step processes the input modal signals in parallel. For example, the physiological signal branch inputs the heart rate time series (dimension 1xT) and skin conductance response (dimension 1xT) into independent 1D convolutional layers to extract their local temporal patterns and output feature maps (dimension C1xT1). The head motion branch takes the 3-axis angular velocity / accelerometer sequence (dimension 3xT) acquired by the IMU and performs 1D convolution and pooling operations to obtain a dimensionality-reduced feature representation (dimension C2xT2). The eye-tracking trajectory branch and vehicle motion branch also use similar operations, processing the 5-dimensional eye-tracking sequence to obtain an eye-tracking trajectory-related feature representation (dimension C3×T3) and processing the 3-dimensional vehicle motion sequence to obtain a vehicle motion-related feature representation (dimension C4×T4), thereby completing the specific feature extraction for each modality.

[0038] S102, through the cross-modal feature fusion layer, calculates the correlation weights between temporal features of different modalities based on the multi-head self-attention mechanism, and weights and fuses the temporal features of each modality to generate a cross-modal fused feature vector.

[0039] In this step, the cross-modal feature fusion layer first interpolates the feature maps output from each branch to unify them to the same temporal length T0, achieving temporal alignment. Subsequently, the fusion layer introduces a multi-head self-attention mechanism to calculate the correlation weights between features from different modalities. A key operation is calculating the Pearson correlation coefficient between head motion features and vehicle motion features, serving as a weighting factor to quantify the degree of motion coupling between the two. Finally, the weighted features from all modalities are concatenated to generate a unified fusion feature vector rich in cross-modal information (dimension C_fusion x T0, C_fusion = C1 + C2 + C3 + C4).

[0040] S103, the cross-modal fusion feature vector is input into the multi-task regression output layer to calculate the driver's stress index, motion sickness index and attention level index in parallel.

[0041] Specifically, the multi-task regression output layer includes three parallel fully connected network branches, corresponding to the calculation of the stress index, motion sickness index, and attention level index, respectively; wherein: The network branch that calculates the stress index is configured to output weights that depend on the feature dimensions related to heart rate and skin conductance in the cross-modal fusion feature vector; The network branch that calculates motion sickness indices is configured to have its inputs fused with a coupling coefficient determined by the temporal correlation between head motion signals and vehicle motion signals; The network branch that calculates the attention level index is configured to have an attention weight matrix fused to its input, which is generated based on the key regions of the driver's field of vision where the eye-tracking gaze point is located. The three parallel fully connected network branches are configured to share the cross-modal fusion feature vector as the basic input and are trained collaboratively using a joint loss function to learn different weights. Through the constraints of the joint loss function, they simultaneously output physiologically and logically related stress indicators, motion sickness indicators, and attention level indicators.

[0042] S104. Based on the tension index, motion sickness index and attention level index, a comfort score is generated by weighted fusion.

[0043] This step performs a weighted fusion of the three driver comfort-related state indicators output by S103 to generate a final comprehensive comfort score. For example, learnable weight coefficients are set, and the tension index, motion sickness index, and attention level index are linearly weighted and summed. Then, the sigmoid function is used to map the weighted sum to the interval [0, 1], and this scalar value is the final comfort score used to guide planning decisions.

[0044] Understandably, this embodiment ensures that the uniqueness of each modal signal (such as heart rate and head movement) is fully preserved through multi-branch feature extraction. Then, through cross-modal fusion, key correlations such as the coupling relationship between head movement and vehicle movement are accurately captured. Finally, through multi-task output, different dimensions of state indicators such as tension and motion sickness are simultaneously quantified, thereby generating a comprehensive, accurate and physiologically interpretable comfort score, providing a reliable decision-making basis for subsequent personalized planning.

[0045] In one possible embodiment, step S2, which involves planning multiple candidate trajectories based on the current environmental perception results, includes sub-step S201: S201, based on the current environmental perception results, generates multiple feasible candidate trajectories with differences in speed, acceleration and / or path curvature, under the premise of meeting preset safety conditions, traffic regulations and vehicle dynamics constraints.

[0046] For example, based on environmental perception, the system detects a slow-moving vehicle ahead. Under the premise of ensuring safety (such as no collision risk and compliance with traffic regulations) and meeting vehicle dynamics constraints (such as acceleration and curvature within physical limits), it plans three candidate trajectories: Trajectory A follows the vehicle smoothly with a small acceleration; Trajectory B changes to the adjacent lane with a moderate acceleration; and Trajectory C is planned to decelerate smoothly in advance and maintain the original lane. The three trajectories have significant differences in longitudinal acceleration and lateral curvature, providing diverse and feasible options for subsequent personalized decisions based on comfort.

[0047] In one possible embodiment, step S2, which involves calculating the predicted comfort value for each candidate trajectory based on the mapping relationship between vehicle motion state parameters and comfort scores in the personalized comfort prediction model, includes sub-steps S202-S204: S202, calculate the predicted vehicle motion state parameters corresponding to each candidate trajectory. The predicted vehicle motion state parameters include: acceleration, jerkiness, lateral acceleration, yaw rate, curvature, and rate of change of curvature.

[0048] For example, for a planned candidate trajectory (such as a lane-change trajectory), a dynamic simulation is performed to deduce the state change sequence of the vehicle traveling along the candidate trajectory. Specifically, the longitudinal motion parameters (such as acceleration values ​​and their rate of change "jerk"), lateral motion parameters (such as lateral acceleration and yaw rate), and trajectory geometric parameters (such as path curvature and its rate of change) at each time point on the trajectory are calculated, thus obtaining a complete set of parameter sequences describing the motion characteristics of the trajectory. Through similar calculation methods, the state change sequence of each candidate trajectory is deduced.

[0049] S203, input the predicted vehicle motion state parameters corresponding to the current candidate trajectory into the pre-trained personalized comfort prediction model, output the comfort prediction sequence of the current candidate trajectory execution process according to the mapping relationship between the vehicle motion state parameters and the comfort score, and use the weighted average or minimum value of the comfort prediction sequence as the comfort prediction value of the current candidate trajectory.

[0050] For example, a personalized comfort prediction model maps instantaneous comfort predictions at each moment during the execution of the entire trajectory, forming a comfort prediction sequence. Then, by taking a weighted average of this sequence (to reflect the overall comfort level) or selecting its minimum value (to avoid the most uncomfortable moments), the final, single comfort prediction value for that candidate trajectory is used for subsequent multi-objective decision comparisons.

[0051] S204, loop through S203, and traverse all candidate trajectories.

[0052] It is understood that this embodiment transforms the abstract candidate trajectory into a specific sequence of vehicle motion parameters (such as acceleration and jerk) and uses a personalized model to predict the changes in comfort during the entire trajectory execution process. This achieves a refined and forward-looking assessment of driving comfort, enabling the system to avoid trajectories that may cause continuous discomfort or sudden severe discomfort in advance, thereby ensuring that comfort is quantitatively guaranteed in the planning stage.

[0053] In one possible embodiment, step S3 includes sub-steps S01 to S304: S301, Construct a multi-objective decision function, which is defined as a weighted sum of safety cost, efficiency cost, and comfort cost.

[0054] This step involves constructing a multi-objective decision function, quantifying the three objectives of safety, efficiency, and comfort into calculable cost terms. By assigning weight coefficients to each cost and establishing a weighted summation model, the originally mutually constraining multi-objective optimization problem is transformed into a comparison problem of a single comprehensive cost, providing a mathematical basis for trajectory decision-making.

[0055] S302, for each candidate trajectory, maps its comfort prediction value to a comfort cost, and simultaneously calculates its safety cost and efficiency cost.

[0056] This step quantifies each cost item. For example, the comfort cost is mapped from the comfort score output by the prediction model, reflecting the quantitative integration of subjective feelings; the safety cost is derived by calculating parameters such as the minimum distance between the trajectory and obstacles and the collision time; and the efficiency cost is calculated based on the trajectory's expected travel time or speed maintenance capability, achieving a multi-dimensional performance evaluation of each trajectory.

[0057] S303, substitute the safety cost, efficiency cost and comfort cost into the multi-objective decision function to calculate the total decision cost of each candidate trajectory, thereby unifying the indicators of different dimensions into comparable scalar values.

[0058] S304: Select the trajectory with the minimum total decision cost among all candidate trajectories as the optimal trajectory.

[0059] Understandably, this embodiment, while ensuring a safety baseline, dynamically adjusts the driving strategy based on the real-time calculated comfort cost, realizing a shift from fixed rules to a decision-making process that can intelligently balance safety, efficiency, and comfort, thereby generating a driving trajectory that better meets the driver's personalized comfort needs.

[0060] In one possible embodiment, such as Figure 2 As shown, the method further includes step S4: S4, the actual comfort score is calculated based on the vehicle operation signals and multimodal biological signals collected when the vehicle executes the optimal trajectory; The actual comfort score is compared with the predicted comfort value of the current trajectory, and the personalized comfort prediction model is adaptively optimized based on the comparison result.

[0061] For example, after the system executes an acceleration-lane-change trajectory, it collects the driver's actual heart rate increase data and the vehicle's lateral acceleration during the trajectory. Using an occupant state analysis model, the actual comfort score is calculated to be 0.3 (low), and compared with the comfort prediction value for the planned trajectory (e.g., 0.6) to obtain an error signal. Based on this error signal, the system fine-tunes the weighting parameters of lateral acceleration in the personalized comfort prediction model online, making subsequent comfort predictions for similar aggressive lane-change maneuvers closer to the driver's actual discomfort, thereby achieving continuous personalized optimization of the model.

[0062] Figure 3 A structural diagram of an intelligent driving system based on driver biofeedback provided in an embodiment of the present invention is shown below. Figure 3 As shown, an intelligent driving system based on driver biofeedback includes a data acquisition module, an analysis module, a decision-making module, and an execution module, wherein: The acquisition module is configured to acquire multimodal biosignals of the driver, vehicle motion signals, and current environmental information around the vehicle in real time. The analysis module, which is communicatively connected to the acquisition module, is configured to analyze the multimodal biosignals and vehicle motion signals through a preset driver state analysis model to obtain a comfort score representing the driver's subjective feelings. The decision-making module, which is communicatively connected to the acquisition module and the analysis module, is configured to: plan multiple candidate trajectories based on the current environmental information; calculate the comfort prediction value corresponding to each candidate trajectory based on the mapping relationship between vehicle motion state parameters and comfort scores in the personalized comfort prediction model; and use the comfort prediction value to participate in path cost estimation to determine the optimal trajectory from the multiple candidate trajectories. The execution module, which is communicatively connected to the decision module, is configured to control the vehicle to travel along the optimal trajectory.

[0063] In one possible implementation, such as Figure 4 As shown, the system also includes: The update module, which communicates with the analysis module and the decision module, is configured as follows: The analysis module calculates the actual comfort score when the vehicle executes the optimal trajectory. The actual comfort score is compared with the corresponding comfort prediction value used by the decision module when making the decision, and a difference signal is generated; Based on the difference signal, the personalized comfort prediction model is adaptively fine-tuned so that the prediction output of the personalized comfort prediction model is close to the driver's actual feeling.

[0064] It is understood that the intelligent driving system based on driver biofeedback provided by the present invention corresponds to the intelligent driving method based on driver biofeedback provided in the foregoing embodiments. The relevant technical features of the intelligent driving system based on driver biofeedback can be referred to the relevant technical features of the intelligent driving method based on driver biofeedback, and will not be repeated here.

[0065] Please see Figure 5 , Figure 5 This is a schematic diagram illustrating an embodiment of the electronic device provided in this invention. For example... Figure 5 As shown, this embodiment of the invention provides an electronic device 500, including a memory 510, a processor 520, and a computer program 511 stored in the memory 510 and executable on the processor 520. When the processor 520 executes the computer program 511, it performs the following steps: S1, through a preset driver state analysis model, analyzes the real-time collected vehicle motion signals and the driver's multimodal biological signals to obtain a comfort score representing the driver's subjective feelings. S2, based on the current environmental perception results, plan multiple candidate trajectories, and based on the mapping relationship between vehicle motion state parameters and comfort scores in the personalized comfort prediction model, calculate the comfort prediction value corresponding to each candidate trajectory; S3, using the comfort prediction value to participate in the path cost estimation, and deciding the optimal trajectory from the multiple candidate trajectories.

[0066] Please see Figure 6 , Figure 6 This is a schematic diagram illustrating an embodiment of a computer-readable storage medium provided by the present invention. (See diagram below.) Figure 6 As shown, this embodiment provides a computer-readable storage medium 600, on which a computer program 511 is stored. When the computer program 511 is executed by a processor, it performs the following steps: S1, through a preset driver state analysis model, analyzes the real-time collected vehicle motion signals and the driver's multimodal biological signals to obtain a comfort score representing the driver's subjective feelings. S2, based on the current environmental perception results, plan multiple candidate trajectories, and based on the mapping relationship between vehicle motion state parameters and comfort scores in the personalized comfort prediction model, calculate the comfort prediction value corresponding to each candidate trajectory; S3, using the comfort prediction value to participate in the path cost estimation, and deciding the optimal trajectory from the multiple candidate trajectories.

[0067] This invention provides an intelligent driving method, system, and storage medium based on driver biofeedback. It achieves real-time fusion of multimodal biosignals (such as heart rate collected by a steering wheel capacitive sensor and eye movement trajectories captured by a DMS camera) and vehicle motion signals (such as acceleration and yaw rate obtained from the CAN bus). A multi-branch feature extraction network model is used for cross-modal feature extraction (e.g., coupling head movement and vehicle movement through an attention mechanism to calculate motion sickness indicators). Based on multi-task learning, it outputs quantitative indicators such as tension and motion sickness in parallel. Furthermore, a personalized comfort prediction model maps the trajectory's dynamic parameters (such as jerkiness and rate of curvature change) to comfort prediction values. Finally, in a multi-objective cost function, it collaboratively optimizes with safety and efficiency costs to achieve trajectory decision-making. The model is continuously adaptively adjusted through closed-loop feedback (comparing predicted and measured comfort levels).

[0068] This invention transforms autonomous driving planning from relying on static rules to making personalized decisions based on dynamic responses to the driver's physiological state. For example, it avoids aggressive actions that cause a sudden increase in heart rate by adjusting the acceleration curve in real time, or it reduces head-vehicle motion coupling by optimizing lateral movement to alleviate motion sickness. This significantly improves ride comfort while ensuring safety, making autonomous driving behavior closer to the personalized preferences and real-time state of human drivers.

[0069] It should be noted that the descriptions of each embodiment in the above embodiments have different focuses. For parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0070] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0071] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0072] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0073] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0074] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0075] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. An intelligent driving method based on driver biofeedback, characterized in that, include: S1, through a preset driver state analysis model, analyzes the real-time collected vehicle motion signals and the driver's multimodal biological signals to obtain a comfort score representing the driver's subjective feelings. S2, based on the current environmental perception results, plan multiple candidate trajectories, and based on the mapping relationship between vehicle motion state parameters and comfort scores in the personalized comfort prediction model, calculate the comfort prediction value corresponding to each candidate trajectory; S3, using the comfort prediction value to participate in the path cost estimation, and deciding the optimal trajectory from the multiple candidate trajectories.

2. The intelligent driving method based on driver biofeedback according to claim 1, characterized in that, Step S1 also includes preprocessing the vehicle motion signal and the driver's multimodal biosignals, wherein the multimodal biosignals include at least physiological signals, head motion signals and eye movement trajectory signals; The preprocessing includes: The collected vehicle motion signals were smoothed using Kalman filtering, and the collected multimodal biological signals were processed using the 3σ criterion to remove outliers. After removing outliers, the vehicle motion signals and multimodal biological signals were normalized, data augmented, and time-aligned.

3. The intelligent driving method based on driver biofeedback according to claim 1, characterized in that, The driver state analysis model includes a multi-branch feature extraction network, a cross-modal feature fusion layer, and a multi-task regression output layer; step S1 includes: S101, vehicle motion signals and multimodal biological signals are input into a parallel multi-branch feature extraction network to extract modality-specific features and obtain the temporal features of each modality; S102, through the cross-modal feature fusion layer, calculates the correlation weight between temporal features of different modalities based on the multi-head self-attention mechanism, and weights and fuses the temporal features of each modality to generate a cross-modal fusion feature vector; S103, input the cross-modal fusion feature vector into the multi-task regression output layer, and calculate the driver's stress index, motion sickness index and attention level index in parallel. S104. Based on the tension index, motion sickness index and attention level index, a comfort score is generated by weighted fusion.

4. The intelligent driving method based on driver biofeedback according to claim 3, characterized in that, Step S103 includes: The cross-modal fusion feature vector is input into a multi-task regression output layer, which includes three parallel network branches corresponding to the calculation of stress index, motion sickness index, and attention level index, respectively; wherein: The network branch that calculates the stress index is configured to output weights that depend on the feature dimensions related to heart rate and skin conductance in the cross-modal fusion feature vector; The network branch that calculates motion sickness indices is configured to have its inputs fused with a coupling coefficient determined by the temporal correlation between head motion signals and vehicle motion signals; The network branch that calculates the attention level index is configured to have an attention weight matrix fused to its input, which is generated based on the key regions of the driver's field of vision where the eye-tracking gaze point is located. The three parallel network branches are configured to share the cross-modal fusion feature vector as the basic input and are trained collaboratively using a joint loss function to simultaneously output physiologically related stress, motion sickness and attention levels.

5. The intelligent driving method based on driver biofeedback according to claim 1, characterized in that, In step S2, the planning of multiple candidate trajectories based on the current environmental perception results includes: S201, based on the current environmental perception results, generates multiple feasible candidate trajectories with differences in speed, acceleration and / or path curvature, under the premise of meeting preset safety conditions, traffic regulations and vehicle dynamics constraints.

6. The intelligent driving method based on driver biofeedback according to claim 1, characterized in that, In step S2, the calculation of the predicted comfort value for each candidate trajectory based on the mapping relationship between vehicle motion state parameters and comfort scores in the personalized comfort prediction model includes: S202, calculate the predicted vehicle motion state parameters corresponding to each candidate trajectory, the predicted vehicle motion state parameters include: acceleration, jerkiness, lateral acceleration, yaw rate, curvature and rate of change of curvature; S203, input the predicted vehicle motion state parameters corresponding to the current candidate trajectory into the personalized comfort prediction model, output the comfort prediction sequence of the current candidate trajectory execution process according to the mapping relationship between the vehicle motion state parameters and the comfort score, and use the weighted average or minimum value of the comfort prediction sequence as the comfort prediction value of the current candidate trajectory. S204, loop through S203, and traverse all candidate trajectories.

7. The intelligent driving method based on driver biofeedback according to claim 1, characterized in that, Step S3 includes: S301, Construct a multi-objective decision function, which is defined as a weighted sum of safety cost, efficiency cost, and comfort cost; S302, for each candidate trajectory, its comfort prediction value is mapped to a comfort cost, and its safety cost and efficiency cost are calculated at the same time; S303, Substitute the safety cost, efficiency cost and comfort cost into the multi-objective decision function to calculate the total decision cost for each candidate trajectory; S304: Select the trajectory with the minimum total decision cost among all candidate trajectories as the optimal trajectory.

8. The intelligent driving method based on driver biofeedback according to claim 1, characterized in that, Also includes: The actual comfort score is calculated based on the vehicle operation signals collected when the vehicle executes the optimal trajectory and the multimodal biological signals. The actual comfort score is compared with the predicted comfort value of the current trajectory, and the personalized comfort prediction model is adaptively optimized based on the comparison result.

9. An intelligent driving system based on driver biofeedback, characterized in that, include: The acquisition module is configured to acquire multimodal biosignals of the driver, vehicle motion signals, and current environmental information around the vehicle in real time. The analysis module, which is communicatively connected to the acquisition module, is configured to analyze the multimodal biosignals and vehicle motion signals through a preset driver state analysis model to obtain a comfort score representing the driver's subjective feelings. The decision-making module, which is communicatively connected to the acquisition module and the analysis module, is configured to: plan multiple candidate trajectories based on the current environmental information; calculate the comfort prediction value corresponding to each candidate trajectory based on the mapping relationship between vehicle motion state parameters and comfort scores in the personalized comfort prediction model; and use the comfort prediction value to participate in path cost estimation to determine the optimal trajectory from the multiple candidate trajectories. The execution module, which is communicatively connected to the decision module, is configured to control the vehicle to travel along the optimal trajectory.

10. The intelligent driving system based on driver biofeedback according to claim 9, characterized in that, Also includes: The update module, which communicates with the analysis module and the decision module, is configured as follows: The analysis module calculates the actual comfort score when the vehicle executes the optimal trajectory. The actual comfort score is compared with the corresponding comfort prediction value used by the decision module when making the decision, and a difference signal is generated; Based on the difference signal, the personalized comfort prediction model is adaptively fine-tuned.

Images