Driving decision sharing method and system for autonomous vehicle, and medium

Abstract

The invention relates to the technical field of automatic driving, and discloses a driving decision sharing method and system of an automatic driving vehicle and a medium, and the driving decision sharing method of the automatic driving vehicle comprises the steps: collecting facial expressions, sound features and physiological signal data of passengers through multiple sensors, and constructing a multi-channel emotion perception system; extracting cultural background features of passengers, and constructing an emotion recognition model sensitive to cultural dimensions; historical driving data and emotion feedback of the user are analyzed, and unique driving habits and preference features of the user are extracted; combining the emotion evaluation values of the culture layer and the individual layer to construct a culture individual double-layer emotion model; based on the current emotional state and the individual preference, generating a personalized comfort score, and selecting an optimal driving decision by adopting a multi-objective optimization mode; according to the invention, the user acceptability and commercialization potential of the automatic driving system are improved, and the technical problem that the automatic driving system in the prior art lacks the ability to perceive the emotional state and driving preference of passengers is solved.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and more particularly, to a method, system, and medium for sharing driving decisions of an autonomous driving vehicle. Background Art

[0002] With the rapid development of artificial intelligence and sensor technologies, autonomous driving technology has moved from the laboratory to the forefront of commercial applications. Currently, autonomous driving systems primarily focus on core technical modules such as perception, positioning, planning, and control. These technologies ensure safe and efficient operation in complex environments. However, existing autonomous driving systems still have significant shortcomings in terms of human-machine interaction and user experience.

[0003] Traditional autonomous driving decision-making systems primarily optimize for safety and efficiency, using preset driving parameters and decision-making rules. They lack consideration for passengers' emotional states and personal preferences. This "car-centric" design philosophy prevents the system from adjusting driving behavior based on real-time passenger feedback, resulting in a poor passenger experience and even causing human-machine decision-making conflicts. For example, the system may choose the shortest route or fastest speed, but these choices may cause passengers to feel uncomfortable or anxious.

[0004] In the field of affective computing, while some technologies have attempted to apply emotion recognition to driving scenarios, these technologies typically employ single-modality approaches, such as relying solely on facial expressions or voice features. This makes it difficult to fully and accurately capture a passenger's complex emotional state. Furthermore, existing emotion recognition systems generally ignore the impact of cultural differences on emotional expression and driving preferences, making them unable to adapt to the demands of a global market.

[0005] Furthermore, existing technologies lack effective mechanisms for deeply integrating passenger emotional states with driving decision-making systems. Most systems only make simple adjustments when detecting extreme emotional states (such as fear or anger), failing to achieve fine-grained emotional responses and a personalized driving experience. This disconnected design makes it difficult for autonomous driving systems to gain user trust and acceptance.

[0006] Based on the above background, the present invention proposes a driving decision-sharing method and system for autonomous driving vehicles, aiming to establish a human-machine collaborative decision-sharing model through multi-channel emotion perception, culturally sensitive emotion recognition and personalized driving preference learning, thereby improving the user experience and acceptance of the autonomous driving system. Summary of the Invention

[0007] The present invention provides a method, system and medium for sharing driving decisions in autonomous vehicles, which solves the technical problems in related technologies such as the lack of perception of passengers' emotional states and driving preferences, the inability to adjust driving behavior based on passengers' real-time emotional feedback, and the resulting poor driving experience and conflicts between human and machine decisions.

[0008] The present invention provides a driving decision sharing method for an autonomous vehicle, comprising the following steps:

[0009] Build a multi-channel emotion perception system by collecting passengers' facial expressions, voice characteristics, and physiological signal data through multiple sensors;

[0010] Extract the cultural background characteristics of passengers and build a culturally sensitive emotion recognition model;

[0011] Analyze users' historical driving data and emotional feedback to extract their unique driving habits and preference characteristics;

[0012] Combining the emotional evaluation values of the cultural layer and the individual layer, a dual-level emotional model of culture and individual is constructed. The dual-level emotional model of culture and individual uses an adaptive weight adjustment mechanism to dynamically adjust the weights of the cultural layer and the individual layer in the final emotional evaluation.

[0013] Based on the current emotional state and individual preferences, a personalized comfort score is generated, and a multi-objective optimization method is used to select the best driving decision that meets both safety and functionality while maximizing passenger comfort.

[0014] In a specific embodiment, the step of constructing a multi-channel emotion perception system includes:

[0015] Collect facial expression data, voice feature data, and physiological signal data;

[0016] Preprocess the collected multimodal emotion data to extract facial action unit features, Mel-frequency cepstral coefficient features, and physiological signal statistical features;

[0017] The attention-weighted fusion mechanism is used to fuse the emotional features of different modalities to generate a unified emotional state vector.

[0018] In a specific embodiment, the step of constructing a culturally sensitive emotion recognition model includes:

[0019] Build a multicultural driving preference database containing driving preference data of users from different cultural backgrounds;

[0020] Extract cultural feature vectors based on passengers’ geographic location, language preference, and profile;

[0021] Construct a culturally sensitive weight adjustment function to map cultural feature vectors into weight vectors of each feature dimension in the emotion recognition model;

[0022] The culturally sensitive weight vector is combined with the sentiment feature vector to calculate the culturally sensitive sentiment evaluation value.

[0023] In a specific embodiment, the step of extracting the user's unique driving habits and preference characteristics includes:

[0024] Collect user emotional response data, proactive intervention behaviors, and explicit feedback in different driving scenarios;

[0025] Extracting acceleration preference, turning radius preference, following distance preference, lane change preference, speed stability preference, route selection preference, and other driving behavior characteristics from the collected historical interaction data;

[0026] Build a personalized driving preference model based on the extracted driving behavior characteristics;

[0027] Based on the user's personalized driving preference vector and current driving decision, the individual-level emotion evaluation value is calculated.

[0028] In a specific embodiment, the adaptive weight adjustment mechanism of the cultural-individual dual-level emotion model dynamically calculates the weights of the cultural layer and the individual layer based on the amount of historical interaction data of the user. For new users or users with less data, the cultural layer weight is higher; for long-term users or users with rich data, the individual layer weight is higher.

[0029] In one embodiment, the step of generating a personalized comfort score includes:

[0030] Quantitatively assess the impact of specific driving decisions on passenger comfort based on comprehensive emotion evaluation values and current emotional state;

[0031] Different weights are assigned to different evaluation dimensions of comfort to form a comprehensive comfort score.

[0032] In one embodiment, the multi-objective optimization method selects the best driving decision by the following formula:

[0033] The driving decision that maximizes the weighted sum of the safety score, efficiency score, and comfort score is selected, where the weights are dynamically adjusted based on the current road conditions, traffic environment, and passenger emotional state.

[0034] In one specific embodiment, the driving decision sharing method of the autonomous driving vehicle further includes the following steps:

[0035] Continuously monitor passengers' emotional reactions;

[0036] Form a closed-loop feedback loop based on monitoring results;

[0037] Continuously optimize the driving decision-making process.

[0038] In one embodiment, a driving decision sharing system for an autonomous vehicle, configured to execute a driving decision sharing method for an autonomous vehicle, includes:

[0039] A multi-channel emotion perception module, which collects passengers' facial expressions, voice characteristics, and physiological signal data through multiple sensors;

[0040] A culturally sensitive emotion recognition module, which is used to extract cultural background characteristics of passengers and build a corresponding emotion recognition model;

[0041] Individual driving style feature extraction module, which is used to analyze the user's historical driving data and emotional feedback to extract the user's unique driving habits and preference characteristics;

[0042] The cultural-individual dual-level emotion model module is used to combine the emotion evaluation values of the cultural layer and the individual layer, and dynamically adjust the weights through an adaptive weight adjustment mechanism;

[0043] The personalized comfort scoring and decision optimization module generates a comfort score based on the current emotional state and individual preferences, and uses a multi-objective optimization method to select the best driving decision.

[0044] In one embodiment, a computer-readable storage medium is provided for storing computer-readable instructions that, when read by a computer, can operate a driving decision sharing system for an autonomous vehicle.

[0045] The beneficial effects of the present invention are:

[0046] The multi-channel emotion perception system constructed by the present invention can simultaneously collect and analyze passengers' facial expressions, voice characteristics and physiological signals, greatly improving the accuracy and robustness of emotion recognition, enabling the system to fully understand the passengers' complex emotional states.

[0047] The culturally sensitive emotion recognition model introduced in this invention effectively solves the technical difficulties of cross-cultural emotion understanding, enabling the system to adapt to the emotional expressions and driving preferences of passengers from different cultural backgrounds, significantly enhancing the global adaptability of the system.

[0048] The cultural individual dual-level emotion model proposed in this invention innovatively combines group cultural characteristics and individual preference characteristics, and realizes a smooth transition from "culture-based coarse-grained adaptation" to "individual-based fine-grained customization" through an adaptive weight adjustment mechanism, greatly improving the system's service quality for new and old users.

[0049] The personalized comfort scoring mechanism and multi-objective optimization decision-making method designed in this invention incorporate passengers' emotional experience into decision-making considerations while ensuring safety and efficiency, realizing human-machine sharing of driving decisions and effectively improving passengers' trust and satisfaction with the autonomous driving system.

[0050] The technical solution of the present invention continuously optimizes the driving decision-making process by continuously learning passengers' emotional feedback, forming a closed-loop feedback mechanism, enabling the system to continuously improve service quality as usage time increases, and laying a solid foundation for the large-scale commercial application of autonomous driving technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 is a flow chart of a driving decision sharing method for an autonomous driving vehicle of the present invention;

[0052] Figure 2 It is a bar chart comparing the accuracy of the multimodal emotion data fusion of the present invention;

[0053] Figure 3 is a radar chart of the impact of cultural dimensions on driving style preferences of the present invention;

[0054] Figure 4 It is a line graph of the emotion prediction accuracy of the cultural individual two-level model of the present invention and the traditional model;

[0055] Figure 5 It is a line graph showing how the multi-objective optimization decision of the present invention improves user satisfaction. DETAILED DESCRIPTION

[0056] The subject matter described herein will now be discussed with reference to example embodiments. It should be understood that these embodiments are discussed solely to enable those skilled in the art to better understand and implement the subject matter described herein, and that the functions and arrangements of the elements discussed may be varied without departing from the scope of this specification. Various examples may omit, substitute, or add various processes or components as needed. Furthermore, features described in some examples may be combined in other examples.

[0057] At least one embodiment of the present invention discloses a method for sharing driving decisions of an autonomous vehicle, such as Figure 1 As shown, the following steps are included:

[0058] Step 1: Use multiple sensors to collect passengers’ facial expressions, voice characteristics, and physiological signal data to build a multi-channel emotion perception system;

[0059] It includes the following sub-steps:

[0060] Step 1.1, collect multimodal emotion data;

[0061] The system collects multimodal emotion data including facial expressions, vocal features, and physiological signals.

[0062] Specifically:

[0063] Facial expression data: The high-definition camera in the car collects key facial information of passengers, including the position and movement trajectory of facial features such as eyebrows, eyes, and mouth corners;

[0064] Voice feature data: The microphone array collects passenger voice data and extracts acoustic features such as pitch, volume, speaking rate, and voice tremor.

[0065] Physiological signal data: The seat's built-in pressure sensors, temperature sensors, and optional heart rate monitoring devices collect physiological signal data such as changes in passengers' posture, body temperature, and heart rate.

[0066] The output result is the original multimodal emotion dataset D raw , which contains sentiment data of different modalities.

[0067] Step 1.2, preprocessing multimodal sentiment data;

[0068] The system collects the original multimodal emotional data D raw Perform preprocessing, including noise reduction, feature extraction, and normalization.

[0069] For facial expression data, extract facial action unit (FAUs) features;

[0070] For the sound feature data, Mel-Frequency Cepstral Coefficients (MFCC) and fundamental frequency contour features are extracted;

[0071] For physiological signal data, statistical features and frequency domain features are extracted.

[0072] The preprocessing process can be expressed as:

[0073] D processed =f preprocess (D raw );

[0074] Among them, f preprocess Represents a preprocessing function, which is used to perform noise reduction, feature extraction, and standardization on the original data; D raw Represents the raw multimodal emotion data collected from multiple sensors, including facial expressions, voice features, and physiological signal data; D processed It indicates that the multimodal emotion data has been preprocessed, noise reduction, feature extraction and normalization have been completed, and can be directly used for subsequent emotion recognition analysis.

[0075] Step 1.3: Fuse multimodal emotional features to construct a real-time emotional state vector;

[0076] In addition, the system uses a multimodal feature fusion algorithm to fuse the emotional features of different modalities to generate a unified emotional state vector. This implementation adopts an attention-weighted fusion mechanism to adaptively adjust the weights of different modal features to adapt to the changes in the reliability of each modality in different situations.

[0077] In its implementation, the attention-weighted fusion mechanism dynamically adjusts the reliability of emotion recognition in different driving scenarios. For example, in low-light nighttime driving, facial expression recognition may be limited. In this case, the system automatically downweights facial features while increasing the weights of vocal features and physiological signals. Conversely, in noisy environments, the system downweights vocal features while increasing the weights of other modalities. This adaptive adjustment ensures that the system maintains high emotion recognition accuracy even when the quality of some sensor data degrades.

[0078] The calculation formula of the emotional state vector is as follows:

[0079]

[0080] Among them, e t represents the emotional state vector at the current time t, which contains the various dimensional features of the passenger's emotions; m modal Indicates the number of emotional modalities, that is, the total number of emotional data types collected by the system; Represents the weight of the i-th emotional modality, which is used to adjust the importance of different modalities in the fusion process; Represents the feature extraction function of the i-th emotional modality, which is used to extract the emotion from the preprocessed data D processed Extract the emotional features of this modality; D processed Represents preprocessed multimodal sentiment data, including data after noise reduction, standardization, and other processing.

[0081] Weight Dynamically calculated through the following attention mechanism:

[0082]

[0083] in, represents the weight of the i-th emotional modality, which is used to determine the importance of this modality in multimodal fusion; exp(q i ) represents the exponential function value of the attention score of the i-th modality; Represents the sum of the exponential function values of all modal attention scores, which is used for normalization to ensure that the sum of all weights is 1; q i represents the attention score of the i-th modality, which is learned through neural network; m modalIt represents the number of emotional modalities, such as the number of different modalities such as facial expressions, vocal features, and physiological signals; exp represents the natural exponential function.

[0084]

[0085] Among them, q i represents the attention score of the i-th modality; V is the parameter vector of the attention mechanism, which is used to map the intermediate features into scalar attention scores; v T Represents the transpose of vector v, which is used to implement vector inner product operation; W modal is the weight matrix used to perform linear transformation on modal features; represents the eigenvector of the i-th mode extracted from the preprocessed data; b modal is a bias vector used to adjust the baseline value of the linear transformation; tanh is the hyperbolic tangent activation function, which compresses the eigenvalues to the range [-1, 1], enhancing the model's nonlinear expressiveness. These parameters are automatically learned during the model training process to maximize the accuracy of emotion recognition.

[0086] In some embodiments, the multimodal feature fusion process can adopt a more complex hierarchical fusion architecture. For example, the system can first internally fuse the same type of perceptual data (such as facial expression data from multiple cameras) and then fuse features from different modalities at a higher level. Alternatively, the system can also adopt a multimodal fusion method based on graph neural networks, representing each modal feature as a graph node and the emotional association relationship as an edge, so as to better capture the interrelationships and dependencies between modalities.

[0087] Therefore, the output result is the real-time emotional state vector e t ,This vector contains information about the passenger’s current emotional state, such as dimensions such as pleasure, arousal, and sense of control.

[0088] Step 2: Extract the cultural background characteristics of passengers and build a culturally sensitive emotion recognition model;

[0089] It includes the following sub-steps:

[0090] Step 2.1, construct a multicultural driving preference database;

[0091] The system collects driving preference data from users of different cultural backgrounds and builds a multicultural driving preference database. This database contains user preference data for various driving behaviors (such as acceleration, turning radius, following distance, etc.) from different cultural backgrounds.

[0092] Each record in the database can be represented as:

[0093] R i =(ci ,P i ,E i );

[0094] Among them, R i represents the i-th record in the multicultural driving preference database, which is used to store the relationship between driving preference and emotional response in a specific cultural context; c i Represents the cultural background feature vector, which contains multiple dimensions describing cultural characteristics and is used to quantify the cultural background characteristics of users; P i Represents a set of driving behavior parameters, including acceleration, deceleration, turning radius, following distance, lane change frequency and other driving behavior related parameters, which are used to describe the quantitative indicators of a specific driving style; E i Represents the corresponding emotional response set, including values of emotional dimensions such as pleasure, arousal, sense of control, and comfort, and is used to record the user's emotional experience data under specific driving behaviors.

[0095] The output is a multicultural driving preference database DB culture .

[0096] Step 2.2, extract cultural feature vector;

[0097] The system extracts a cultural feature vector based on the passenger's geographic location, language preference, profile, and other information. The cultural feature vector c contains multiple dimensions that describe cultural differences, such as:

[0098]

[0099] Among them, c represents the cultural feature vector, which is used to describe the cultural background characteristics of passengers; c1, c2, Represents the 1st, 2nd, kth cult cultural dimension components; k cult is the number of cultural dimensions.

[0100] These dimension values can be obtained from a preset cultural dimension database based on the passenger's country / region information, or can be dynamically adjusted through interactive learning.

[0101] In some embodiments, in addition to Hofstede's cultural dimensions, the system can also integrate other cultural dimension models, such as Schwartz's value theory or Glober's cultural dimensions, to obtain a more comprehensive representation of cultural characteristics. Alternatively, the system can construct a hybrid cultural characteristic vector that extracts complementary characteristics from multiple cultural dimension theories.

[0102] Therefore, the output result is the cultural feature vector c of the passenger.

[0103] Step 2.3, construct a culturally sensitive weight adjustment function;

[0104] In addition, the system constructs a culturally sensitive weight adjustment function to adjust the parameters of the emotion recognition model according to the cultural feature vector.

[0105] This function maps the cultural feature vector c to the weight vector w(c) of each feature dimension in the emotion recognition model:

[0106] w(c)=f culture (c);

[0107] Where w(c) represents the weight vector generated according to the cultural feature vector, which is used to adjust the importance of each feature dimension in the emotion recognition model; c represents the cultural feature vector, which contains multiple dimension values describing the cultural background of the passenger; f culture It is a culturally sensitive mapping function that converts cultural feature vectors into weight parameters of the emotion recognition model. It can be implemented as a neural network or other machine learning model. Its purpose is to capture the differences in emotional expression and understanding in different cultural backgrounds.

[0108] In the specific implementation, f culture It can be expressed as:

[0109] f culture (c)=softmax(W c c+b c );

[0110] Among them, W c is a learnable parameter matrix used to map the cultural feature vector c to the weight space, capturing the influence of different cultural dimensions on emotion recognition; b c is a learnable bias vector used to adjust the baseline value of the mapping and provide model flexibility; c is a cultural feature vector containing multiple dimensional values describing the user's cultural background; the softmax function is a normalization function that converts the output into a probability distribution form to ensure that the sum of all weights is 1, which facilitates the reasonable allocation of the importance of each feature during the emotion recognition process.

[0111] In practice, this culturally sensitive weight adjustment function is optimized specifically for users in different regions. For example, for passengers from East Asian cultural backgrounds, the system will place greater weight on smoothness and predictability in driving behavior, based on the region's high uncertainty avoidance. Meanwhile, for passengers from North American cultural backgrounds, the system may prioritize personal control and reaction speed, based on the culture's high individualism. In practice, this model automatically loads the corresponding initial cultural parameters from the existing Hofstede cultural dimension database and then continuously refines and adjusts them based on passenger feedback, forming a more personalized set of culturally adapted parameters.

[0112] Step 2.4, calculate culturally sensitive emotion evaluation values;

[0113] Finally, the system combines the culturally sensitive weight vector w(c) with the sentiment feature vector to calculate the culturally sensitive sentiment evaluation value.

[0114] The formula for calculating the culturally sensitive sentiment evaluation value is:

[0115]

[0116] Among them, E c (s, c) represents the evaluation value of the emotional stimulus S under the cultural background C, that is, the comprehensive score of the emotional response calculated based on the specific cultural background; s represents the emotional stimulus, which refers to a specific driving behavior or decision; c represents the cultural feature vector, which contains multiple dimension values describing the passenger's cultural background; n feat The number of dimensions representing emotional features, that is, the total number of features used to describe emotional responses; w i (c) represents the weight of the i-th characteristic dimension under cultural background C, reflecting the differences in the degree of importance attached to each emotional characteristic in different cultural backgrounds; represents the value of the emotional stimulus S on the i-th cultural characteristic dimension, that is, the original score of the specific driving behavior on each emotional dimension; It represents the weighted summation of all emotional feature dimensions, integrating the emotional responses of each dimension into an overall evaluation value.

[0117] To sum up, the output result is the culturally sensitive sentiment evaluation value E c (s, c), the evaluation value reflects the passenger's emotional response to a specific driving behavior in a specific cultural context.

[0118] Step 3: Analyze the user's historical driving data and emotional feedback to extract the user's unique driving habits and preference characteristics;

[0119] It includes the following sub-steps:

[0120] Step 3.1, collect user historical interaction data;

[0121] When a user uses an autonomous vehicle, the system collects historical user interaction data, including:

[0122] User emotional response data in different driving scenarios;

[0123] Active user intervention in the automated driving system, such as taking over control or adjusting settings;

[0124] Explicit feedback from users (such as ratings, reviews, etc.).

[0125] These data constitute the user historical interaction dataset Hu , recorded users’ emotional responses and preference expressions under different driving decisions.

[0126] Step 3.2, extract driving behavior features;

[0127] The system extracts driving behavior features from the collected historical interaction data to form the user's driving behavior feature set.

[0128] These characteristics include, but are not limited to:

[0129] Acceleration preference: user response to different acceleration levels;

[0130] Turn radius preference: User response to different turn radii;

[0131] Following distance preference: user response to different following distances;

[0132] Lane change preference: User response to different lane change methods;

[0133] Speed stability preference: the user's sensitivity to speed fluctuations;

[0134] Route selection preferences: User responses to different route selection strategies.

[0135] Driving behavior feature extraction can be expressed as:

[0136] F d =f extract (H u );

[0137] Among them, F d represents the driving behavior feature set, including the user's preference feature vectors in multiple dimensions such as acceleration, turning radius, and following distance; f extract represents the feature extraction function, which is an algorithm or mapping relationship that converts user historical interaction data into structured driving behavior features; H u represents a historical user interaction dataset, containing historical data such as users' emotional responses, proactive interventions, and explicit feedback in different driving scenarios. This formula describes the conversion process from raw user interaction data to structured driving behavior features, which is a fundamental step in building a personalized driving model.

[0138] Step 3.3, building a personalized driving preference model;

[0139] In addition, the system is based on the extracted driving behavior features F d , build a personalized driving preference model that can predict users' emotional responses and comfort scores to different driving decisions.

[0140] This implementation adopts a collaborative filtering algorithm based on tensor decomposition, organizes users, driving behaviors, and emotional responses into third-order tensors, and learns implicit user preference features through tensor decomposition.

[0141] The personalized driving preference model can be expressed as:

[0142] p=f predference (F d );

[0143] Where p represents the user's personalized driving preference vector, which includes the user's preference for different driving behavior characteristics; f preference represents the preference model function, which is a mathematical function that maps driving behavior characteristics to preference vectors; d Represents a set of driving behavior features, including various driving behavior characteristics extracted from historical user interaction data, such as acceleration preference, turning radius preference, and following distance preference. This formula describes how to convert a user's driving behavior characteristics into a quantized personalized driving preference vector through a specific preference model function for subsequent driving decision optimization.

[0144] In the specific implementation, f preferance It can be expressed as:

[0145] f preferance (F d )=W p ·F d +b p ;

[0146] Among them, W p represents a learnable parameter matrix, which is used to convert driving behavior characteristics into preference vectors, reflecting the influence of different driving behavior characteristics on user preferences; F d represents the user's driving behavior feature set, including the user's preference features in various driving dimensions; b p represents a learnable bias vector used to adjust the baseline value of the model to ensure that reasonable preference predictions can be generated even when the eigenvalue is zero; f preference (F d ) represents the output of the preference model function, that is, the user's personalized driving preference vector.

[0147] In some embodiments, the personalized driving preference model may employ nonlinear representation methods to better capture the complexity and diversity of user preferences. For example, the system may construct a personalized driving preference model using a deep neural network, comprising multiple fully connected layers and nonlinear activation functions, to learn more complex user preference characteristics. Alternatively, the system may employ an attention mechanism to focus on differences in user preferences for specific driving scenarios (e.g., congested roads, highways, mountain roads, etc.).

[0148] In other implementations, the personalized driving preference model can be combined with reinforcement learning methods to continuously adjust and optimize model parameters based on real-time user feedback. For example, the system can use positive user emotional reactions as reward signals and negative user emotional reactions as penalty signals. Using reinforcement learning algorithms, the system can optimize driving decision-making strategies, gradually adapting to the user's personalized preferences.

[0149] Step 3.4, calculate the individual-level sentiment evaluation value;

[0150] The system calculates the individual-level emotional evaluation value based on the user's personalized driving preference vector p and current driving decision d.

[0151] The calculation formula for the emotional evaluation value at the individual level is:

[0152] E p (a, p) = f individual (a, p);

[0153] Among them, E p (a, p) represents the emotional evaluation value of driving behavior a under individual preference p, that is, the emotional response score to a specific driving behavior based on the user's personal preference; a represents the current driving behavior, including multi-dimensional features such as acceleration, turning radius, and following distance; p represents the user's personalized driving preference vector, reflecting the user's personal preference strength in each driving dimension; f individual It is an individual-level emotional evaluation function that is used to match driving behavior characteristics with user preferences and output the user's emotional evaluation value of the driving behavior.

[0154] In the specific implementation, f individual It can be expressed as:

[0155]

[0156] Where m represents the number of dimensions of driving behavior features, such as the total number of features such as acceleration, turning radius, and following distance; p i represents the user's preference value on the i-th driving behavior feature, reflecting the user's emphasis on or preference intensity for this feature; g i (a) represents the value of driving behavior a on the i-th feature dimension, that is, the specific performance of the current driving behavior on this feature dimension; f individual (a, p) represents the individual-level sentiment evaluation function. By weightedly summing the preference values for each feature dimension with the actual driving behavior values, the user's overall sentiment evaluation of their current driving behavior is calculated. This formula quantifies the match between a user's personalized preferences and their actual driving behavior.

[0157] In summary, the output result is the individual level sentiment evaluation value E p (a, p), the evaluation value reflects the personalized emotional response of a specific user to a specific driving behavior.

[0158] Step 4: Combine the emotional evaluation values of the cultural layer and the individual layer to construct a dual-level emotional model of culture and individual. The dual-level emotional model of culture and individual uses an adaptive weight adjustment mechanism to dynamically adjust the weights of the cultural layer and the individual layer in the final emotional evaluation.

[0159] It includes the following sub-steps:

[0160] Step 4.1, design a two-level model structure;

[0161] The structure of the cultural-individual two-level emotional model is systematically designed, including the cultural layer, the individual layer and the fusion layer.

[0162] The cultural layer processes emotional evaluations related to culture, the individual layer processes emotional evaluations related to individual preferences, and the fusion layer integrates the evaluation results of the two levels.

[0163] The overall structure of the model can be expressed as:

[0164] E combined (a, c, p) = f fusion (E c (a,c),E p (a, p));

[0165] Among them, E combined (a, c, p) represents the emotional evaluation value of driving behavior a based on the comprehensive cultural background c and individual preference p, that is, the final emotional response prediction value obtained after considering both cultural factors and personal preferences; f fusion represents the fusion function, which is used to integrate the emotional evaluation results of the cultural layer and the individual layer to generate the final comprehensive evaluation value; E c (a, c) represent the emotional evaluation values of the cultural layer, reflecting the general emotional response to driving behavior in a specific cultural context; E p (a, p) represents the emotional evaluation value at the individual level, reflecting the personalized emotional response to driving behavior based on personal historical preferences.

[0166] Step 4.2, implement adaptive layer weight adjustment;

[0167] The system implements an adaptive layer weight adjustment mechanism, which dynamically adjusts the weights of the cultural layer and the individual layer in the final evaluation based on the passenger's usage history and current context.

[0168] For new users or users with less data, the cultural layer has a higher weight;

[0169] For long-term users or users with rich data, the individual layer weight is higher.

[0170] In the specific implementation, the calculation formula of the cultural individual two-level emotion model is:

[0171] E combined (a, c, p) = α cult ·E c (a, c)+β indiv ·E p (a, p);

[0172] Among them, E combined (a, c, p) represents the total emotional evaluation value after integrating cultural background and individual preferences; a represents the current driving behavior; c represents the cultural background vector; p represents the user's personalized driving preference vector; E c (a, c) represent the emotional evaluation values of the cultural layer, reflecting the evaluation of driving behavior in a specific cultural context; E p (a, p) represents the emotional evaluation value at the individual level, reflecting the evaluation of driving behavior based on personal preferences; α cult , β indiv Represent the weight coefficients of the cultural layer and the individual layer, respectively, satisfying α cult +β indiv =1, ensuring that the sum of the weights of the two levels is 1.

[0173] The adaptive weight is calculated as:

[0174]

[0175] Among them, α cult represents the weight coefficient of the cultural layer in the emotional evaluation, which decreases as the user data increases; β indiv represents the weight coefficient of the individual layer in the sentiment evaluation, which increases with the increase of user data; n data Represents the amount of user historical interaction data, including the total number of driving records and emotional feedback data of users in the past using the system; γ scale It is a scaling parameter that controls the rate of change of weights as the amount of data increases. A larger γ scale Values will allow the system to move more quickly from cultural-level evaluation to individual-level evaluation.

[0176] In practical applications, the cultural-individual dual-level emotional model dynamically balances the influence of the cultural layer and the individual layer according to the length of time the user has used the system and the data accumulated. For example, for new users who have just started using the autonomous driving system, the system mainly relies on the evaluation of the cultural layer (higher α cultvalue), based on the user's country / region and language preference, the general sentiment evaluation criteria in the corresponding cultural context are applied; as the user continues to use the system and accumulates more personal data, the weight of the individual layer will gradually increase (higher β indiv value), the system makes predictions based more on the user's own historical emotional reactions and preference characteristics.

[0177] This dual-layer architecture is particularly well-suited for cross-cultural travel scenarios, such as when international businesspeople use autonomous driving services in different countries. The system can maintain the user's personal preferences while appropriately incorporating local cultural characteristics, providing a driving experience that is both familiar and adaptable to the local environment. For example, for a user accustomed to the German Autobahn driving style using an autonomous driving service in an Asian city, the system will retain the user's preference for higher speeds while appropriately prioritizing smooth driving in congested traffic.

[0178] Step 4.3, generate personalized comfort score;

[0179] In addition, the system is based on the comprehensive emotion evaluation value E(a,c,p) and the current emotional state e t , generating a personalized comfort score that quantitatively assesses the impact of specific driving decisions on passenger comfort.

[0180] The formula for calculating the personalized comfort score is:

[0181]

[0182] Where C(d) represents the comfort score of driving decision d, that is, the quantitative evaluation value of the comfort level of a specific driving decision; n represents the number of evaluation dimensions, including multiple evaluation indicators such as smoothness, speed sense, and turning comfort; α i represents the weight of the i-th dimension, reflecting the importance of this dimension in the overall comfort evaluation; f i (e t , p, d) represents the current emotional state e t and individual preference p, the scoring function of driving decision d on the i-th dimension; e t represents the passenger's current emotional state, such as calm, nervous, or happy; p represents the passenger's individual preference vector, which includes the intensity of preference for each driving feature; and d represents the driving decision to be evaluated, which includes multiple parameters such as speed, acceleration, and path selection.

[0183] Therefore, the output result is the personalized comfort score C(d).

[0184] Step 5: Based on the current emotional state and individual preferences, a personalized comfort score is generated, and a multi-objective optimization approach is used to select the best driving decision that satisfies both safety and functionality while maximizing passenger comfort.

[0185] It includes the following sub-steps:

[0186] Step 5.1, generate a candidate driving decision set;

[0187] The system generates a set of candidate driving decision sets D = {d1, d2, ..., d k Where D represents the set of all possible candidate driving decisions; d1, d2, d k represent the 1st, 2nd, and kth candidate driving decisions respectively; k represents the total number of candidate driving decisions.

[0188] Step 5.2, evaluate the comfort level of candidate decisions;

[0189] The system calculates each decision d in the candidate driving decision set i , use the personalized comfort model to calculate its comfort score C(d i ):

[0190]

[0191] Among them, C(d i ) represents the comfort score of the i-th candidate driving decision, that is, the quantitative evaluation value of the comfort level of this specific driving decision; n represents the number of evaluation dimensions, including multiple evaluation indicators such as smoothness, speed sense, and cornering comfort; α j represents the weight coefficient of the jth dimension, reflecting the importance of this dimension in the overall comfort evaluation; f j (e t ,p,d i ) indicates the current emotional state e t and individual preference p, driving decision d i Scoring function on the jth dimension; e t represents the passenger's current emotional state, such as calm, nervous, happy, etc.; p represents the passenger's individual preference vector, including the preference intensity for each driving feature; d i represents the i-th driving decision to be evaluated, including multiple parameters such as speed, acceleration, and path selection.

[0192] Therefore, the output is a set of comfort scores for candidate decisions.

[0193] Step 5.3, multi-objective optimization decision selection;

[0194] The system performs multi-objective optimization based on safety, functionality and comfort scores to select the best driving decision.

[0195] The multi-objective optimization problem can be expressed as:

[0196]

[0197] Among them, d * represents the best driving decision, that is, the driving decision that the system finally chooses to execute; d i represents the i-th candidate decision in the candidate driving decision set D; argmax represents the parameter value when the objective function reaches the maximum value; Safety(d i ) represents decision d i The safety score of the decision is used to quantitatively evaluate the safety of the decision; Efficiency (d i ) represents decision d i The efficiency score is used to quantitatively evaluate the driving efficiency of the decision, such as the time to reach the destination, fuel / electricity consumption, etc.; C(d i ) represents decision d i The comfort score is the personalized comfort evaluation value calculated in the previous steps; λ1, λ2, and λ3 represent the weight coefficients of safety, efficiency, and comfort, respectively, satisfying λ1+λ2+λ3=1, ensuring that the sum of the three weights is 1.

[0198] In practical applications, the multi-objective optimization decision-making process dynamically adjusts the weight parameters according to different scenarios and passenger status. For example, in a congested urban road environment, the system may increase the comfort score C(d i ) weight λ3, providing a smoother and more comfortable driving experience; on the highway or in an emergency, the system will increase the safety score Safety (d i ) weight λ1 to ensure driving safety.

[0199] This optimized decision-making method is particularly effective when road conditions change in real time. For example, when a vehicle enters a city road from a highway, the system automatically adjusts the weighting based on the change in road type, ensuring a smooth transition in driving style. Furthermore, when the system detects passenger anxiety, such as in unfamiliar areas or in inclement weather, it increases the weighting for comfort, providing a more anxiety-relieving driving experience.

[0200] Through this dynamic weight adjustment, the system can find the optimal balance between the environment and the emotional state of the passengers while ensuring safety. It neither sacrifices comfort for efficiency nor significantly prolongs the journey time due to excessive focus on comfort.

[0201] Step 5.4, update driving control parameters;

[0202] The system will select the best driving decision d *Converted into specific driving control parameters, such as accelerator pedal position, brake pedal position, steering angle, etc., and sent to the vehicle control system for execution.

[0203] At the same time, the system continuously monitors passengers' emotional reactions, forming a closed-loop feedback loop and continuously optimizing the decision-making process.

[0204] To sum up, the output results are the updated driving control parameters and execution results.

[0205] Application examples of this implementation:

[0206] Application scenario: cross-cultural shared travel services;

[0207] The autonomous driving vehicle driving decision sharing method of the present invention is deployed on a shared travel service platform in a certain international metropolis.

[0208] The platform serves passengers from different cultural backgrounds every day, including local residents and international business travelers.

[0209] Autonomous driving fleets need to adapt to the diverse driving preferences and emotional needs of different passengers to provide a personalized riding experience.

[0210] Before system deployment, the platform faced the following challenges:

[0211] International travelers have low acceptance of autonomous vehicles and often give negative reviews because the driving style does not meet expectations;

[0212] The high rate of manual takeover in autonomous driving mode affects service efficiency and safety;

[0213] The same vehicle configuration performs very differently in user areas with different cultural backgrounds;

[0214] Even within the same cultural background, individual users have significantly different preferences for driving styles.

[0215] Therefore, the platform decided to introduce the driving decision sharing method of the present invention to achieve an emotion-driven personalized driving experience.

[0216] Implementation example of multi-channel emotion perception system:

[0217] The ride-sharing platform has installed the following sensor equipment on its autonomous vehicle fleet:

[0218] In-car HD cameras: Installed above the central control screen and in the rearview mirror, capturing high-definition facial images at 120 frames per second;

[0219] Microphone array: Consists of four directional microphones installed in the roof lining, providing 360° sound collection capabilities;

[0220] Seat sensors: Pressure sensors and temperature sensors are embedded in the seat back and seat cushion, with a sampling frequency of 10Hz;

[0221] Optional biometric sensors: For premium models, heart rate monitoring sensors are installed in the steering wheel and armrests.

[0222] In actual application, the system provided services to a passenger of East Asian background who was using the service for the first time. After the vehicle started, the multi-channel emotion perception system began to work:

[0223] Collection phase: The system simultaneously collects passengers' facial expressions (micro-expressions such as raised eyebrows and upturned corners of the mouth), voice data (speaking tone, volume, and speaking speed), and physiological signals (sitting posture changes, hand pressure, and heart rate).

[0224] Preprocessing stage: The system performs noise reduction and feature extraction on the collected raw data.

[0225] For facial images, 68 facial key points and their movement trajectories are extracted;

[0226] For speech data, extract the fundamental frequency contour and volume change curve;

[0227] For physiological signals, pressure distribution maps and heart rate variability indicators are extracted.

[0228] Fusion phase: The system integrates data from various modalities using an attention-weighted fusion mechanism. During a rapid turn, the camera detected a passenger's slightly tense facial expression. Physiological sensors also detected a slight increase in heart rate and a tightening grip on the seat armrest. The fusion algorithm synthesizes these features, assigning a weight of 0.3 to facial expression, 0.2 to vocal features, and 0.5 to physiological signals, generating an emotional state vector for "mild anxiety."

[0229] Through this process, the system generates an emotional state vector in real time that includes dimensions such as pleasure (-0.2, indicating slight discomfort), arousal (0.4, indicating moderate alertness), and sense of control (-0.3, indicating slight uneasiness about vehicle control).

[0230] Implementation example of the dual-level emotional model of cultural individuals:

[0231] For the East Asian passengers mentioned above, the working process of the cultural individual two-level emotion model is as follows:

[0232] Cultural feature extraction: The system identifies the passenger's cultural background as East Asia from the passenger's booking information and loads the corresponding cultural feature vector from the database, including dimensions such as the uncertainty avoidance index (0.85, indicating high) and the individualism index (0.35, indicating low).

[0233] Cultural Sentiment Assessment: Based on Hofstede's cultural dimension model, the system identifies a typical preference for a smooth, predictable driving style for this cultural background, particularly in maneuvers that can introduce uncertainty, such as turning and lane changes. A culturally sensitive weighting function adjusts the weights for smoothness and predictability to 0.4 and 0.3, respectively, while reducing the weight for reaction speed to 0.2.

[0234] Individual-level emotional assessment: Since this is the first use, there is no historical data on the passenger, and the system can only learn from the real-time feedback collected during the current trip. After an acceleration operation, the system detected a slight increase in the passenger's happiness, inferring that the passenger was not very happy with the moderate acceleration (1.2m / s 2 ) has a higher acceptance rate.

[0235] Adaptive adjustment of two-tier weighting: For new users, the system assigns a higher initial weight to the cultural layer (α = 0.8) and a lower weight to the individual layer (β = 0.2). As the journey progresses, the system continuously accumulates data on the passenger's emotional response and dynamically adjusts this ratio.

[0236] Comprehensive Emotional Assessment: The system combines the results of the two layers of assessment to generate a comprehensive emotional assessment of a specific driving decision. For example, for a turning maneuver, the comprehensive assessment value is: E(turning maneuver) = 0.8 × (-0.5) + 0.2 × (-0.2) = -0.44, indicating that the passenger may be experiencing significant discomfort with the current turning maneuver.

[0237] Personalized Comfort Score: The system further incorporates the current emotional state to generate a comfort score for the predicted driving decision. For example, for an upcoming lane change maneuver, the system calculates a comfort score of 0.3 (on a scale of -1 to 1), predicting that the passenger will be slightly accepting.

[0238] Multi-objective optimization decision-making implementation example:

[0239] Based on the passenger's emotional state and comfort score, the system needs to decide how to proceed when approaching a busy intersection. The system generates three candidate driving decisions:

[0240] Decision A: Pass through the intersection quickly with an acceleration of 2.0 m / s 2 , pass in the shortest time.

[0241] Decision B: Pass at a moderate speed, with an acceleration of 1.2 m / s 2 , keep steady.

[0242] Decision C: Slow down and pass carefully, with an acceleration of 0.8 m / s 2 , leaving more reaction time.

[0243] The system performs a multi-objective evaluation for each decision:

[0244] Safety score:

[0245] Decision A: 0.75 (faster speed is slightly less safe at busy intersections);

[0246] Decision B: 0.85 (medium speed provides good safety);

[0247] Decision C: 0.95 (low speed provides the highest safety);

[0248] Efficiency Rating:

[0249] Decision A: 0.90 (high efficiency passed);

[0250] Decision B: 0.75 (moderate passing efficiency);

[0251] Decision C: 0.60 (low efficiency);

[0252] Comfort score (based on passenger emotion model):

[0253] Decision A: 0.40 (passengers are not comfortable with rapid acceleration);

[0254] Decision B: 0.85 (passengers have a high acceptance of moderate acceleration);

[0255] Decision C: 0.70 (too slow may cause anxiety);

[0256] The system dynamically adjusts the weights based on the current situation: taking into account the passenger's mild anxiety and the busy traffic environment, the system sets the safety weight λ1 = 0.4, the efficiency weight λ2 = 0.2, and the comfort weight λ3 = 0.4.

[0257] Final weighted score:

[0258] Decision A: 0.4 × 0.75 + 0.2 × 0.90 + 0.4 × 0.40 = 0.63;

[0259] Decision B: 0.4 × 0.85 + 0.2 × 0.75 + 0.4 × 0.85 = 0.83;

[0260] Decision C: 0.4 × 0.95 + 0.2 × 0.60 + 0.4 × 0.70 = 0.78;

[0261] The system selects decision B (passing at medium speed) with the highest comprehensive score and converts it into specific control parameters: the accelerator pedal position is 30%, the steering wheel angle remains unchanged, and the target speed is adjusted to 40 km / h.

[0262] like Figures 2 to 5As shown, there is a comparison of the accuracy of multimodal emotion data fusion; the impact of cultural dimensions on driving style preferences; the emotion prediction accuracy of the cultural individual two-level model and the traditional model; and the improvement of user satisfaction through multi-objective optimization decision-making.

[0263] Overall, this application establishes a human-machine collaborative decision-making sharing model by deeply integrating emotional computing technology with the autonomous driving decision-making system, effectively improving the user acceptance and commercialization potential of the autonomous driving system, and is a key technical support for the development of autonomous driving technology to a higher level.

[0264] The above describes an embodiment of the present invention, but this embodiment is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Ordinary technicians in this field can also make more forms of equivalent embodiments based on the inspiration of this embodiment, all of which are protected by this embodiment.

Claims

1. A driving decision sharing method for an autonomous vehicle, characterized in that: The following steps are involved: Build a multi-channel emotion perception system by collecting passengers' facial expressions, voice characteristics, and physiological signal data through multiple sensors; Extract the cultural background characteristics of passengers and build a culturally sensitive emotion recognition model; Analyze users' historical driving data and emotional feedback to extract their unique driving habits and preference characteristics; Combining the emotional evaluation values of the cultural layer and the individual layer, a dual-level emotional model of culture and individual is constructed. The dual-level emotional model of culture and individual uses an adaptive weight adjustment mechanism to dynamically adjust the weights of the cultural layer and the individual layer in the final emotional evaluation. Based on the current emotional state and individual preferences, a personalized comfort score is generated, and a multi-objective optimization method is used to select the best driving decision that meets both safety and functionality while maximizing passenger comfort.

2. The driving decision sharing method for an autonomous driving vehicle according to claim 1, characterized in that: The steps of constructing the multi-channel emotion perception system include: Collect facial expression data, voice feature data, and physiological signal data; Preprocess the collected multimodal emotion data to extract facial action unit features, Mel-frequency cepstral coefficient features, and physiological signal statistical features; The attention-weighted fusion mechanism is used to fuse the emotional features of different modalities to generate a unified emotional state vector.

3. The driving decision sharing method for an autonomous driving vehicle according to claim 1, characterized in that: The steps of constructing a culturally sensitive emotion recognition model include: Build a multicultural driving preference database containing driving preference data of users from different cultural backgrounds; Extract cultural feature vectors based on passengers’ geographic location, language preference, and profile; Construct a culturally sensitive weight adjustment function to map cultural feature vectors into weight vectors of each feature dimension in the emotion recognition model; The culturally sensitive weight vector is combined with the sentiment feature vector to calculate the culturally sensitive sentiment evaluation value.

4. The driving decision sharing method for an autonomous driving vehicle according to claim 1, characterized in that: The step of extracting the user's unique driving habits and preference features includes: Collect user emotional response data, proactive intervention behaviors, and explicit feedback in different driving scenarios; Extracting acceleration preference, turning radius preference, following distance preference, lane change preference, speed stability preference, route selection preference, and other driving behavior characteristics from the collected historical interaction data; Build a personalized driving preference model based on the extracted driving behavior characteristics; Based on the user's personalized driving preference vector and current driving decision, the individual-level emotion evaluation value is calculated.

5. The driving decision sharing method for an autonomous driving vehicle according to claim 1, characterized in that: The adaptive weight adjustment mechanism of the cultural-individual dual-level emotion model dynamically calculates the weights of the cultural layer and the individual layer based on the amount of historical user interaction data. For new users or users with less data, the cultural layer has a higher weight; for long-term users or users with rich data, the individual layer has a higher weight.

6. The driving decision sharing method for an autonomous driving vehicle according to claim 1, characterized in that: The steps of generating a personalized comfort score include: Quantitatively assess the impact of specific driving decisions on passenger comfort based on comprehensive emotion evaluation values and current emotional state; Different weights are assigned to different evaluation dimensions of comfort to form a comprehensive comfort score.

7. The driving decision sharing method for an autonomous driving vehicle according to claim 1, characterized in that: The multi-objective optimization method selects the best driving decision through the following formula: The driving decision that maximizes the weighted sum of the safety score, efficiency score, and comfort score is selected, where the weights are dynamically adjusted based on the current road conditions, traffic environment, and passenger emotional state.

8. The driving decision sharing method for an autonomous vehicle according to claim 1, characterized in that: The following steps are also included: Continuously monitor passengers' emotional reactions; Form a closed-loop feedback loop based on monitoring results; Continuously optimize the driving decision-making process.

9. A driving decision sharing system for an autonomous vehicle, configured to execute the driving decision sharing method for an autonomous vehicle according to any one of claims 1 to 8, characterized in that: include: A multi-channel emotion perception module, which collects passengers' facial expressions, voice characteristics, and physiological signal data through multiple sensors; A culturally sensitive emotion recognition module, which is used to extract cultural background characteristics of passengers and build a corresponding emotion recognition model; Individual driving style feature extraction module, which is used to analyze the user's historical driving data and emotional feedback to extract the user's unique driving habits and preference characteristics; The cultural-individual dual-level emotion model module is used to combine the emotion evaluation values of the cultural layer and the individual layer, and dynamically adjust the weights through an adaptive weight adjustment mechanism; The personalized comfort scoring and decision optimization module generates a comfort score based on the current emotional state and individual preferences, and uses a multi-objective optimization method to select the best driving decision.

10. A computer-readable storage medium, characterized in that It is used to store computer-readable instructions, which, when read by a computer, can run the driving decision sharing system for an autonomous driving vehicle as described in claim 9.

Images