Driver trust state evaluation method based on physiological signals

Abstract

The present application relates to a kind of driver trust state evaluation method based on physiological signal, belong to intelligent driving technical field.The method includes: S1: heart rate signal and skin electricity signal are preprocessed and feature extraction;S2: signal fusion: the heart rate feature and skin electricity feature extracted in step S1 are normalized, then the normalized each feature of heart rate and each feature of skin electricity are fused into feature matrix respectively;S3: model input is built;S4: trust evaluation model is built, including convolution module, deep channel attention module, Transformer module and regression prediction module;S5: model output: the target vector is defined as [0,1,2], then the output between regression prediction module and target vector is associated, that is, linear mapping is used to map the regression output to the interval of [0,2];Finally, threshold mapping is carried out, and the classification result is converted using threshold according to the output after mapping.

Description

Technical Field

[0001] This invention belongs to the field of intelligent driving technology and relates to a method for assessing driver trust status based on physiological signals. Background Technology

[0002] Currently, the Society of Automotive Engineers (SAE) in the United States has proposed a classification system for the level of automation in vehicle driving. This system divides the level of automation in vehicle driving into six levels, from no automation (L0) to full automation (L5). Among them, L3 autonomous driving is called conditional automation, where the autonomous driving system can continuously perform all dynamic driving tasks within its designed operating conditions, but the user needs to be ready to take over the driving task at any time. Currently, the autonomous driving level of the in-vehicle infotainment systems of mass-produced cars in my country is transitioning from L2 to L3, and some in-vehicle infotainment systems have reached L4 level in limited scenarios. Autonomous driving has become an important direction for the development of the automotive industry in recent years, but achieving fully driverless autonomous driving is not something that can be achieved overnight and requires a long period of development. Therefore, for a long time to come, autonomous vehicles will be in a human-machine co-driving mode. During this stage, human-machine trust will be an important factor affecting the cooperation between the driver and the autonomous vehicle.

[0003] The concept of trust originated in sociology, encompassing the relationship between the trustor and the trustee. It means that the trustee is aware of the trustor's trust, prioritizes the trustor's interests, and acts out of goodwill towards the trustor. In the field of automated trust, the definition of trust proposed by Lee et al. is widely accepted, defining trust as "an individual's (e.g., a driver's) attitude that an agent (e.g., an autonomous driving system) can help them achieve a certain goal (e.g., a driving task) in uncertain or vulnerable situations." Research on automated trust covers the identification and verification of trust influencing factors, the measurement and modeling of automated trust, and the calibration of automated trust. Different research frameworks exist for automated trust, such as human-robot interaction trust, human-computer interaction trust, and human-autonomous driving interaction trust. In human-robot trust, the trustor is a human, and the trustee is a robot. Since it is difficult for humans and robots to generate mutual emotional interaction, human-robot trust is often manifested as instantaneous trust based on a human's immediate judgment of the robot. Human-computer trust refers to the trust relationship between a human and a computer system, where the trustor is a human, and the trustee is a computer. Human-autonomous driving interaction trust refers to the trust relationship between the driver and the autonomous driving system in the field of autonomous driving, where the driver is the trustee and the autonomous driving system is the trustee.

[0004] Human-autonomous driving interaction trust involves three states: insufficient trust, excessive trust, and appropriate trust. Insufficient trust leads to reduced driver use of autonomous driving functions, increased driver fatigue, and decreased driving comfort; excessive trust may reduce the driver's situational awareness of the autonomous vehicle, posing safety hazards. Appropriate trust, also known as calibrated trust, refers to the consistency between the driver's subjective actual level of trust and the objective level of trust. It aims to ensure the correct use of autonomous vehicles by the driver and is the core objective of human-machine interaction trust research in the field of autonomous driving.

[0005] In the context of autonomous driving, research on human-machine trust plays a crucial role in promoting the development of the current automotive industry and has significant research value in practical driving applications. A primary condition for calibrating driver trust during autonomous driving is the ability to grasp the driver's real-time trust level. Existing methods for real-time trust measurement often use trust scales, which require interrupting the experimental process, resulting in high invasiveness. Furthermore, measurement methods using behavior monitoring are susceptible to interference from ambient light and line-of-sight obstructions, leading to missing or invalid data and reduced model measurement accuracy. Therefore, addressing the issues of high invasiveness and low accuracy in existing trust measurement methods, a method that can accurately assess the driver's real-time trust state is urgently needed. Summary of the Invention

[0006] In view of this, the purpose of the present invention is to provide a driver trust status assessment method based on physiological signals to improve the accuracy of trust measurement.

[0007] To achieve the above objectives, the present invention provides the following technical solution:

[0008] A method for assessing driver trust status based on physiological signals, specifically including the following steps:

[0009] S1: Preprocessing and feature extraction of heart rate and electrodermal signals;

[0010] S2: Signal fusion: Normalize the heart rate features and skin conductance features extracted in step S1, and then fuse the normalized heart rate features and skin conductance features into a feature matrix respectively.

[0011] S3: Construct model input;

[0012] S4: Construct a trust assessment model, including a convolution module, a deep channel attention module, a Transformer module, and a regression prediction module;

[0013] S5: Model Output: Define the target vector as [0,1,2], where 0 represents no trust, 1 represents moderate trust, and 2 represents excessive trust; then establish a correlation between the output of the regression prediction module and the target vector, that is, use linear mapping to map the regression output to the interval [0,2]; finally, perform threshold mapping, and use the threshold to convert the mapped output into a classification result.

[0014] Further, in step S1, the heart rate signal is preprocessed and features are extracted, specifically including: firstly, the RR variability signal RRV is extracted from the ECG, and HRV = 60 / RRV is calculated; the RR period is estimated as the time difference between R consecutive peaks; the RRV signal and the HRV signal mainly contain two components: 1) a low-frequency baseline component, which is due to some mutations caused by trust-induced events; 2) a periodic component, including parasympathetic and sympathetic nerve activity; the latter can be modeled using a P-order autoregressive process AR; the RR interval at time i is denoted as rr. i :

[0015]

[0016] Where b and a are the baselines of the signal and AR coefficients, respectively; c is a random error term representing all unexplained variations; and the noise is Gaussian distributed. Baseline b is a variable affected by trust-inducing events and used for trust detection. The i-th index indicates that these variables are time-varying, i.e.:

[0017] b i =b i-1 +ε b

[0018]

[0019] Where, ε b and ε a It is Gaussian distributed noise; the state-space form is:

[0020] x i =x i-1 +R

[0021] y i =H i x i +Q

[0022] Where, x i =[a1,a2,...,a P ,b i ], y i =rr i R represents process noise, Q represents observation noise, and matrix H i =diag[rr i-1,rr i-2 ,...,rr i-P [1]; In order to estimate x i For each new sample y i Both can use the ordinary Kalman filter equation:

[0023] x i =x i-1

[0024]

[0025] Where K is the Kalman gain matrix, S is the matrix used to calculate the Kalman gain, represents the innovation covariance, represents the uncertainty of the current observation, and x... i and V i These are the Kalman filter estimates of the hidden state and the error covariance matrix at time step i, respectively. and These are the updated hidden state estimates and error covariance matrix at time step i. To better estimate the model's hidden state, this method uses a Kalman smoother. The Kalman smoothing update equation is:

[0026]

[0027] in, and V represents the Kalman smoothing estimate of the hidden state and error covariance matrix at time step i, respectively. i * This is the smoothing gain matrix, used for smoothing the update of the estimate. T denotes the transpose of the matrix, which is used when updating the error covariance matrix. This method uses windows of a specific length to compute features from the physiological signal. Given the current information, a Kalman smoother can be applied to each new window only for the past observations contained within the window. Initial estimates of the AR coefficients and observation noise R, as well as the initial estimate of the baseline, are extracted using the initial segment of the signal. The first 5 minutes of the signal are used for all initialization. This method uses the diagonal Q, setting the elements corresponding to the AR coefficients to 10. -8 The element corresponding to the baseline is set to 10. -6 In a Kalman filter, the baseline parameter b changes over time, therefore the average baseline value b needs to be calculated within a specific window. i The new feature that appears will be denoted as B. k Where k is the window indicator, which inherits from b i The changing behavior of B is controlled by two components: 1) very slow changes and 2) correlation with trust-inducing events; since the first component changes very slowly, this method assumes that B... k N(u) follows a Gaussian distribution kTherefore, under external stimuli, it is expected to increase rapidly, i.e.:

[0028]

[0029] Where δ is the average increase in the centrality of trust events; u k It is also time-varying, u k The estimation was performed in a BN environment, using Kalman filtering and b i It is derived from the estimated decomposition.

[0030] Further, in step S1, the EDA signal is preprocessed and features are extracted, specifically including: first, the EDA signal is downsampled to 1Hz, and then a smoothing filter is used; the EDA signal consists of two parts: a slowly changing component, namely the skin conductance level SCL, and a component related to a sudden increase in confidence, namely the skin conductance response SCR; these rapidly increasing indicators are measured using the first absolute difference (FAD).

[0031]

[0032] Among them, s i Let be the EDA at time i, and N be the total number of time points. The method of dividing the FAD measure by the current SCL level eliminates the influence of SCL on SCR; this method can be easily applied to real-time estimation. The transformation applied to the FAD measure is 1 / SCL. l It is SCL l The nonlinear function, and This function is in SCL l For large (1 / SCL) l When <1), FAD is reduced, and SCL is reduced. l For small (1 / SCL) l >1) amplified FAD. Using SCL l A logistic function:

[0033]

[0034] The transformed FAD(nFAD) is nFAD = FAD·g(SCL) l The parameters α and β were optimized to achieve a higher relevance to the trust metric derived in the annotation.

[0035] Furthermore, in step S2, signal fusion specifically includes the following steps:

[0036] S21: Feature Normalization

[0037] For each feature, normalization is performed using the min-max normalization method; first, the numerical range of each feature is determined, i.e., the minimum value X is found. min and maximum value X max Then, a normalization formula is used to normalize the features:

[0038]

[0039] Where X′ is the normalized value; each feature of heart rate and skin conductance is calculated one by one; finally, all normalized feature values ​​are output to ensure that the values ​​are within the range of [0,1].

[0040] S22: Weighted Fusion

[0041] Heart rate features can be selected from mean heart rate, square root of heart rate variation, and low-to-high frequency power spectral density, while skin conductance features can be selected from mean, median, variance, standard deviation, and maximum and minimum values ​​of SCR and SCL. The various features of heart rate and skin conductance are then fused into a feature matrix.

[0042] Y1 = [F HR F RMSSD F LF F HF ]

[0043] Y2 = [F mean_SCR F mean_SCL ,…,F max_SCR F max_SCL ]

[0044] In this model, each row of the feature matrix represents a sample. Then, a target vector y is constructed, which sets the driver's trust state as the target vector y, where 0 represents no trust, 1 represents moderate trust, and 2 represents excessive trust. The target vector is then combined with the feature matrix to construct a training set (features Y1, Y2 and target vector y) for model training.

[0045] Machine learning training is used to automatically determine feature weights; considering the non-linear relationship between physiological signals and trust levels, a decision tree method is employed; after training, feature weights are extracted from the model; the weights ω1, ω2, ..., ω of the extracted n features are assigned... n-1 ,ω n For feature fusion, the fusion formula is as follows:

[0046] F fused1 =ω1×F HR +…+ω4×F HF

[0047] F fused2 =ω5×F mean_SCR+…+ω n-1 ×F max_SCR +ω n ×F max_SCL

[0048] Among them, F HR F mean_SCR F mean_SCL Let F be the feature vector of heart rate and skin conductance. fused1 F fused2 These are the fusion characteristics of heart rate and skin conductance, respectively.

[0049] Furthermore, step S3 specifically includes: fusing the heart rate and skin conductance features into a single feature matrix Y:

[0050] Y = [F fused1 F fused2 ]

[0051] Since the first layer of the trust evaluation model constructed by this method is a convolutional neural network, for one-dimensional data or feature matrices, the input shape needs to be adjusted to a 4D tensor. The data after signal fusion is a 2D array, specifically [number of samples, number of features]. The feature matrix needs to be reshaped by adding width and number of channels after the number of samples and number of features, and then raising it to a 4D tensor, specifically [number of samples, number of features, width, number of channels]. The width and number of channels are then filled with 1.

[0052] Furthermore, in step S4, constructing the trust assessment model specifically includes:

[0053] (1) Convolutional module

[0054] First, the original features are extracted using a convolutional module, which maps the features to different feature spaces. Then, convolutional kernels are used in the convolutional layers to extract spatial features and learn useful information from the data. The fused features are then passed through a 2D-CNN with stride s and padding p, and then through a Rectified Linear Unit (ReLU) layer.

[0055] (2) Deep Channel Attention Module

[0056] In this module, multi-layer efficient channel attention (ECA) is used to extract deeper local features;

[0057] First, spatial dimensionality is compressed while preserving key information from different channels; then, Global Average Pooling (GAP) is used, which allows for rapid computation of features along the channel dimension while avoiding dimensionality reduction; the next part utilizes channel attention to capture local cross-channel interactions; given aggregated features y:

[0058] ω=σ(Wy )

[0059] Where ω represents the feature vector after nonlinear transformation, and σ(·) represents a nonlinear activation function used to increase the expressive power of the model; the ECA module uses the bandpass matrix W k Let's learn about channel attention, W k It contains k×C parameters, where k represents the number of neighboring nodes and C represents the number of channels; by doing so, y i The weights will only consider his k neighbors, so the following result can be obtained:

[0060]

[0061] Where, ω i This represents the final attention weight for the i-th channel, which determines the importance of that channel in the feature output. This represents the weight of the i-th channel on the j-th neighbor channel. This represents the feature value of the j-th neighbor channel, which belongs to the neighbor set of channel i. Indicates y i The set of k adjacent channels; this method uses one-dimensional convolution, where k represents the size of the convolution kernel; to avoid unnecessary computation, the size of the convolution kernel depends on the channel dimension C. The size of the convolution kernel k can be adaptively determined:

[0062]

[0063] Among them, |t| odd γ represents the nearest odd number of t, and γ is used to control the kernel size and computational cost; y and b are set to 2 and 1 respectively; after convolution, the extracted features are non-linearly transformed by the activation function to facilitate normalization in the next step; the final step is to scale the original input using these normalized features to emphasize more information elements and suppress fewer correlated elements; a residual network is also used between each ECA layer to prevent gradient explosion.

[0064] (3) Transformer module

[0065] Considering the global correlation in physiological electrical signals, extracting global features from multimodal information is extremely beneficial for improving the robustness of the model. This module selects a model from Transformer to extract global features, compensating for the limited receptive field of the convolution module; the output of the previous module is linearly transformed and then fed into the query Q, key K, and value V elements.

[0066] The encoder architecture utilizes dot product operations to determine the correlation between various features. To reduce the risk of gradient vanishing and ensure the stability of the training process, a scaling factor is introduced. Subsequently, the resulting matrix is ​​normalized using the Softmax function to generate a weighted matrix. This attention score is then applied to the V matrix through another dot product, ultimately forming a weighted representation that reflects the model's attention-focusing mechanism. In the case of self-attention, the contents of Q, K, and V remain consistent. The calculation process is as follows:

[0067]

[0068] Where, d k The length of the feature is represented by h. Furthermore, a multi-sensor head attention (MHA) mechanism is employed to further enhance feature diversity. In this mechanism, the feature is uniformly divided into h parts; each part is individually fed into the self-attention module for computation; finally, the results of each part are concatenated to obtain the final output. The entire process can be represented as:

[0069] MHA(Q,K,V)=[head0;...;head h-1 ],

[0070] head l =Attention(Q) l ,K l V l )

[0071] Among them, Q l ,K l V l These represent the query, key, and value matrix of the l-th attention head, respectively. Furthermore, a residual connection is established between the feedforward layer and the MHA layer, followed by layer normalization (LN) to enhance the model's fitting ability. The input and output dimensions of this module remain unchanged. The encoder structure is repeated N times throughout the entire Transformer layer.

[0072] (4) Regression Prediction Module

[0073] This module transforms features into continuous output values ​​bounded between 0 and 1. First, the LayerNorm module normalizes each input feature. After normalization, an AvgPool layer reduces data dimensionality while preserving essential information. Then, a linear layer performs a linear transformation on the data, mapping it to the singular output value of each instance to conform to the regression objective. Finally, the tanh activation function is applied and further adjusted to shift and scale the output range to [0,1], ensuring that the module's output is best suited for regression tasks requiring bounded continuous prediction.

[0074] Furthermore, in step S5, the regression output is mapped to the interval [0,2] using the following linear mapping:

[0075] y′=a·y+b

[0076] Where y′ is the mapped value, a is the scaling factor, and b is the offset; since the output range of the regression prediction module is [0,1], a and b are 2 and 0 respectively; then threshold mapping is performed, and the output after mapping is converted into a classification result using a threshold. If the output is less than 0.5, it is predicted as 0; if the output is less than 1.5 but greater than 0.5, it is predicted as 1; if the output is greater than 1.5, it is predicted as 2.

[0077] The beneficial effects of this invention are as follows: This invention constructs a trust assessment model and combines the fusion analysis of heart rate and skin electrical signals to predict and assess the driver's real-time trust status. This solves the problem in the prior art that the measurement method is easily affected by ambient light interference and visual obstruction, resulting in data loss or failure and reduced model measurement accuracy. This greatly improves the predictive accuracy of the trust model.

[0078] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0079] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein:

[0080] Figure 1 This is a flowchart of the driver trust status assessment method based on physiological signals according to the present invention;

[0081] Figure 2 This is a flowchart of a trust assessment model. Detailed Implementation

[0082] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0083] Please see Figures 1-2 This invention provides a method for assessing driver trust status based on physiological signals, such as... Figure 1 As shown, the method specifically includes the following steps:

[0084] 1. Data processing and feature extraction methods

[0085] (1) Methods for heart rate signal data processing and feature extraction

[0086] First, the RR variability signal (RRV) is extracted from the ECG, and HRV = 60 / RRV is calculated. The RR interval is estimated as the time difference between R consecutive peaks. The RRV signal (and HRV signal) mainly has two components: 1) a low-frequency baseline component, which undergoes some abrupt changes due to trust-induced events; 2) a periodic component, including parasympathetic and sympathetic activity. The latter can be modeled using a P-order autoregressive process (AR). The RR interval at time i is denoted as rr. i :

[0087]

[0088] Where b and a are the baselines for the signal and AR coefficients, respectively, and the noise is Gaussian distributed. Baseline b is a variable affected by trust-inducing events and used for trust detection. The i-th index indicates that these variables are time-varying, i.e.:

[0089] b i =b i-1 +ε b

[0090]

[0091] Where, ε b and ε a It is Gaussian distributed noise. The state-space form is:

[0092] x i =x i-1 +R

[0093] y i =H i x i +Q

[0094] Where x i =[a1,a2,...,a P ,b i ], y i =rr i R represents process noise, Q represents observation noise, and matrix H i =diag[rr i-1 ,rr i-2,...,rr i-P [1]. In order to estimate x i For each new sample y i Both can use the ordinary Kalman filter equation:

[0095] x i =x i-1

[0096]

[0097] Where K is the Kalman gain matrix, x i and V i These are the Kalman filter estimates of the hidden state and the error covariance matrix at time step i, respectively. To better estimate the hidden state of the model, this method uses a Kalman smoother. The Kalman smoothing update equation is:

[0098]

[0099] in, and Kalman smoothing estimates of the hidden state and error covariance matrix at time step i are given. This method uses windows of a specific length to compute features from the physiological signal. Given the current information, the Kalman smoother can be applied to each new window only for the past observations contained within that window. Initial estimates of the AR coefficients and observation noise R, as well as the baseline, are extracted using the initial segment of the signal. The first 5 minutes of the signal are used for all initializations. This method uses a diagonal Q, setting the elements corresponding to the AR coefficients to 10. -8 The element corresponding to the baseline is set to 10. -6 In a Kalman filter, the baseline parameter b varies over time, therefore the average baseline value b needs to be calculated within a specific window. i This method denotes the newly emerging feature as B. k Where k is the window indicator, which inherits from b i The behavior of B is characterized by changes controlled by two components: 1) very slow changes and 2) association with trust-inducing events. Since the first component changes very slowly, this method assumes that B... k N(u) follows a Gaussian distribution k Therefore, under external stimuli, it is expected to increase rapidly, i.e.:

[0100]

[0101] Where δ represents the average increase in the centrality of trust events. k It is also time-varying. k The estimation was performed in a BN environment, using Kalman filtering and b iIt is derived from the estimated decomposition.

[0102] (2) Skin Electrodermal Signal Data Processing and Feature Extraction Methods

[0103] First, the EDA signal is downsampled to 1 Hz and then smoothed using a smoothing filter. The EDA signal consists of two main components: a slowly changing component, namely the skin conductance level (SCL), and a component associated with a sudden increase in confidence, namely the skin conductance response (SCR). These rapidly increasing metrics are measured using the first absolute difference (FAD):

[0104]

[0105] The s here i Let i be the EDA at time i. The effect of SCL on SCR is eliminated by dividing the FAD measure by the current SCL level. This method can be easily applied to real-time estimation. The transformation applied to the FAD measure is 1 / SCL. l It is SCL l The nonlinear function, and This function is in SCL l For large (1 / SCL) l When <1), FAD is reduced, and SCL is reduced. l For small (1 / SCL) l >1) amplifies FAD. This method uses SCL. l A logistic function:

[0106]

[0107] The transformed FAD(nFAD) is nFAD = FAD·g(SCL) l The parameters α and β were optimized to achieve a higher relevance to the trust metric derived in the annotation.

[0108] 2. Signal fusion

[0109] (1) Feature normalization

[0110] For each feature, normalization is performed using the min-max normalization method. First, the numerical range of each feature is determined, i.e., the minimum value X is found. min and maximum value X max Then, a normalization formula is used to normalize the features:

[0111]

[0112] Here, X′ is the normalized value. Each feature of heart rate and skin conductance is calculated individually. Finally, all normalized feature values ​​are output, ensuring the values ​​are within the range [0,1].

[0113] (2) Weighted fusion

[0114] Heart rate features can be selected from mean heart rate, square root of heart rate variation, low- and high-frequency power spectral density, etc. Skin conductance features can be selected from mean, median, variance, standard deviation, maximum and minimum values ​​of SCR and SCL, etc. The various features of heart rate and skin conductance are then fused into a feature matrix:

[0115] Y1 = [F HR F RMSSD F LF F HF ]

[0116] Y2 = [F mean_SCR F mean_SCL ,…,F max_SCR F max_SCL ]

[0117] In this model, each row of the feature matrix represents a sample. A target vector y is then constructed, where the driver's trust state is set as the target vector y, where 0 represents distrust, 1 represents moderate trust, and 2 represents excessive trust. The target vector is then combined with the feature matrix to construct a training set (features Y1, Y2 and target vector y) for model training.

[0118] This method uses machine learning training to automatically determine feature weights. Considering the non-linear relationship between physiological signals and trust levels, a decision tree approach is employed. After training, feature weights are extracted from the model. The weights ω1, ω2, ..., ω of the extracted n features are then assigned... n-1 ,ω n For feature fusion, the fusion formula is as follows:

[0119] F fused1 =ω1×F HR +…+ω4×F HF

[0120] F fused2 =ω5×F mean_SCR +…+ω n-1 ×F max_SCR +ω n ×F max_SCL

[0121] Among them, F HR F mean_SCR F mean_SCL These are the feature vectors of heart rate and skin conductance.

[0122] 3. Model Input Construction

[0123] The combined features of heart rate and skin conductance are fused into a single feature matrix Y:

[0124] Y = [F fused1 F fused2 ]

[0125] Since the first layer of the trust evaluation model constructed by this method is a convolutional neural network, for one-dimensional data or feature matrices, the input shape needs to be adjusted to a 4D tensor. The data after signal fusion is a 2D array, specifically [number of samples, number of features]. The feature matrix needs to be reshaped by adding width and number of channels after the number of samples and number of features, thus increasing it to a 4D tensor, specifically [number of samples, number of features, width, number of channels], and padding the width and number of channels with 1.

[0126] 4. Trust Assessment Model

[0127] The model structure is as follows: Figure 2 As shown, it includes:

[0128] (1) Convolutional module

[0129] First, the original features are extracted using a convolutional module, mapping the features to different feature spaces. Then, spatial features are extracted using convolutional kernels in the convolutional layers, learning useful information from the data. The fused features are passed through a 2D-CNN with a stride of s and padding of p, and then through a Rectified Linear Unit (ReLU) layer.

[0130] (2) Deep Channel Attention Module

[0131] In this module, Efficient Channel Attention (ECA) is used to extract deeper local features.

[0132] First, spatial dimensionality is compressed while preserving key information from different channels. Global Average Pooling (GAP) is then used, allowing for rapid computation of features along the channel dimension while avoiding dimensionality reduction. The next part leverages channel attention to capture local cross-channel interactions. Given aggregated features y:

[0133] ω=σ(W y )

[0134] The ECA module uses a bandpass matrix W k Let's learn about channel attention, W k It contains k×C parameters, where k represents the number of neighboring nodes and C represents the number of channels. By doing so, y iThe weights will only consider his k neighbors, so the following result can be obtained:

[0135]

[0136] in, Indicates y i The set of k neighboring channels. This method is implemented using one-dimensional convolution, where k represents the kernel size. To avoid unnecessary computation, the kernel size depends on the channel dimension C. The kernel size k can be adaptively determined:

[0137]

[0138] Among them, |t| odd Let represent the nearest odd number of t. Set y and b to 2 and 1 respectively. After convolution, the extracted features undergo a non-linear transformation through an activation function to facilitate normalization in the next step. The final step scales the original input using these normalized features, emphasizing more informative elements and suppressing fewer correlated ones. A residual network is also used between each ECA layer to prevent gradient explosion.

[0139] (3) Transformer module

[0140] Considering the global correlation in physiological electrical signals, extracting global features from multimodal information is extremely beneficial for improving the robustness of the model. This module selects a Transformer model to extract global features, compensating for the limited receptive field of the convolutional module. The output of the preceding module is linearly transformed and then fed into the Q, K, V (query, key, value) components.

[0141] The encoder architecture utilizes dot product operations to determine the correlations between various features. To mitigate the risk of gradient vanishing and ensure the stability of the training process, a scaling factor is introduced. Subsequently, the resulting matrix is ​​normalized using the Softmax function to generate a weighted matrix. This attention score is then applied to the V matrix through another dot product, ultimately forming a weighted representation that reflects the model's attention-focusing mechanism. In the case of self-attention, the contents of Q, K, and V remain consistent. The calculation process is as follows:

[0142]

[0143] Where, d kThe length of the feature is represented by h. Furthermore, a multi-sensor head attention (MHA) mechanism is employed to further enhance feature diversity. In this mechanism, the feature is uniformly divided into h parts. Each part is individually fed into a self-attention module for computation. Finally, the results of each part are concatenated to obtain the final output. The entire process can be represented as:

[0144] MHA(Q,K,V)=[head0;...;head h-1 ],

[0145] head l =Attention(Q) l ,K l V l )

[0146] Among them, Q l K l V l These represent the query, key, and value matrix of the l-th attention head, respectively. Furthermore, a residual connection is established between the feedforward layer and the MHA layer, followed by layer normalization (LN) to enhance the model's fitting ability. The input and output dimensions of this module remain unchanged. The encoder structure is repeated N times throughout the Transformer layer.

[0147] (4) Regression Prediction Module

[0148] This layer transforms features into continuous output values ​​bounded between 0 and 1. First, the LayerNorm module normalizes the input features. After normalization, an AvgPool layer reduces data dimensionality while preserving essential information. Then, a linear layer performs a linear transformation on the data, mapping it to the singular output value for each instance to conform to the regression objective. Finally, the tanh activation function is applied and further adjusted to shift and scale the output range to [0,1], ensuring that the layer's output is best suited for regression tasks requiring bounded continuous prediction.

[0149] 5. Model Output

[0150] This method defines the target vector as [0, 1, 2], where 0 represents no trust, 1 represents moderate trust, and 2 represents excessive trust. It is necessary to establish a correlation between the output of the regression prediction module and the target vector. First, the regression output is mapped to the interval [0, 2] using a linear mapping:

[0151] y′=a·y+b

[0152] Where t′ is the mapped value, a is the scaling factor, and b is the offset. Since the output range of the regression prediction module is [0,1], a and b are 2 and 0, respectively. Then, threshold mapping is performed. Based on the mapped output, a threshold is used to convert it into a classification result. If the output is less than 0.5, the prediction is 0; if the output is less than 1.5 but greater than 0.5, the prediction is 1; and if the output is greater than 1.5, the prediction is 2.

[0153] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for assessing driver trust status based on physiological signals, characterized in that, The method specifically includes the following steps: S1: Preprocessing and feature extraction of heart rate and electrodermal signals; S2: Signal fusion: Normalize the heart rate features and skin conductance features extracted in step S1, and then fuse the normalized heart rate features and skin conductance features into a feature matrix respectively. S3: Construct model input; S4: Construct a trust assessment model, including a convolution module, a deep channel attention module, a Transformer module, and a regression prediction module; (1) Convolutional module First, the original features are extracted using a convolutional module, which maps the features to different feature spaces. Then, convolutional kernels are used in the convolutional layers to extract spatial features, learning useful information from the data. The fused features are then processed through a 2D-CNN with stride s and padding p, and then through a rectified linear unit layer. (2) Deep channel attention module In this module, multi-layer efficient channel attention (ECA) is used to extract deep local features; First, spatial dimensionality is compressed while retaining key information from different channels; then global average pooling is used; the next part utilizes channel attention to capture local cross-channel interactions; given aggregated feature y: σ （ ） in, This represents the eigenvector after nonlinear transformation. This represents a non-linear activation function used to increase the expressive power of the model; the ECA module uses a bandpass matrix. Let's learn about channel attention. It contains k × C parameters, where k represents the number of neighboring nodes and C represents the number of channels; by doing so, The weights will only consider his k neighbors, so the following result can be obtained: in, Indicates the first i The final attention weights for each channel, Indicates the first i The first channel in the j The weight of each neighbor channel Indicates the first j The feature values ​​of each neighboring channel belong to the channel. i The neighborhood group, express The set of k adjacent channels; this method uses one-dimensional convolution, where k represents the size of the convolution kernel; to avoid unnecessary computation, the size of the convolution kernel depends on the channel dimension C; the size of the convolution kernel k can be adaptively determined: odd Among them, |t| odd Denotes the nearest odd number of t. Used to control the size of the convolution kernel and computational cost; y and b are set to 2 and 1 respectively; after convolution, the extracted features are non-linearly transformed by the activation function to facilitate normalization in the next step; the final step is to scale the original input using these normalized features; a residual network is also used between each ECA layer to prevent gradient explosion. (3) Transformer module This module selects a model from Transformer to extract global features, making up for the limited receptive field of the convolution module; the output of the previous module is linearly transformed and then fed into the query Q, key K, and value V elements. The encoder architecture uses dot product operations to determine the correlation between various features; then, the resulting matrix is ​​normalized using the Softmax function to generate a weighted matrix; this attention score is then applied to the V matrix through another dot product, ultimately forming a weighted representation that reflects the model's attention-focusing mechanism; in the case of self-attention, the contents of Q, K, and V remain consistent; the calculation process is as follows: in, The length of the feature is represented by h. Furthermore, a multi-sensor head attention (MHA) mechanism is employed to further enhance feature diversity. In this mechanism, the feature is uniformly divided into h parts; each part is individually fed into the self-attention module for computation; finally, the results of each part are concatenated to obtain the final output. The entire process is represented as follows: in, These represent the query, key, and value matrix of the l-th attention head, respectively; in addition, a residual connection is made between the feedforward layer and the MHA layer, followed by layer normalization; the encoder structure is repeated N times throughout the Transformer layer; (4) Regression Prediction Module This module transforms features into continuous output values ​​bounded between 0 and 1. First, the LayerNorm module normalizes each input feature. After normalization, an AvgPool layer reduces the data dimensionality while preserving essential information. Then, a linear layer performs a linear transformation on the data, mapping it to the singular output value of each instance to conform to the regression objective. Finally, the tanh activation function is applied and further adjusted to shift and scale the output range to [0,1], ensuring that the module's output is suitable for regression tasks requiring bounded continuous prediction. S5: Model Output: Define the target vector as [0,1,2], where 0 represents no trust, 1 represents moderate trust, and 2 represents excessive trust; then establish a correlation between the output of the regression prediction module and the target vector, that is, use linear mapping to map the regression output to the interval [0,2]; finally, perform threshold mapping, and use the threshold to convert the mapped output into a classification result.

2. The driver trust status assessment method according to claim 1, characterized in that, In step S1, the heart rate signal is preprocessed and features are extracted, specifically including: firstly, the RR variability signal RRV is extracted from the ECG, and HRV = 60 / RRV is calculated; the RR period is estimated as the time difference between R consecutive peaks; the RRV signal and the HRV signal mainly contain two components: 1) a low-frequency baseline component, which is a mutation caused by trust-induced events; 2) a periodic component, including parasympathetic and sympathetic activity; the latter is modeled using a P-order autoregressive process AR; the RR interval at time i is denoted as : in, b and a These are the baselines for the signal and the AR coefficient, respectively. c It is a random error term, representing all the variation not explained by the model; the noise is Gaussian distributed; baseline. b These are variables influenced by trust-inducing events and used for trust detection; the first... i These indicators indicate that these variables are time-varying, that is: in, and It is Gaussian distributed noise; the state-space form is: in, = [ a 1, a 2,..., a P , b i ], = , R Let Q be the process noise and Q be the observation noise. The matrix... = diag[ rr i - 1 , rr i - 2 , ... , rr i - P [1]; In order to estimate For each new sample Using the ordinary Kalman filter equation: i-1 S K i ( ) i ( ) Where K is the Kalman gain matrix; S is the matrix used to calculate the Kalman gain, representing the innovation covariance and the uncertainty of the current observation; and These are the hidden state and the error covariance matrix at the time step, respectively. i Kalman filter estimation, i and i They are at the time step i The updated hidden state estimate and error covariance matrix are obtained. To better estimate the hidden state of the model, this method uses a Kalman smoother. The Kalman smoothing update equation is: i i （ i i+1 ） i ( i+1 i+1 ) T i in, i and i They are time steps i Kalman smoothing estimation of the hidden state and error covariance matrix. This is the smoothing gain matrix, used for smoothing the update of the estimate; T denotes the transpose of the matrix, which is used when updating the error covariance matrix; a window of a specific length is used to compute features from the physiological signal, and given the current information, a Kalman smoother is applied to each new window only for the past observations contained within the window; initial estimates of the AR coefficients and observation noise R, as well as the initial estimate of the baseline, are extracted using the initial segment of the signal; the first 5 minutes of the signal are used for all initialization; the elements corresponding to the AR coefficients are set to 10 using the diagonal Q. -8 The element corresponding to the baseline is set to 10. -6 In a Kalman filter, the baseline parameter b changes over time, therefore the average baseline value b needs to be calculated within a specific window. i The new features that appear will be denoted as Where k is the window indicator, which inherits from The behavior of change is controlled by two parts: 1) very slow change and 2) related to trust-inducing events; since the first component changes very slowly, it is assumed that... Follows a Gaussian distribution Therefore, under external stimuli, it is expected to increase rapidly, that is: Where δ is the average increase in the centrality of trust events; It is time-varying. The estimation was performed in a BN environment, using Kalman filtering and b i It is derived from the estimated decomposition.

3. The driver trust status assessment method according to claim 1, characterized in that, In step S1, the EDA signal is preprocessed and features are extracted, specifically including: first, the EDA signal is downsampled to 1Hz, and then a smoothing filter is used; the EDA signal consists of two parts: a slowly changing component, namely the skin conductance level SCL, and a component related to a sudden increase in confidence, namely the skin conductance response SCR; the first absolute difference (FAD) is used to measure the rapidly increasing indicators. FAD in, It is a moment i EDA, N This represents the total number of time points; the method of eliminating the influence of SCL on SCR by dividing the FAD measure by the current SCL level can be easily applied to real-time estimation; the transformation applied to the FAD measure using this method is 1 / It is The nonlinear function, and = (1 / N) ;use A logistic function: The transformed FAD (nFAD) is nFAD = FAD · g ( We optimize the parameters α and β to achieve a higher relevance to the trust metric derived from the annotations.

4. The driver trust status assessment method according to claim 3, characterized in that, In step S2, signal fusion specifically includes the following steps: S21: Feature Normalization For each feature, normalization is performed using the min-max normalization method; first, the numerical range of each feature is determined, i.e., the minimum value is found. and maximum value Then, a normalization formula is used to normalize the features: in, These are the normalized values; each feature of heart rate and skin conductance is calculated one by one; finally, all normalized feature values ​​are output, ensuring that the values ​​are within the range of [0,1]. S22: Weighted Fusion Heart rate features were selected from mean heart rate, square root of heart rate variation, and low-to-high frequency power spectral density. Skin conductance features were selected from mean, median, variance, standard deviation, and maximum and minimum values ​​of SCR and SCL. The various features of heart rate and skin conductance were then fused into feature matrices. Each row of the feature matrix represents a sample; then, a target vector y is constructed, which sets the driver's trust state as the target vector y, where 0 represents no trust, 1 represents moderate trust, and 2 represents excessive trust; then, the target vector is combined with the feature matrix to construct a training set for model training. Machine learning training is used to automatically determine feature weights; considering the non-linear relationship between physiological signals and trust levels, a decision tree method is employed; after training, feature weights are extracted from the model; the extracted weights are then... n Weights of each feature For feature fusion, the fusion formula is as follows: in, These are the feature vectors for heart rate and skin conductance. , These are the fusion characteristics of heart rate and skin conductance, respectively.

5. The driver trust status assessment method according to claim 4, characterized in that, Step S3 specifically includes: fusing the heart rate and skin conductance features into a single feature matrix Y. For one-dimensional data or feature matrices, the input shape needs to be adjusted to a 4D tensor. The data after signal fusion is a 2D array, specifically [number of samples, number of features]. The feature matrix needs to be reshaped by adding width and number of channels after the number of samples and number of features, and then raising it to a 4D tensor, specifically [number of samples, number of features, width, number of channels]. The width and number of channels are then filled with 1.

6. The driver trust status assessment method according to claim 1, characterized in that, In step S5, the regression output is mapped to the interval [0,2] using the following linear mapping: in, It is the mapped value. a It is a scaling factor. b It is the offset; since the output range of the regression prediction module is [0,1], a and b are 2 and 0 respectively; then threshold mapping is performed, and the output after mapping is converted into a classification result using the threshold. If the output is less than 0.5, it is predicted as 0; if the output is less than 1.5 but greater than 0.5, it is predicted as 1; if the output is greater than 1.5, it is predicted as 2.

Images