Driver trust state evaluation method based on physiological signals

Abstract

The invention relates to a driver trust state evaluation method based on physiological signals, and belongs to the technical field of intelligent driving. The method comprises the following steps: S1, carrying out preprocessing and feature extraction on a heart rate signal and a skin electric signal; s2, signal fusion: normalizing the heart rate features and the skin electricity features extracted in the step S1, and then fusing each normalized heart rate feature and each normalized skin electricity feature into a feature matrix; s3, constructing model input; s4, constructing a trust evaluation model, wherein the trust evaluation model comprises a convolution module, a deep channel attention module, a Transform module and a regression prediction module; s5, model output: defining a target vector as [0, 1, 2], and then establishing association between the output of a regression prediction module and the target vector, that is, adopting linear mapping to map regression output to an interval of [0, 2]; and finally, carrying out threshold mapping, and converting the output into a classification result by using a threshold according to the mapped output.

Description

Technical Field

[0001] The present invention belongs to the technical field of intelligent driving and relates to a driver trust state assessment method based on physiological signals. Background Art

[0002] At present, the Society of Automotive Engineers (SAE) has proposed a level classification for the degree of automation of vehicle driving. This classification system divides the degree of automation of vehicle driving into six levels, from no automation (L0) to full automation (L5). Among them, L3 autonomous driving is called conditional autonomous driving. The autonomous driving system can continuously perform all dynamic driving tasks within its designed operating conditions, but the user needs to be ready to take over the driving task at any time. At present, the autonomous driving level of the vehicle-mounted system of mass-produced cars in my country is transitioning from L2 to L3, and the autonomous driving level of some vehicle-mounted systems has reached L4 in limited scenarios. Autonomous driving is an important direction for the development of the automotive industry in recent years, but the realization of complete unmanned autonomous driving is not achieved overnight and requires a long period of development. Therefore, autonomous driving cars will be in a human-machine co-driving mode for a long time in the future. At this stage, human-machine trust will be an important factor affecting the coordination between drivers and autonomous driving cars.

[0003] The concept of trust originally originated from sociology, which covers the relationship between the trustor and the trustee, meaning that the trustee can be aware of the trust of the trustor, take the interests of the trustor as the priority, and act out of good intentions for the trustor. In the field of automation trust, the definition of trust proposed by scholars such as Lee is widely accepted. They define trust as "the attitude of an individual (such as a driver) who believes that an agent (such as an autonomous driving system) can help him achieve a certain goal (such as a driving task) in an uncertain or vulnerable situation." Research on automation trust covers the identification and verification of trust influencing factors, the measurement and modeling of automation trust, and the calibration of automation trust. There are different research systems on automation trust, such as human-robot interaction trust, human-computer interaction trust, and human-autonomous driving interaction trust. In human-robot trust, the trustor is a human and the trustee is a robot. It is difficult for humans and robots to have mutual emotional interactions, so human-robot trust is more often reflected as instant trust based on people's instant judgment of robots. Human-computer trust refers to the trust relationship between humans and computer systems, with the trustor being a human and the trustee being a computer. Human-autonomous driving interaction trust refers to the trust relationship between the driver and the autonomous driving system in the field of autonomous driving, where the driver is the trustor and the autonomous driving system is the trustee.

[0004] Trust in human-autonomous driving interaction involves three states: insufficient trust, excessive trust, and appropriate trust. Insufficient trust will cause drivers to reduce their use of autonomous driving functions, increase driving supervision fatigue, and reduce driving comfort; excessive trust may reduce drivers' situational awareness of autonomous driving cars, posing safety risks. Appropriate trust, also known as calibrated trust, refers to the consistency between the driver's subjective actual trust level and the objective trust level. It aims to ensure that the driver correctly uses the autonomous driving foundation of the autonomous driving car and is the core goal of human-computer interaction trust research in the field of autonomous driving.

[0005] In the context of autonomous driving, the study of human-machine trust plays an important role in promoting the development of the current automotive industry and has significant research value in practical driving applications. In the process of autonomous driving, the driver's trust must be calibrated. The first condition is to be able to grasp the driver's real-time trust level. Existing methods for real-time measurement of trust mostly use trust scale measurement methods. This type of method requires interrupting the experimental process when used, resulting in high intrusion during measurement. In addition, the measurement method using behavioral monitoring is easily affected by ambient light interference and line of sight occlusion, resulting in missing or invalid data and reducing the accuracy of model measurement. Therefore, in view of the high intrusion and low accuracy of existing trust measurement methods, there is an urgent need for a method that can accurately evaluate the real-time trust status of the driver. Summary of the invention

[0006] In view of this, an object of the present invention is to provide a driver trust state assessment method based on physiological signals, so as to improve the accuracy of trust measurement.

[0007] In order to achieve the above object, the present invention provides the following technical solutions:

[0008] A method for evaluating driver trust status based on physiological signals specifically comprises the following steps:

[0009] S1: Preprocessing and feature extraction of heart rate signal and skin electrical signal;

[0010] S2: signal fusion: normalize the heart rate features and skin electrical features extracted in step S1, and then fuse the normalized heart rate features and skin electrical features into a feature matrix;

[0011] S3: construct model input;

[0012] S4: Build a trust assessment model, including convolution module, deep channel attention module, Transformer module and regression prediction module;

[0013] S5: Model output: The target vector is defined as [0, 1, 2], where 0 represents distrust, 1 represents moderate trust, and 2 represents overtrust. Then, an association is established between the output of the regression prediction module and the target vector, that is, a linear mapping is used to map the regression output to the interval [0, 2]. Finally, threshold mapping is performed, and the threshold is used to convert the mapped output into a classification result.

[0014] Further, in step S1, the heart rate signal is preprocessed and feature extracted, specifically including: first extracting the RR variability signal RRV from the ECG, and calculating HRV=60 / RRV; the RR period is estimated as the time difference between R consecutive peaks; the RRV signal and the HRV signal mainly contain two components: 1) a low-frequency baseline component, some mutations due to trust-induced events; 2) a periodic component, including parasympathetic and sympathetic nerve activities; the latter can be modeled by a P-order autoregressive process AR; the RR interval at time i is denoted as rr i :

[0015]

[0016] Among them, b and a are the baselines of the signal and AR coefficient respectively, c is a random error term representing all the variations not explained by the model, and the noise is Gaussian distributed; the baseline b is the variable affected by the trust-inducing event and used for trust detection; the i-th indicator indicates that these variables are time-varying, that is:

[0017] b i =b i-1 +ε b

[0018]

[0019] Among them, ε b and ε a is Gaussian distributed noise; the state space form is:

[0020] x i =x i-1 +R

[0021] y i =H i x i +Q

[0022] Among them, x i =[a1,a2,...,a P ,b i ],y i =rr i , R is process noise, Q is observation noise, matrix H i =diag[rr i-1,rr i-2 ,...,rr i-P ,1]; In order to estimate x i , for each new sample y i The ordinary Kalman filter equation can be used:

[0023] x i =x i-1

[0024]

[0025] Among them, K is the Kalman gain matrix, S is the matrix for calculating the Kalman gain, represents the innovation covariance, represents the uncertainty of the current observation, and x i and V i are the Kalman filter estimates of the hidden state and error covariance matrix at time step i, and is the updated hidden state estimate and error covariance matrix at time step i; in order to better estimate the hidden state of the model, this method uses a Kalman smoother. The Kalman smoothing update equation is:

[0026]

[0027] in, and are the Kalman smoothing estimates of the hidden state and error covariance matrix at time step i, V i * is the smoothing gain matrix used to update the smoothing estimate, T represents the transpose of the matrix, which is used when updating the error covariance matrix; this method uses a window of a specific length to calculate features from the physiological signal, and given the information at that time, the Kalman smoother can be applied to each new window only on the past observations contained in the window; the initial segment of the signal is used to extract the initial estimates of the AR coefficients and the observation noise R, as well as the initial estimate of the baseline; the first 5 minutes of the signal are used for all initializations; this method uses the diagonal Q and sets the elements corresponding to the AR coefficients to 10 -8 , the element corresponding to the baseline is set to 10 -6 ; In the Kalman filter, the baseline parameter b changes over time, so it is necessary to calculate the average baseline value b within a specific window i ; The new feature that appears is recorded as B k , where k is the window indicator, which inherits b i The changing behavior of is controlled by two components: 1) very slowly changing and 2) related to trust-inducing events; since the first component is very slowly changing, this method assumes that B k Obeying Gaussian distribution N(u k,σ0), and is therefore expected to increase rapidly in the case of external stimulation, i.e.:

[0028]

[0029] Where δ is the average increase in the central rate of trust events; u k It is also time-varying, u k The estimation is performed in the BN environment by Kalman filtering and b i The estimate is decomposed.

[0030] Further, in step S1, the skin electrical signal is preprocessed and feature extracted, specifically including: first downsampling the skin electrical signal to 1Hz, and then using a smoothing filter; the EDA signal includes two parts: a slowly changing component, namely the skin conductance level SCL, and a component related to a sudden increase in trust, namely the skin conductance response SCR; the first absolute difference FAD is used to measure these rapidly increasing indicators:

[0031]

[0032] Among them, s i is the EDA at time i, and N is the total number of time points; the method of dividing the FAD measure by the current SCL level is used to eliminate the influence of SCL on SCR. This method can be easily applied to real-time estimation; the transformation of this method applied to the FAD measure is 1 / SCL l , it is SCL l is a nonlinear function of This function is in SCL l For large (1 / SCL l <1) reduces the FAD, and at SCL l is smaller than (1 / SCL l >1) when the FAD is enlarged. Use SCL l A logistic function of :

[0033]

[0034] The transformed FAD (nFAD) is nFAD = FAD g (SCL l ), the parameters α and β are optimized to obtain a higher correlation with the trust metric derived from the annotations.

[0035] Further, in step S2, signal fusion specifically includes the following steps:

[0036] S21: Feature Normalization

[0037] For each feature, the minimum-maximum normalization method is used for normalization; first determine the value range of each feature, that is, find the minimum value X min and the maximum value X max , and then use a normalization formula to normalize the features:

[0038]

[0039] Among them, X′ is the normalized value; each feature of heart rate and skin electricity is calculated one by one; finally, all normalized feature values ​​are output to ensure that the values ​​are in the range of [0,1];

[0040] S22: Weighted Fusion

[0041] Heart rate features can be selected from average heart rate, square root of heart rate variability, and low-high frequency power spectrum density, etc. Skin electrical features can be selected from mean, median, variance, standard deviation, maximum and minimum values ​​of SCR and SCL, etc. Each feature of heart rate and each feature of skin electrical are fused into a feature matrix:

[0042] Y1=[F HR , F RMSSD , F LF , F HF ]

[0043] Y2=[F mean_SCR , F mean_SCL ,…,F max_SCR , F max_SCL ]

[0044] Each row of the feature matrix represents a sample. Then, a target vector y is constructed, and the driver’s trust status is set as the target vector y, where 0 represents distrust, 1 represents moderate trust, and 2 represents excessive trust. Then, the target vector is combined with the feature matrix to construct a training set (features Y1, Y2 and target vector y) for model training.

[0045] The machine learning training method is used to automatically confirm the feature weights. Considering the nonlinear correspondence between physiological signals and trust levels, the decision tree method is used. After the training is completed, the feature weights are extracted from the model. The weights of the extracted n features ω1, ω2, ···, ω n-1 ,ω n For feature fusion, the fusion formula is as follows:

[0046] F fused1 =ω1×F HR +…+ω4×F HF

[0047] F fused2 =ω5×F mean_SCR+…+ω n-1 ×F max_SCR +ω n ×F max_SCL

[0048] Among them, F HR 、···、F mean_SCR 、F mean_SCL etc. are the characteristic vectors of heart rate and skin electricity, F fused1 、F fused2 They are the fusion features of heart rate and skin electricity respectively.

[0049] Further, step S3 specifically includes: fusing the fusion features of heart rate and skin electricity into a feature matrix Y:

[0050] Y=[F fused1 , F fused2 ]

[0051] Since the first layer of the trust assessment model constructed by this method is a convolutional neural network, for one-dimensional data or feature matrix, the input shape needs to be adjusted to a 4D tensor; the data after signal fusion is a 2D array, specifically [number of samples, number of features], and the feature matrix needs to be reshaped by adding width and number of channels after the number of samples and number of features to promote it to a 4D tensor, specifically [number of samples, number of features, width, channels], and fill the width and number of channels with 1.

[0052] Further, in step S4, building a trust assessment model specifically includes:

[0053] (1) Convolutional module

[0054] First, the convolution module is used to extract the original features, map the features to different feature spaces, and use the convolution kernel in the convolution layer to extract spatial features and learn useful information from the data; the fused features pass through a 2D-CNN with a step size of s and a padding of p, and then pass through a rectified linear unit (ReLU) layer;

[0055] (2) Deep channel attention module

[0056] In this module, multi-layer Efficient Channel Attention (ECA) is used to extract deeper local features;

[0057] The first step is to compress the spatial dimension while retaining the key information of different channels; and use global average pooling (GAP), which allows fast calculation of features along the channel dimension while avoiding dimensionality reduction; the next part uses channel attention to capture local cross-channel interactions; given the aggregate feature y:

[0058] ω=σ(Wy )

[0059] Among them, ω represents the feature vector after nonlinear transformation, σ(·) represents a nonlinear activation function, which is used to increase the expressive power of the model; the ECA module uses the bandpass matrix W k To learn channel attention, W k Contains k×C parameters, where k represents the number of neighbor nodes and C represents the number of channels; by doing so, y i The weight of will only consider its k neighbors, so the following results can be obtained:

[0060]

[0061] Among them, ω i represents the final attention weight of the i-th channel, which determines the importance of the channel in the feature output. represents the weight of the i-th channel on the j-th neighbor channel, represents the eigenvalue of the jth neighbor channel, belonging to the neighbor set of channel i, Represents y i The method is implemented using one-dimensional convolution, where k represents the size of the convolution kernel. To avoid unnecessary calculations, the size of the convolution kernel depends on the channel dimension C. The size of the convolution kernel k can be determined adaptively:

[0062]

[0063] Where |t| odd represents the nearest odd number of t, and γ is used to control the convolution kernel size and the amount of computation; y and b are set to 2 and 1 respectively; after the convolution, the extracted features are nonlinearly transformed through the activation function to facilitate normalization in the next step; the last step is to scale the original input through these normalized features to emphasize more information elements and suppress less relevant elements; between each ECA layer, a residual network is also used to prevent gradient explosion.

[0064] (3) Transformer module

[0065] Considering the global correlation in physiological electrical signals, extracting the global features of multimodal information is extremely beneficial to improving the robustness of the model. This module selects a model from Transformer to extract global features, which makes up for the problem of limited receptive field of convolution module; the output of the previous module is linearly transformed and sent to the query Q, key K, value V components;

[0066] The encoder architecture uses dot product operations to determine the correlation between various features; in order to reduce the risk of gradient vanishing and ensure the stability of the training process, a scaling factor is introduced. The resulting matrix is ​​then normalized by the Softmax function to generate a weighted matrix; this attention score is then applied to the V matrix through another dot product, ultimately forming a weighted representation that reflects the model's focused attention mechanism; in the case of self-attention, the contents of Q, K, and V remain consistent; the calculation process is as follows:

[0067]

[0068] Among them, d k Indicates the length of the feature; in addition, the multi-sensor head attention MHA mechanism is used to further enhance the diversity of features; in this mechanism, the features are evenly divided into h parts; each part is sent to the self-attention module for calculation; finally, the results of each part are spliced ​​to obtain the final output result; the whole process can be expressed as:

[0069] MHA(Q,K,V)=[head0;...;head h-1 ],

[0070] head l =Attention(Q l ,K l ,V l )

[0071] Among them, Q l ,K l ,V l Represent the query, key, and value matrices of the lth attention head, respectively; In addition, a residual connection is made between the feedforward layer and the MHA layer, followed by layer normalization (LN), which enhances the model's fitting ability; The input and output dimensions of this module remain unchanged. Within the entire Transformer layer, the encoder structure is repeated N times.

[0072] (4) Regression prediction module

[0073] This module converts features into continuous output values ​​limited to 0-1; first, the LayerNorm module normalizes each input feature; after normalization, the AvgPool layer is used to reduce the data dimension while retaining the essential information. Subsequently, the data is linearly transformed through the linear layer to map it to a singular output value for each instance to meet the regression objective; the process ends with the application of the tanh activation function and further adjustments to move and scale the output range to [0,1], thereby ensuring that the output of this module is best suited for regression tasks that require bounded continuous predictions.

[0074] Furthermore, in step S5, the regression output is mapped to the interval [0,2], which is implemented by the following linear mapping:

[0075] y′=a·y+b

[0076] Among them, y′ is the mapped value, a is the scaling factor, and b is the offset. Since the output range of the regression prediction module is [0,1], a and b are 2 and 0 respectively. Then threshold mapping is performed, and the threshold is used to convert it into a classification result according to the mapped output. If the output is less than 0.5, it is predicted to be 0, if the output is less than 1.5 and greater than 0.5, it is predicted to be 1, and if the output is greater than 1.5, it is predicted to be 2.

[0077] The beneficial effects of the present invention are as follows: the present invention constructs a trust assessment model, and combines the fused analysis of heart rate and skin electrical signals to predict and assess the driver's real-time trust status, thereby solving the problem in the prior art that the measurement method is easily affected by ambient light interference and line of sight obstruction, resulting in missing or invalid data and reduced model measurement accuracy, thereby greatly improving the prediction accuracy of the trust model.

[0078] Other advantages, objectives and features of the present invention will be described in the following description to some extent, and to some extent, will be obvious to those skilled in the art based on the following examination and study, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0079] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below in conjunction with the accompanying drawings, wherein:

[0080] Figure 1 is a flow chart of a driver trust state assessment method based on physiological signals of the present invention;

[0081] Figure 2 A block diagram of the trust assessment model. DETAILED DESCRIPTION

[0082] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments only illustrate the basic concept of the present invention in a schematic manner, and the following embodiments and features in the embodiments can be combined with each other without conflict.

[0083] See also Figure 1-2 , the present invention provides a driver trust state assessment method based on physiological signals, such as Figure 1 As shown, the method specifically comprises the following steps:

[0084] 1. Data processing and feature extraction methods

[0085] (1) Heart rate signal data processing and feature extraction method

[0086] First, the RR variability signal (RRV) is extracted from the ECG, and HRV = 60 / RRV is calculated. The RR interval is estimated as the time difference between R consecutive peaks. The RRV signal (as well as the HRV signal) has two main components: 1) a low-frequency baseline component, which undergoes some mutations due to trust-inducing events; 2) a periodic component, including parasympathetic and sympathetic nerve activity. The latter can be modeled by a P-order autoregressive process (AR). The RR interval at time i is denoted as rr i :

[0087]

[0088] Where b and a are the baselines of the signal and AR coefficient, respectively, and the noise is Gaussian distributed. The baseline b is the variable affected by the trust-inducing event and used for trust detection. The i-th indicator indicates that these variables are time-varying, namely:

[0089] b i =b i-1 +ε b

[0090]

[0091] Among them, ε b and ε a is Gaussian distributed noise. The state space form is:

[0092] x i =x i-1 +R

[0093] y i =H i x i +Q

[0094] where x i =[a1,a2,...,a P ,b i ],y i =rr i , R is process noise, Q is observation noise, matrix H i =diag[rr i-1 ,rr i-2,...,rr i-P ,1]. In order to estimate x i , for each new sample y i The ordinary Kalman filter equation can be used:

[0095] x i =x i-1

[0096]

[0097] Where K is the Kalman gain matrix, x i and V i are the Kalman filter estimates of the hidden state and error covariance matrix at time step i. In order to better estimate the hidden state of the model, this method uses a Kalman smoother. The Kalman smoothing update equation is:

[0098]

[0099] in, and are Kalman smoothed estimates of the hidden state and error covariance matrix at time step i, respectively. This method uses windows of a specific length to compute features from the physiological signal. Given the information at that time, a Kalman smoother can be applied to each new window only on the past observations contained in the window. An initial segment of the signal is used to extract initial estimates of the AR coefficients and the observation noise R, as well as an initial estimate of the baseline. The first 5 minutes of the signal are used for all initializations. This method uses a diagonal Q, setting the elements corresponding to the AR coefficients to 10 -8 , the element corresponding to the baseline is set to 10 -6 In the Kalman filter, the baseline parameter b changes over time, so it is necessary to calculate the average baseline value b within a specific window. i In this method, the new features that appear are recorded as B k , where k is the window indicator, which inherits b i The changing behavior of is controlled by two components: 1) very slowly changing and 2) related to trust-inducing events. Since the first component is very slowly changing, this method assumes that B k Obeying Gaussian distribution N(u k ,σ0), and is therefore expected to increase rapidly in the case of external stimulation, i.e.:

[0100]

[0101] Where δ is the average increase in the central rate of trust events. k It is also time-varying. k The estimation is performed in the BN environment by Kalman filtering and b iThe estimate is decomposed.

[0102] (2) Skin electrical signal data processing and feature extraction methods

[0103] The skin conductance signal is first downsampled to 1Hz and then a smoothing filter is applied. The EDA signal consists of two main components: a slowly changing component, the skin conductance level (SCL), and a component associated with sudden increases in trust, the skin conductance response (SCR). These rapid increases are measured using the first absolute difference (FAD):

[0104]

[0105] Here's i is the EDA at time i. The method of dividing the FAD measure by the current SCL level eliminates the influence of SCL on SCR. This method can be easily applied to real-time estimation. The transformation applied to the FAD measure is 1 / SCL l , it is SCL l is a nonlinear function of This function is in SCL l For large (1 / SCL l <1) reduces the FAD, and at SCL l is smaller than (1 / SCL l >1) amplifies the FAD. This method uses SCL l A logistic function of :

[0106]

[0107] The transformed FAD (nFAD) is nFAD = FAD g (SCL l ), the parameters α and β are optimized to obtain a higher correlation with the trust metric derived from the annotations.

[0108] 2. Signal Fusion

[0109] (1) Feature Normalization

[0110] For each feature, the minimum-maximum normalization method is used for normalization. First, determine the value range of each feature, that is, find the minimum value X min and the maximum value X max , and then use a normalization formula to normalize the features:

[0111]

[0112] Among them, X′ is the normalized value. Each feature of heart rate and skin electricity is calculated one by one. Finally, all normalized feature values ​​are output to ensure that the values ​​are in the range of [0,1].

[0113] (2) Weighted Fusion

[0114] Heart rate features can be selected from average heart rate, square root of heart rate variability, low-high frequency power spectrum density, etc. Skin electrical features can be selected from mean, median, variance, standard deviation, maximum and minimum values ​​of SCR and SCL, etc. Each feature of heart rate and each feature of skin electrical are fused into a feature matrix:

[0115] Y1=[F HR , F RMSSD , F LF , F HF ]

[0116] Y2=[F mean_SCR , F mean_SCL ,…,F max_SCR , F max_SCL ]

[0117] Each row of the feature matrix represents a sample. Then, a target vector y is constructed. This method sets the driver's trust status as the target vector y, where 0 represents distrust, 1 represents moderate trust, and 2 represents excessive trust. The target vector is then combined with the feature matrix to construct a training set (features Y1, Y2 and target vector y) for model training.

[0118] This method uses machine learning training to automatically confirm feature weights. Considering the nonlinear correspondence between physiological signals and trust levels, a decision tree method is used. After training, feature weights are extracted from the model. The weights of the extracted n features are ω1, ω2, ···, ω n-1 ,ω n For feature fusion, the fusion formula is as follows:

[0119] F fused1 =ω1×F HR +…+ω4×F HF

[0120] F fused2 =ω5×F mean_SCR +…+ω n-1 ×F max_SCR +ω n ×F max_SCL

[0121] Among them, F HR 、···、F mean_SCR 、F mean_SCL etc. are the feature vectors of heart rate and skin electricity.

[0122] 3. Model input construction

[0123] The fusion features of heart rate and skin electricity are combined into a feature matrix Y:

[0124] Y=[F fused1 , F fused2 ]

[0125] Since the first layer of the trust assessment model constructed by this method is a convolutional neural network, for one-dimensional data or feature matrix, the input shape needs to be adjusted to a 4D tensor. The data after signal fusion is a 2D array, specifically [number of samples, number of features]. The feature matrix needs to be reshaped by adding width and number of channels after the number of samples and number of features to promote it to a 4D tensor, specifically [number of samples, number of features, width, channels], and fill the width and number of channels with 1.

[0126] 4. Trust Assessment Model

[0127] The model structure is as follows Figure 2 As shown, including:

[0128] (1) Convolutional module

[0129] First, the original features are extracted using a convolution module, the features are mapped to different feature spaces, and the convolution kernel is used in the convolution layer to extract spatial features and learn useful information from the data. The fused features pass through a 2D-CNN with a step size of s and a padding of p, and then pass through a rectified linear unit (ReLU) layer.

[0130] (2) Deep channel attention module

[0131] In this module, multi-layer Efficient Channel Attention (ECA) is used to extract deeper local features.

[0132] The first step is to compress the spatial dimension while retaining the key information of different channels. And use global average pooling (GAP), which allows fast calculation of features along the channel dimension while avoiding dimensionality reduction. The next part uses channel attention to capture local cross-channel interactions. Given the aggregated feature y:

[0133] ω=σ(W y )

[0134] The ECA module uses the bandpass matrix W k To learn channel attention, W k contains k×C parameters, where k represents the number of neighbor nodes and C represents the number of channels. By doing so, y iThe weight of will only consider its k neighbors, so the following results can be obtained:

[0135]

[0136] in, Represents y i The method is implemented using one-dimensional convolution, where k represents the size of the convolution kernel. To avoid unnecessary calculations, the size of the convolution kernel depends on the channel dimension C. The size of the convolution kernel k can be determined adaptively:

[0137]

[0138] Where |t| odd Denotes the nearest odd integer of t. Set y and b to 2 and 1 respectively. After convolution, the extracted features are transformed nonlinearly through activation functions to facilitate normalization in the next step. The last step is to scale the original input by these normalized features to emphasize more informative elements and suppress less relevant elements. Between each ECA layer, a residual network is also used to prevent gradient explosion.

[0139] (3) Transformer module

[0140] Considering the global correlation in physiological electrical signals, extracting the global features of multimodal information is extremely beneficial to improving the robustness of the model. This module selects a model from Transformer to extract global features, which makes up for the limited receptive field of the convolution module. The output of the previous module is linearly transformed and sent to the Q, K, V (query, key, value) components.

[0141] The encoder architecture uses dot product operations to determine the correlation between various features. In order to reduce the risk of gradient vanishing and ensure the stability of the training process, a scaling factor is introduced. The resulting matrix is ​​then normalized by the Softmax function to generate a weighted matrix. This attention score is then applied to the V matrix through another dot product, ultimately forming a weighted representation that reflects the model's focused attention mechanism. In the case of self-attention, the contents of Q, K, and V remain consistent. The calculation process is as follows:

[0142]

[0143] Among them, d kIndicates the length of the feature. In addition, the multi-sensor head attention (MHA) mechanism is used to further enhance the diversity of features. In this mechanism, the features are evenly divided into h parts. Each part is sent to the self-attention module for calculation. Finally, the results of each part are spliced ​​to obtain the final output result. The whole process can be expressed as:

[0144] MHA(Q,K,V)=[head0;...;head h-1 ],

[0145] head l =Attention(Q l ,K l ,V l )

[0146] Among them, Q l , K l , V l denote the query, key, and value matrices of the lth attention head, respectively. In addition, a residual connection is made between the feedforward layer and the MHA layer, followed by layer normalization (LN), which enhances the model's fitting ability. The input and output dimensions of this module remain unchanged. Within the entire Transformer layer, the encoder structure is repeated N times.

[0147] (4) Regression prediction module

[0148] This layer converts features into continuous output values ​​bounded between 0 and 1. First, there is the LayerNorm module, which normalizes the individual features of the input. After normalization, the AvgPool layer is used to reduce the dimensionality of the data while retaining the essential information. Subsequently, the data is linearly transformed through a linear layer, mapping it to a singular output value for each instance to meet the regression objective. The process ends with the application of the tanh activation function and further adjustments to shift and scale the output range to [0,1], ensuring that the output of this layer is best suited for regression tasks that require bounded continuous predictions.

[0149] 5. Model Output

[0150] This method defines the target vector as [0,1,2], where 0 represents distrust, 1 represents moderate trust, and 2 represents excessive trust. It is necessary to establish an association between the output of the regression prediction module and the target vector. First, the regression output is mapped to the interval [0,2], and linear mapping is used to achieve this:

[0151] y′=a·y+b

[0152] Among them, t′ is the mapped value, a is the scaling factor, and b is the offset. Since the output range of the regression prediction module is [0,1], a and b are 2 and 0 respectively. Then the threshold mapping is performed, and the threshold is used to convert it into a classification result according to the mapped output. If the output is less than 0.5, it is predicted to be 0, if the output is less than 1.5 and greater than 0.5, it is predicted to be 1, and if the output is greater than 1.5, it is predicted to be 2.

[0153] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution, which should be included in the scope of the claims of the present invention.

Claims

1. A driver trust state assessment method based on physiological signals, characterized in that: The method specifically comprises the following steps: S1: Preprocessing and feature extraction of heart rate signal and skin electrical signal; S2: signal fusion: normalize the heart rate features and skin electrical features extracted in step S1, and then fuse the normalized heart rate features and skin electrical features into a feature matrix; S3: construct model input; S4: Build a trust assessment model, including convolution module, deep channel attention module, Transformer module and regression prediction module; S5: Model output: The target vector is defined as [0, 1, 2], where 0 represents distrust, 1 represents moderate trust, and 2 represents overtrust. Then, an association is established between the output of the regression prediction module and the target vector, that is, a linear mapping is used to map the regression output to the interval [0, 2]. Finally, threshold mapping is performed, and the threshold is used to convert the mapped output into a classification result.

2. The driver trust status evaluation method according to claim 1, characterized in that: In step S1, the heart rate signal is preprocessed and features are extracted, specifically including: first extracting the RR variability signal RRV from the ECG, and calculating HRV=60 / RRV; the RR period is estimated as the time difference between R consecutive peaks; the RRV signal and the HRV signal mainly contain two components: 1) a low-frequency baseline component, which is a mutation caused by a trust-induced event; 2) a periodic component, including parasympathetic and sympathetic nerve activities; the latter is modeled by a P-order autoregressive process AR; the RR interval at time i is denoted as rr i : Among them, b and a are the baselines of the signal and AR coefficient respectively, c is a random error term representing all the variations not explained by the model, and the noise is Gaussian distributed; the baseline b is the variable affected by the trust-inducing event and used for trust detection; the i-th indicator indicates that these variables are time-varying, that is: b i =b i-1 +e b Among them, ε b and ε a is Gaussian distributed noise; the state space form is: x i =x i-1 +R y i =H i x i +Q Among them, x i =[a1,a2,...,a P ,b i ],y i =rr i , R is process noise, Q is observation noise, matrix H i =diag[rr i-1 ,rr i-2 ,...,rr i-P ,1]; In order to estimate x i , for each new sample y i Using the normal Kalman filter equation: x i =x i-1 Where K is the Kalman gain matrix; S is the matrix for calculating the Kalman gain, represents the innovation covariance, and represents the uncertainty of the current observation; x i and V i are the Kalman filter estimates of the hidden state and error covariance matrix at time step i, and are the updated hidden state estimate and error covariance matrix at time step i respectively; in order to better estimate the hidden state of the model, this method uses a Kalman smoother; the Kalman smoothing update equation is: in, and are the Kalman smoothing estimates of the hidden state and error covariance matrix at time step i, is the smoothing gain matrix, used to update the smoothing estimate; T represents the transpose of the matrix, which is used when updating the error covariance matrix; uses windows of specific length to calculate features from the physiological signal, and for each new window, applies the Kalman smoother only to past observations contained in the window, given the current information; uses the initial segment of the signal to extract initial estimates of the AR coefficients and the observation noise R, as well as an initial estimate of the baseline; uses the first 5 minutes of the signal for all initializations; uses the diagonal Q, and sets the elements corresponding to the AR coefficients to 10 -8 , the element corresponding to the baseline is set to 10 -6 ; In the Kalman filter, the baseline parameter b changes over time, so it is necessary to calculate the average baseline value b within a specific window i ; The new feature that appears is recorded as B k , where k is the window indicator, which inherits b i The behavior of B is controlled by two components: 1) very slowly changing and 2) related to trust-inducing events; since the first component is very slowly changing, we assume that B k Obeying Gaussian distribution N(u k ,σ0), and is therefore expected to increase rapidly in the case of external stimulation, i.e.: Where δ is the average increase in the central rate of trust events; u k It is time-varying, u k The estimation is performed in the BN environment by Kalman filtering and b i The estimate is decomposed.

3. The driver trust status evaluation method according to claim 1, characterized in that: In step S1, the skin electrical signal is preprocessed and features are extracted, specifically including: first, the skin electrical signal is downsampled to 1Hz, and then a smoothing filter is used; the EDA signal consists of two parts: a slowly changing component, namely the skin conductance level SCL, and a component related to a sudden increase in trust, namely the skin conductance response SCR; the first absolute difference FAD is used to measure the index of rapid increase: Among them, s i is the EDA at time i, and N is the total number of time points; the method of dividing the FAD measure by the current SCL level is used to eliminate the influence of SCL on SCR. This method can be easily applied to real-time estimation; the transformation of this method applied to the FAD measure is 1 / SCL l , it is SCL l is a nonlinear function of Using SCL l A logistic function of : The transformed FAD (nFAD) is nFAD = FAD g (SCL l ), the parameters α and β are optimized to obtain a higher correlation with the trust metric derived from the annotations.

4. The driver trust status evaluation method according to claim 3, characterized in that: In step S2, signal fusion specifically includes the following steps: S21: Feature Normalization For each feature, the minimum-maximum normalization method is used for normalization; first determine the value range of each feature, that is, find the minimum value X min and the maximum value X max , and then use a normalization formula to normalize the features: Among them, X′ is the normalized value; each feature of heart rate and skin electricity is calculated one by one; finally, all normalized feature values ​​are output to ensure that the values ​​are in the range of [0,1]; S22: Weighted Fusion The heart rate features include the average heart rate, the square root of heart rate variability and the low-frequency power spectrum density. The skin electrical features include the mean, median, variance, standard deviation and maximum and minimum values ​​of SCR and SCL. The various features of heart rate and skin electrical are fused into feature matrices: Y1=[F HR ,F RMSSD ,F LF ,F HF ] Y2=[F mean_SCR ,F mean_SCL ,…,F max_SCR ,F max_SCL ] Each row of the feature matrix represents a sample. Then, a target vector y is constructed, and the driver's trust status is set as the target vector y, where 0 represents distrust, 1 represents moderate trust, and 2 represents excessive trust. Then, the target vector is combined with the feature matrix to construct a training set for model training. The machine learning training method is used to automatically confirm the feature weights. Considering the nonlinear correspondence between physiological signals and trust levels, the decision tree method is adopted. After the training is completed, the feature weights are extracted from the model. The weights of the extracted n features ω1, ω2, ···, ω n-1 ,ω n For feature fusion, the fusion formula is as follows: F fused1 =ω1×F HR +…+ω4×F HF F fused2 =ω5×F mean_SCR +…+oh n-1 ×F max_SCR +oh n ×F max_SCL Among them, F HR 、···、F mean_SCR 、F mean_SCL is the characteristic vector of heart rate and skin electricity, F fused1 、F fused2 They are the fusion features of heart rate and skin electricity respectively.

5. The driver trust status evaluation method according to claim 4, characterized in that: Step S3 specifically includes: fusing the fusion features of heart rate and skin electricity into a feature matrix Y: Y=[F fused1 ,F fused2 ] For one-dimensional data or feature matrix, the input shape needs to be adjusted to a 4D tensor; the data after signal fusion is a 2D array, specifically [number of samples, number of features]. The feature matrix needs to be reshaped by adding width and number of channels after the number of samples and number of features to promote it to a 4D tensor, specifically [number of samples, number of features, width, channels], and fill the width and number of channels with 1.

6. The driver trust status evaluation method according to claim 1, characterized in that: In step S4, building a trust assessment model specifically includes: (1) Convolutional module First, the convolution module is used to extract the original features, map the features to different feature spaces, and use the convolution kernel in the convolution layer to extract spatial features and learn useful information from the data; the fused features pass through a 2D-CNN with a step size of s and a padding of p, and then pass through a rectified linear unit layer; (2) Deep channel attention module In this module, multi-layer efficient channel attention ECA is used to extract deep local features; The first step is to compress the spatial dimension while retaining the key information of different channels; and use global average pooling; the next part uses channel attention to capture local cross-channel interactions; given the aggregate feature y: ω=σ(W y ) Among them, ω represents the feature vector after nonlinear transformation, σ(·) represents a nonlinear activation function, which is used to increase the expressive power of the model; the ECA module uses the bandpass matrix W k To learn channel attention, W k Contains k×C parameters, where k represents the number of neighbor nodes and C represents the number of channels; by doing so, y i The weight of will only consider its k neighbors, so the following results can be obtained: Among them, ω i represents the final attention weight of the i-th channel, represents the weight of the i-th channel on the j-th neighbor channel, represents the eigenvalue of the jth neighbor channel, belonging to the neighbor set of channel i, Represents y i The method is implemented using one-dimensional convolution, where k represents the size of the convolution kernel. To avoid unnecessary calculations, the size of the convolution kernel depends on the channel dimension C. The size of the convolution kernel k can be determined adaptively: Where |t| odd represents the nearest odd number of t, and γ is used to control the convolution kernel size and the amount of computation; y and b are set to 2 and 1 respectively; after convolution, the extracted features are nonlinearly transformed by the activation function to facilitate normalization in the next step; the last step is to scale the original input by these normalized features; between each ECA layer, a residual network is also used to prevent gradient explosion; (3) Transformer module This module selects a model from Transformer to extract global features, which makes up for the limited receptive field of the convolution module; the output of the previous module is linearly transformed and sent to the query Q, key K, value V components; The encoder architecture uses dot product operations to determine the correlation between various features. The resulting matrix is ​​then normalized through the Softmax function to generate a weighted matrix. This attention score is then applied to the V matrix through another dot product, resulting in a weighted representation that reflects the model's focused attention mechanism. In the case of self-attention, the contents of Q, K, and V remain consistent. The calculation process is as follows: Among them, d k Indicates the length of the feature; in addition, the multi-sensor head attention MHA mechanism is used to further enhance the diversity of features; in this mechanism, the features are evenly divided into h parts; each part is sent to the self-attention module for calculation; finally, the results of each part are spliced ​​to obtain the final output result; the whole process is expressed as: MHA(Q,K,V)=[head0;...;head h-1 ], head l =Attention(Q l ,K l ,V l ) Among them, Q l ,K l ,V l denote the query, key, and value matrices of the lth attention head, respectively. In addition, a residual connection is made between the feedforward layer and the MHA layer, followed by layer normalization. Within the entire Transformer layer, the encoder structure is repeated N times. (4) Regression prediction module This module converts features into continuous output values ​​limited to 0-1. The first is the LayerNorm module, which normalizes the input features. After normalization, the AvgPool layer is used to reduce the data dimension while retaining the essential information. Subsequently, the data is linearly transformed through the linear layer to map it to a singular output value for each instance to meet the regression objective. The process ends with the application of the tanh activation function and further adjustments to shift and scale the output range to [0,1], thereby ensuring that the output of this module is suitable for regression tasks that require bounded continuous predictions.

7. The driver trust status evaluation method according to claim 1, characterized in that: In step S5, the regression output is mapped to the interval [0,2], which is implemented by the following linear mapping: y′=a·y+b Among them, y′ is the mapped value, a is the scaling factor, and b is the offset. Since the output range of the regression prediction module is [0,1], a and b are 2 and 0 respectively. Then threshold mapping is performed, and the threshold is used to convert it into a classification result according to the mapped output. If the output is less than 0.5, it is predicted to be 0, if the output is less than 1.5 and greater than 0.5, it is predicted to be 1, and if the output is greater than 1.5, it is predicted to be 2.

Images