Trust degree evaluation method and equipment of driving assistance system and medium

Abstract

The invention discloses a trust degree evaluation method and equipment of a driving assistance system and a medium, and relates to the field of trust evaluation of the driving assistance system.The method comprises the steps that electroencephalogram signal data of a driver is obtained; according to the driving assistance system credibility evaluation model and the electroencephalogram signal data of the driver, evaluating the credibility of the driving assistance system to obtain a credibility evaluation result of the driving assistance system; wherein the driving assistance system credibility evaluation model comprises a separation convolution module, a time sequence convolution module and a classification module. According to the method, the time domain and frequency domain information in the electroencephalogram signals is extracted by adopting the separation convolution module, the time sequence information of the electroencephalogram signals is extracted through the time sequence convolution module, and then the time domain, frequency domain and time sequence characteristics of the electroencephalogram signals are combined to evaluate the credibility of the driving assistance system. And the trained and optimized driving assistance system trust degree evaluation model is adopted during evaluation, so that the accuracy of the driving assistance system trust degree evaluation is effectively improved.

Description

Technical Field

[0001] The present application relates to the technical field of trust evaluation of driving assistance systems, and in particular to a trust evaluation method, device, and medium for a driving assistance system. Background Art

[0002] With the continuous advancement of intelligent driving technology, although driving assistance systems are gradually being widely used, drivers have low trust in the systems, resulting in low utilization of the systems and the inability to fully utilize the true role of driving assistance systems. Therefore, real-time and objective evaluation of drivers' trust in driving assistance systems is an important part of achieving safe and efficient human-machine collaboration. Trust is one of the key factors affecting the interaction between drivers and driving assistance systems, directly affecting the driver's dependence on driving assistance systems and intervention behavior. Both too high and too low trust will cause conflicts between drivers and driving assistance systems. It is necessary to calibrate drivers' trust based on the capabilities of driving assistance systems, thereby enhancing the efficiency and safety of human-machine collaboration. However, existing trust assessment methods mainly use driver behavioral observation data, self-report questionnaires, and physiological signal data as model inputs, which have poor real-time performance and are difficult to accurately and quickly reflect changes in drivers' trust in the system.

[0003] As a highly flexible, real-time, and non-invasive neuroimaging technology, electroencephalogram (EEG) provides a new means for in-depth research and analysis of the dynamics of driver trust. Real-time and objective assessment and analysis of drivers' trust in driver assistance systems, such as warning prompts, can help improve and enhance the applicability and acceptance of these systems. Commonly used EEG analysis methods are mostly based on deep learning theory and primarily utilize feature information in the time or frequency domain, resulting in relatively low accuracy in trust assessment. Summary of the Invention

[0004] After research, the applicant found that the complexity and dynamics of the driver's trust changing over time will also affect the accuracy of the trust assessment. In view of this, the present application provides a trust assessment method, device and medium for a driving assistance system, which can effectively improve the accuracy of the trust assessment of the driving assistance system.

[0005] To achieve the above objectives, this application provides the following solutions:

[0006] In a first aspect, the present application provides a method for trust evaluation of a driving assistance system, comprising:

[0007] Obtain the driver's EEG signal data.

[0008] According to the driving assistance system trust evaluation model and the driver's EEG signal data, the trust of the driving assistance system is evaluated to obtain a trust evaluation result of the driving assistance system; the driving assistance system trust evaluation model is a trained and optimized driving assistance system trust evaluation model; the driving assistance system trust evaluation model includes a separation convolution module, a timing convolution module and a classification module; wherein the separation convolution module is used to perform a separation convolution operation on the input EEG signal data with the goal of extracting time domain information and frequency domain information; the timing convolution module is used to perform a timing convolution operation on the output of the separation convolution module with the goal of extracting timing information; the classification module is used to classify the output of the timing convolution module and output the trust evaluation result; the trust evaluation result includes a high trust category value and a low trust category value.

[0009] In a second aspect, the present application provides a computer device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the trust evaluation method for the driving assistance system described in the first aspect.

[0010] In a third aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the trust evaluation method for the driving assistance system described in the first aspect.

[0011] According to the specific embodiments provided in this application, this application has the following technical effects:

[0012] The present application provides a method, device and medium for evaluating the trustworthiness of a driving assistance system, wherein the evaluation method comprises: obtaining the driver's EEG signal data; evaluating the trustworthiness of the driving assistance system according to the driving assistance system trustworthiness evaluation model and the driver's EEG signal data, and obtaining the trustworthiness evaluation result of the driving assistance system; wherein the driving assistance system trustworthiness evaluation model comprises a separation convolution module, a timing convolution module and a classification module. The present application uses a separation convolution module to extract the time domain information and frequency domain information from the EEG signal, and uses a timing convolution module to extract the timing information of the EEG signal, and then combines the time domain, frequency domain and timing characteristics of the EEG signal to evaluate the trustworthiness of the driving assistance system, and uses a trained and optimized driving assistance system trustworthiness evaluation model during the evaluation. Compared with the prior art method of only using feature information such as time domain or frequency domain for evaluation, the present application effectively improves the accuracy of the trustworthiness evaluation of the driving assistance system. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0014] Figure 1 This is an application environment diagram of a trust evaluation method for a driving assistance system in one embodiment of the present application;

[0015] Figure 2 A flowchart of a trustworthiness assessment method for a driving assistance system provided in one embodiment of the present application;

[0016] Figure 3 A schematic diagram of the structure of a trust evaluation model for a driving assistance system provided in one embodiment of the present application;

[0017] Figure 4 A schematic diagram of the structure of the point convolution layer and the separation convolution layer in the trust evaluation model of the driving assistance system provided in one embodiment of the present application;

[0018] FIG5( a ) is a schematic diagram of a visualization result of a model output correctly classified with high confidence corresponding to Sample 1 provided in an example of this application;

[0019] FIG5( b ) is a schematic diagram of a visualization result of a model output correctly classified with high confidence corresponding to sample 4 provided in an example of this application;

[0020] FIG6( a ) is a schematic diagram of a visualization result of a model output misclassification with high confidence corresponding to sample 6 provided in an example of this application;

[0021] FIG6( b ) is a schematic diagram of a visualization result of a model output misclassification with high confidence corresponding to sample 7 provided in an example of this application;

[0022] Figure 7 A schematic diagram of the model performance comparison results provided for an example of this application;

[0023] Figure 8 A schematic diagram of the comparison results of ablation experiments provided in an example of this application;

[0024] Figure 9 A schematic diagram of the structure of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0025] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0026] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0027] The trust evaluation method of the driving assistance system provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown, the terminal 102 communicates with the server 104 via a network. The data storage system can store data that the server 104 needs to process. The data storage system can be set up separately, integrated on the server 104, or placed on the cloud or other servers. The terminal 102 can send the driver's EEG signal data to the server 104. The server 104 receives the driver's EEG signal data and evaluates the trustworthiness of the driving assistance system based on the driving assistance system trust evaluation model and the driver's EEG signal data, obtaining a trustworthiness evaluation result of the driving assistance system. The server 104 can feed back the obtained trustworthiness evaluation result of the driving assistance system to the terminal 102. In addition, in some embodiments, the trustworthiness evaluation method of the driving assistance system can also be implemented separately by the server 104 or the terminal 102. For example, the terminal 102 can directly perform a trustworthiness evaluation on the driver's EEG signal data, or the server 104 can obtain the driver's EEG signal data from the data storage system and perform a trustworthiness evaluation on the driver's EEG signal data.

[0028] The terminal 102 may be, but is not limited to, various desktop computers, laptop computers, smart phones, tablet computers, IoT devices, and portable wearable devices. The server 104 may be implemented as an independent server or a server cluster consisting of multiple servers, or a cloud server.

[0029] In an exemplary embodiment, Figure 2 As shown, a trust evaluation method for a driving assistance system is provided. The method is executed by a computer device, specifically a computer device such as a terminal or a server, or a terminal and a server. In the embodiment of the present application, the method is applied to Figure 1 The server 104 in the example is used as an example to illustrate the process, including the following steps 201 to 202.

[0030] Step 201: Obtain driver's EEG signal data.

[0031] Step 202: Based on the driving assistance system trust evaluation model (abbreviated as model or TrustNet model) and the driver's EEG signal data, the trust of the driving assistance system is evaluated to obtain a trust evaluation result of the driving assistance system; the driving assistance system trust evaluation model is a trained and optimized driving assistance system trust evaluation model; the driving assistance system trust evaluation model includes a separation convolution module, a timing convolution module and a classification module; wherein the separation convolution module is used to perform a separation convolution operation on the input EEG signal data with the goal of extracting time domain information and frequency domain information; the timing convolution module is used to perform a timing convolution operation on the output of the separation convolution module with the goal of extracting timing information; the classification module is used to classify the output of the timing convolution module and output the trust evaluation result; the trust evaluation result includes a high trust category value and a low trust category value.

[0032] See Figure 3 In another exemplary embodiment of the present application, the separation convolution module includes a point convolution layer and a separation convolution layer; wherein, the input of the point convolution layer is the input of the driving assistance system trust evaluation model, the output of the point convolution layer is the input of the separation convolution layer, and the output of the separation convolution layer is the input of the temporal convolution module.

[0033] The temporal convolution module includes several temporal convolution layers; wherein, the input of the first temporal convolution layer is the output of the separation convolution module, the input of the remaining temporal convolution layers are the output of the previous temporal convolution layer, and the output of the last temporal convolution layer is the input of the classification module.

[0034] The classification module includes a batch normalization layer, a global average pooling layer, a fully connected layer and an activation layer; the input of the batch normalization layer is the output of the temporal convolution module, the output of the batch normalization layer is the input of the global average pooling layer, the output of the global average pooling layer is the input of the fully connected layer, the output of the fully connected layer is the input of the activation layer, and the output of the activation layer is the trust evaluation result of the driving assistance system.

[0035] See Figure 4 In another exemplary embodiment of the present application, the point convolution layer includes 16 point convolution nodes; the separation convolution layer includes 32 separation convolution nodes; the size of the EEG signal data is 32*384; wherein 32 represents the number of channels for collecting EEG signals; and 384 represents the number of sampling points for each channel.

[0036] In another exemplary embodiment of the present application, step 202 specifically includes:

[0037] Step 202.1: Preprocess the driver's EEG signal data (also called raw EEG signal data) to obtain preprocessed EEG signal data.

[0038] In some embodiments, the preprocessing includes: performing denoising and artifact removal on the raw EEG signal data through filtering and principal component analysis (ICA). By denoising the raw EEG signal and removing artifacts irrelevant to the analysis, the signal quality is improved. The specific steps are as follows:

[0039] Step 202.1.1: Apply a bandpass filter to the EEG signal with a frequency range of 0.1-50 Hz to suppress power frequency interference and other high-frequency noise. The filtered EEG signal is denoted as S filtered (t).

[0040] Step 202.1.2: Use independent component analysis to separate the different source signals in the EEG signal. Calculate the artifact ratio using the EEGlab toolkit and remove eye movement and myoelectric artifacts using a 90% threshold.

[0041] The ICA process is as follows:

[0042] 1) Centralization: Centralize the EEG signal to obtain an EEG signal with a mean of zero

[0043]

[0044] Among them, mean() means taking the average.

[0045] 2) Whitening: Calculate the covariance matrix Perform eigenvalue decomposition on the covariance D is the eigenvalue, V is the eigenvector, and then the decorrelated signal (i.e., the eye movement and electromyography artifacts are removed) is obtained by whitening.

[0046] Step 202.2: Evaluate the trustworthiness of the driving assistance system based on the driving assistance system trustworthiness evaluation model and the pre-processed EEG signal data to obtain a trustworthiness evaluation result of the driving assistance system.

[0047] In another exemplary embodiment of the present application, the training process of the driving assistance system trust evaluation model specifically includes:

[0048] Step 301: Collect EEG signal data samples and corresponding trust questionnaire data samples of the subject in the dynamic following driving scenario; the EEG signal data samples are the EEG signal data samples of the subject in the preset time period before and after each driving assistance system warning prompt; the trust questionnaire data samples are the scoring data samples of the subject's score of the trust in the driving assistance system after each driving assistance system warning prompt; the subject is the vehicle driver in the dynamic following driving scenario.

[0049] Step 302: Train the driver assistance system trust evaluation model based on the EEG signal data samples and the corresponding trust questionnaire data samples to obtain a preliminarily trained driver assistance system trust evaluation model; wherein the EEG signal data samples are input data; and the trust questionnaire data samples are label data corresponding to the EEG signal data.

[0050] In the dynamic following driving task, the driving assistance system (such as the front collision warning system) will warn the driver according to different set thresholds. The subjective trust questionnaire data and EEG signals of the subjects are collected simultaneously, where the EEG signals are recorded by a multi-channel EEG device (such as 32 channels, with a sampling frequency of 500Hz); after each warning prompt, the trust questionnaire data is used by the subjects to score the trust in the driving assistance system according to the trust level (0-10 scale). In some embodiments, the EEG signal time window includes 1 second before the warning event and 2 seconds after the warning event to fully capture the changes in trust status before and after the warning.

[0051] In some embodiments, the model training process also includes: dividing the collected electroencephalogram (EEG) signal data samples and the corresponding questionnaire score data samples into a training set, a validation set, and a test set at a ratio of 8:1:1; using the Adam optimization algorithm for gradient descent, setting the initial learning rate to 0.001, and gradually reducing the learning rate to ensure convergence stability.

[0052] In this application, the model first uses a point convolution layer to perform point convolution (also known as point-wise convolution) processing to project the input channel into a new space, reduce the channel dimension and separate the signal components.

[0053] In another exemplary embodiment, for a 3-second window of reliable EEG data X consisting of 32 channels, input , through point-wise convolution, a new signal channel with the number P1 (i.e., the number of point convolution nodes in the point convolution layer) is generated. The operation of the point convolution layer is defined as:

[0054] X pointwise =Conv 1×1 (X input )(2);

[0055]

[0056] Where i = 1, 2, 3, ..., P1; x p,j is the data of the jth sampling point corresponding to the p channel in the input EEG signal data, j = 1, 2, 3, ..., n, and n is the number of sampling points in each channel of the EEG signal.

[0057] Formula (2) is the mathematical expression of the point convolution layer, X input represents the input of the point convolution layer, X pointwise Represents the output of the point convolution layer, Conv 1×1 ()Point convolution operation.

[0058] Formula (3) represents the network’s hidden variable learning process, which is also the operation process of the point convolution nodes in the point convolution layer; Represents the weight of the point convolution layer (also called weight parameter, weight coefficient), that is, the weight of the p-channel corresponding to the i-th point convolution node; Represents the bias of the point convolution layer, that is, the bias of the i-th point convolution node in the convolution layer; Represents the latent variable output of the point convolution layer, that is, the latent variable output of the i-th point convolution node in the point convolution layer; the upper right corner subscript "(1)" of the parameter in the formula indicates that this is the first layer of the model, that is, the point convolution layer, see Figure 3 .

[0059] In another exemplary embodiment, P1 is set to 16, that is, the point convolution layer includes 16 point convolution nodes; the size of the EEG signal data is 32*384, where 32 is the number of channels for collecting EEG signals; 384 is the number of sampling points for each channel; the mathematical model of the point convolution node is shown in formula (3).

[0060] In this embodiment, by setting P1 = 16, which represents half the number of input channels, the purpose is to reduce redundancy and speed up network convergence. In other words, the number of channels of the input data of the point volume layer is twice the number of channels of the output data of the point volume layer (that is, the number of point volume nodes). Figure 4 .

[0061] In this application, the second layer of the model (i.e., the separation convolution layer) uses separation convolution (1×k) to independently learn time domain features from the output signal of the first layer of the network.

[0062] In another exemplary embodiment, each channel of the first layer network output is convolved with two separate convolution nodes respectively, so there are 2P1 separate convolution nodes in total, see Figure 4 The operation of separating the layers is defined as:

[0063] Xdepthwise =Conv 1×l (X pointwise )(4);

[0064]

[0065] Where i = 1, 2, 3, ..., 2P1; j = 1, 2, 3, ..., n-r+1; l represents the kernel size of the separating convolution node; the value of l matches the data sampling frequency to ensure that the model captures relevant temporal patterns. In some embodiments, l is 64.

[0066] Formula (4) is the mathematical expression of the separation convolution layer, X pointwise Represents the input of the separation convolution layer, that is, the output of the point convolution layer, X depthwise Represents the output of the separation convolution layer, Conv 1×l () represents a separable convolution operation.

[0067] In formula (5), the upper right corner subscript "(2)" of the parameter indicates that the parameter is the parameter of the second layer network of the model; and They are all hidden variable outputs of the point convolution layer. Specifically, Indicates the point convolution layer The data corresponding to the j+r-1th sampling point in the latent variable output of the point convolution node; Indicates the point convolution layer The hidden variable output of the point convolution node corresponds to the data of the j+r-1th sampling point; Represents the weight of the separation convolution layer, that is, the corresponding weight of the i-th separation convolution node; Represents the bias of the separation convolution layer, that is, the bias of the i-th separation convolution node in the separation convolution layer; Represents the latent variable output of the separation convolution layer, that is, the latent variable output of the i-th separation convolution node in the separation convolution layer.

[0068] In another exemplary embodiment, the separation convolution layer includes 32 separation convolution nodes, corresponding to P1=16, that is, the point convolution layer includes 16 point convolution nodes; the mathematical model of the separation convolution node is shown in formula (5).

[0069] In this application, we use separable convolution and temporal convolution to perform multi-feature analysis on EEG signals, extracting time-domain, frequency-domain, and temporal-series features and performing classification processing. During point convolution and separable convolution, the model can capture periodic patterns and changing trends in the signal, which is actually learning time-domain and frequency-domain information; the temporal convolution module learns temporal features.

[0070] In this application, the output of the separation convolution is passed into the temporal convolution module (temporal convolutional network), and the temporal convolutional network (TCN) captures long-term dependency features through dilated convolution.

[0071] In another exemplary embodiment, the temporal convolution module includes a first temporal convolution layer, a second temporal convolution layer, and a third temporal convolution layer; the input of the first temporal convolution layer is the input of the temporal convolution module, the output of the first temporal convolution layer is the input of the second temporal convolution layer, the output of the second temporal convolution layer is the input of the third temporal convolution layer, and the output of the third temporal convolution layer is the input of the classification module. The mathematical expression of the dilated convolution of the sth temporal convolution layer is:

[0072]

[0073] Among them, T (s) Represents the output of the s-th temporal convolution layer of the model, s = 3, 4, 5, that is, the first temporal convolution layer is the third layer of the model, the second temporal convolution layer is the fourth layer of the model, and the second temporal convolution layer is the fourth layer of the model; T (s-1) It means that the input of the s-th temporal convolution layer of the model is the output of the s-1th layer network, the input of the first temporal convolution layer is the output of the second layer network, and the input of other temporal convolution layers are all the output of the previous temporal convolution layer; k (s) is the TCN kernel, i.e. the kernel of the sth temporal convolution layer; To achieve the internal difference of TCN; Represents a matrix product operation.

[0074] In this application, the final step of the model is classification processing: the multiple features extracted by the previous convolutional layer are processed by batch normalization and global average pooling, and then pass through the fully connected layer and Softmax classifier to output the trust classification result (i.e., the trust confidence evaluation result). The trust classification categories include high trust and low trust.

[0075] In another exemplary embodiment of the present application, the optimization process of the driving assistance system trust evaluation model specifically includes:

[0076] Step 401: Based on the EEG signal data sample of the subject and the initially trained driving assistance system trust evaluation model, an improved class activation mapping technology is used to generate an EEG feature heat map.

[0077] Step 402: Optimize the initially trained driving assistance system trust evaluation model based on the EEG feature heat map to obtain an optimized driving assistance system trust evaluation model.

[0078] In another exemplary embodiment of the present application, step 401 specifically includes:

[0079] Step 401.1: Determine first activation data based on the outputs of the fully connected layer, the global average pooling layer, and the batch normalization layer for the EEG signal data sample of the subject in the initially trained driving assistance system trust assessment model; the first activation data includes a plurality of first activation values.

[0080] Step 401.2: Sort the first activation values ​​in the first activation data from largest to smallest to obtain a first activation value sorting sequence.

[0081] Step 401.3: Determine a first determination position according to the first activation value sorting sequence; the first determination position is the position of a preset number of first activation values ​​that are ranked first in the activation value sorting sequence in the first activation data.

[0082] Step 401.4: Determine a second discrimination position of the subject's EEG signal data sample based on the first discrimination position.

[0083] Step 401.5: Based on the second discriminant position and the corresponding activation value, the EEG feature heat map is calculated using a Gaussian function.

[0084] The EEG feature heat map generated based on the class activation map can explain the basis of the model decision for further optimization of the model.

[0085] In another exemplary embodiment of the present application, step 401.1 specifically includes:

[0086] According to the first activation model, first activation data corresponding to a trust evaluation result is calculated; the trust evaluation result is obtained based on the preliminarily trained driver assistance system trust evaluation model and the subject's EEG signal data sample; the mathematical expression of the first activation model is:

[0087]

[0088] in, represents the activation value of the corresponding evaluation category (i.e., trust category) c determined according to the output of the fully connected layer; P1 represents the number of point convolution nodes in the point convolution layer; Represents the weight parameter corresponding to channel i and evaluation category c in the fully connected layer; represents the activation value of the corresponding channel i determined according to the output of the global average pooling layer; n represents the number of sampling points of the EEG signal, which is also the number of frames of the input EEG data time series; l represents the size of the kernel of the separation convolution node in the separation convolution layer; represents the j-th activation value of the corresponding channel i determined according to the output of the batch normalization layer in the initially trained driver assistance system trust assessment model; is the first activation value corresponding to trust category c.

[0089] In some embodiments, the bias term may be omitted for simplicity. And the constant factor (n-l+1)=321 will not affect the final generated EEG feature heat map. It can be viewed as the final activation distribution of 2P1×(n-l+1) maps.

[0090] The subscripts “(6)”, “(7)”, and “(8)” in the upper right corner of the parameters refer to the sixth, seventh, and eighth layers of the model, respectively. The sixth layer is the batch normalization layer, the seventh layer is the global average pooling layer, and the eighth layer is the fully connected layer.

[0091] The traditional class activation mapping technique (CAM) uses upsampling To match the size of the input sample. However, since the model of this application uses point-wise convolution to mix signal channels in the first layer of the network, The first dimension of is not aligned with the data structure of the input signal, and the above purpose cannot be achieved. When visualizing, it is necessary to find the layers where the data structure no longer changes during neural network operations. Inspired by the CNN fixation method, the applicant improved the traditional CAM method by tracking the locations in the activation map that contribute most to class activation (the first discriminant locations) and then mapping these locations back to the corresponding regions (also called fragments) in the input sample.

[0092] Specifically, first Sort the values ​​in descending order to get the top N (i.e. the preset number) The corresponding positions are (i0, j0), (i1, j1), ..., (i N ,j N ), that is, the first discriminant position, where 1≤i k ≤2P1, 1≤j k ≤n-l+1; The next goal is to track these first discriminant positions and return to the centers of the regions (segments) (p0,q0), (p1,q1),…, (p N ,q N ), which is the second determination position. These positioning points satisfy 1≤p k ≤m and 1≤q k ≤n, m is the number of sampling channels of EEG signals.

[0093] For an input EEG signal data sample The activation corresponding to the first discriminant position in the second layer of the network Determined by the following formula (8):

[0094]

[0095] in, Indicates the corresponding first discrimination position (i k ,j k )’s activation value; The jth corresponding to the p channel in the EEG signal data sample input into the preliminarily trained driver assistance system trust evaluation model k +r-1 sampling point data; Represents the i-th layer in the separation convolution layer k The rth weight parameter of the separated convolution node; Indicates the point convolution layer The point convolution nodes correspond to the weight parameters of p channels; Indicates the point convolution layer The point convolution node corresponds to the weight parameter of the p channel; l represents the size of the kernel of the separation convolution node in the separation convolution layer.

[0096] Furthermore, in another exemplary embodiment of the present application, step 401.4 specifically includes:

[0097] According to the second discriminant position model, the second discriminant position of the subject's EEG signal data sample is calculated, and the mathematical expression of the second discriminant position model is as follows:

[0098]

[0099] q k =j k +(l-1) / 2 (10);

[0100] Among them, (i k ,j k ) represents the first discrimination position; (p k ,q k ) indicates the first discrimination position (i k ,j k )’s second discriminant position; l represents the size of the kernel of the separation convolution node in the separation convolution layer.

[0101] By using formula (9) and formula (10), it is possible to track the center of the segment that contributes most to activation in the input signal (p k ,q k ), which is the second discriminant position. A segment refers to a data segment obtained by dividing the EEG signal data sample on a time scale according to the size of the kernel of the separation convolution node.

[0102] In another exemplary embodiment of the present application, the mathematical model of the Gaussian function in step 401.5 is:

[0103]

[0104] in, represents the heat map value corresponding to the position (p,q) in the input EEG signal data when the trust category is c; σ is a constant that determines the radius of the influence area of ​​each discriminant point in the input signal; (p,q) is the coordinate of the reference point of the input EEG signal data; (qq k ) 2 Represents the time point (sampling point) q and the second judgment position q k The square difference between them is used to calculate the exponential part of the Gaussian function, reflecting the distance of the judgment position; Represents the sum of all discriminant positions k, where p k Equal to the current channel p.

[0105] That is, by combining all class discrimination points in the input sample with the Gaussian function, the final thermal area map (i.e., EEG feature heat map) can be obtained.

[0106] Through The data are further normalized in the range of (-1, 1) for visualization.

[0107] In some embodiments, σ=1 / 2=32 is set to ensure that the Gaussian function highlights the part of the input signal that contributes the most. For EEG signal data of size 32*384, high heat visualization is performed on 100 EEG signal data points by tracking the first 100 discriminant positions. These 100 discriminant positions account for about 100% of all EEG signal data points. 1%.

[0108] After generating an EEG feature heat map by combining the improved class activation mapping technology and obtaining the visual result response feature, frequency domain mapping can be performed to further illustrate the interpretability. Therefore, in another exemplary embodiment of the present application, the optimization process of the driver assistance system trust evaluation model also includes:

[0109] Step 403: Based on the generated EEG feature heat map, perform Fourier transform on the corresponding segment of the input data of the driving assistance system trust evaluation model to generate EEG signal frequency domain features;

[0110] Step 404: Based on the EEG signal frequency domain characteristics, the optimized driving assistance system trust evaluation model is further optimized.

[0111] In frequency domain mapping, the generated time-domain heatmap is mapped to EEG frequency-domain features via Fourier transform. This allows observation of feature variations under different trust classification states, further verifying the physiological rationality of the model's trust classification. Analysis of correctly and incorrectly classified samples explores the underlying EEG patterns associated with trust and identifies the impact of sensor noise and individual response differences on classification results, thereby uncovering potential areas for model optimization.

[0112] This application introduces an improved Class Activation Mapping (CAM) technique to track the segments of the input signal that contribute most to activation, generating a visual heat map of key feature areas (i.e., an EEG feature heat map) to provide an intuitive explanation of the model's trust classification results. Based on the local activation of each input sample, the multi-channel time-frequency features of the EEG signal for trust classification are interpreted, providing specific directions for subsequent model improvements and EEG signal utilization.

[0113] Figures 5(a), 5(b), 6(a), and 6(b) are individual visualization examples, showing the visualization results of the model output correctly classified high confidence and incorrectly classified high confidence. The upper left corner is the original EEG heat map, the lower left corner is the time domain heat map, and the right side is the heat map of different bands displayed in the frequency domain. It can be seen that the interpretation method of the model proposed in this application can clearly show that the biological characteristics related to trust - specifically, the changes in beta and gamma waves related to the occipital and frontal lobe regions - are highly correlated with fluctuations in trust. In addition, the analysis of misclassified samples shows that sensor noise and individual response time differences are the main factors affecting classification accuracy.

[0114] In one example, the present application verifies the classification accuracy of the model for trust in a dynamic vehicle following task through comparison.

[0115] By validating the model in a dynamic vehicle-following task and evaluating its classification performance and the causes of misclassification, the present invention conducted comparisons with other models and ablation experiments to analyze the feasibility of the model:

[0116] Model performance comparison, using cross-validation and different classification models to compare the TrustNet model's trust classification accuracy and stability. The comparison results are as follows Figure 7 As shown in Figure 2, the compared models include EEGNet, LMDA-Net, and ShallowNet.

[0117] Ablation Experiment Comparison,We conducted ablation experiments to study the contribution of individual network components to the,overall performance of our proposed model. Figure 8 As shown in the figure, NoTCN means removing the TCN module, 1-TCN means retaining one layer of TCN, 2-TCN means retaining two layers of TCN, and No-1DConv means removing the separation convolution module.

[0118] The proposed method combines questionnaire scores with temporal, frequency, and time-series features in EEG signals. By introducing separable convolutional and temporal convolutional networks, it captures complex multidimensional EEG features while reducing the model's computational complexity. Furthermore, this method incorporates class activation mapping techniques to generate decision heatmaps for key feature regions in trust classification, enhancing the model's interpretability.

[0119] The present application also provides an application scenario, which applies the above-mentioned trust evaluation method of the driving assistance system. Specifically: the trust evaluation method of the driving assistance system provided in this embodiment can be applied to the scenario of real-time evaluation of the trust of the driving assistance system. The scenario of real-time evaluation of the trust of the driving assistance system includes an evaluation data acquisition link and a trust evaluation link; the driver's EEG signal data enters the trust evaluation link from the evaluation data acquisition link. The trust evaluation method of the driving assistance system provided in this embodiment belongs to the trust evaluation link. Specifically, after obtaining the driver's EEG signal data, the trust of the driving assistance system is evaluated according to the driving assistance system trust evaluation model and the driver's EEG signal data to obtain the trust evaluation result of the driving assistance system.

[0120] Compared with the existing technology, this application has the following advantages:

[0121] 1. Applicability in dynamic collaborative tasks. This application combines EEG signals with questionnaire scoring data for the first time to evaluate changes in people's trust in assisted driving systems in dynamic collaborative tasks. It is particularly suitable for complex dynamic environments where people collaborate with advanced assisted driving systems. This application expands the scope of application of traditional static scenarios by capturing the dynamic characteristics of trust in real time in dynamic following (following) scenarios, provides practical value for real-time trust monitoring in fields such as intelligent assisted driving, and provides innovative ideas for real-time trust evaluation of human-machine collaborative systems.

[0122] 2. Multi-feature fusion improves trust classification accuracy. This application uses separable convolution and time series convolution networks to extract time-frequency domain and time series features of EEG signals, automatically extract key features of trust changes, and perform data feature mapping learning corresponding to questionnaire label data, thereby improving the accuracy of trust state classification. Compared with traditional methods, it can significantly improve the accuracy and robustness of trust assessment. Compared with traditional methods, it can significantly improve the accuracy and robustness of trust assessment. In addition, it also avoids the traditional trust assessment's reliance on manual features, thereby improving the efficiency of the model.

[0123] 3. Enhanced interpretability combined with class activation mapping. This application uses improved class activation mapping (CAM) technology to generate EEG feature heat maps (also called decision heat maps) to intuitively display the decision basis of the trust classification model. And by generating decision heat maps for key feature areas in trust classification, the interpretability of the model is enhanced. CAM technology provides a visual analysis of trust assessment, reveals the model's focus on time-frequency features, provides data support for the scientific nature of trust assessment and model improvement, and improves the interpretability and credibility of the model.

[0124] In an exemplary embodiment, a computer device is provided. The computer device may be a server or a terminal. The internal structure diagram thereof may be as follows: Figure 9 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O) and a communication interface. The processor, memory and input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database of the computer device is used to store EEG signal data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a trust evaluation method for a driving assistance system is implemented.

[0125] Those skilled in the art will understand that Figure 9 The structure shown in the figure is merely a block diagram of a portion of the structure related to the solution of the present application and does not constitute a limitation on the computer device to which the solution of the present application is applied. A specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement. In an exemplary embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps of the above-mentioned method embodiments when executing the computer program.

[0126] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0127] In an exemplary embodiment, a computer program product is provided, including a computer program. When the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0128] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0129] Those skilled in the art will understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0130] The databases involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processors involved in the various embodiments provided herein may include, but are not limited to, general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic units, data processing logic units based on quantum computing, and the like.

[0131] The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0132] This document uses specific examples to illustrate the principles and implementation methods of this application. The description of the above examples is only intended to help understand the method and core concept of this application. At the same time, for those skilled in the art, based on the concept of this application, there may be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A trust evaluation method for a driving assistance system, characterized in that: The trust evaluation method of the driving assistance system includes: Obtaining driver's EEG signal data; According to the driving assistance system trust evaluation model and the driver's EEG signal data, the trust of the driving assistance system is evaluated to obtain a trust evaluation result of the driving assistance system; the driving assistance system trust evaluation model is a trained and optimized driving assistance system trust evaluation model; the driving assistance system trust evaluation model includes a separation convolution module, a timing convolution module and a classification module; wherein the separation convolution module is used to perform a separation convolution operation on the input EEG signal data with the goal of extracting time domain information and frequency domain information; the timing convolution module is used to perform a timing convolution operation on the output of the separation convolution module with the goal of extracting timing information; the classification module is used to classify the output of the timing convolution module and output the trust evaluation result; the trust evaluation result includes a high trust category value and a low trust category value.

2. The trust evaluation method for a driving assistance system according to claim 1, characterized in that: The training process of the driver assistance system trust evaluation model specifically includes: Collecting EEG signal data samples and corresponding trust questionnaire data samples from a subject in a dynamic car-following driving scenario; the EEG signal data samples are EEG signal data samples of the subject during a preset period before and after each warning prompt from the driving assistance system; the trust questionnaire data samples are scoring data samples of the subject's trust in the driving assistance system after each warning prompt from the driving assistance system; the subject is a vehicle driver in the dynamic car-following driving scenario; The driving assistance system trust evaluation model is trained based on the EEG signal data samples and the corresponding trust questionnaire data samples to obtain a preliminary trained driving assistance system trust evaluation model; wherein the EEG signal data samples are input data; and the trust questionnaire data samples are label data corresponding to the EEG signal data.

3. The trust evaluation method for a driving assistance system according to claim 2, characterized in that: The optimization process of the driver assistance system trust evaluation model specifically includes: generating an EEG feature heat map using an improved class activation mapping technique based on the EEG signal data sample of the subject and the preliminarily trained driver assistance system trust assessment model; The initially trained driving assistance system trust evaluation model is optimized according to the EEG feature heat map to obtain an optimized driving assistance system trust evaluation model.

4. The trust evaluation method for a driving assistance system according to claim 3, characterized in that: Based on the subject's EEG signal data samples and the preliminarily trained driver assistance system trust assessment model, an improved class activation mapping technique is used to generate an EEG feature heat map, specifically including: Determining first activation data based on outputs of a fully connected layer, a global average pooling layer, and a batch normalization layer for the EEG signal data sample of the subject in the preliminarily trained driver assistance system trust assessment model; the first activation data includes a plurality of first activation values; sorting the first activation values ​​in the first activation data from largest to smallest to obtain a first activation value sorting sequence; Determining a first determination position according to the first activation value sorting sequence; the first determination position is a position in the first activation data of a preset number of first activation values ​​that are ranked first in the activation value sorting sequence; Determining a second discrimination position of the subject's EEG signal data sample based on the first discrimination position; According to the second discrimination position and the corresponding activation value, the EEG feature heat map is calculated using a Gaussian function.

5. The trust evaluation method for a driving assistance system according to claim 4, characterized in that: Determining first activation data according to outputs of a fully connected layer, a global average pooling layer, and a batch normalization layer for the EEG signal data sample of the subject in the preliminarily trained driver assistance system trust evaluation model, specifically comprising: According to the first activation model, first activation data corresponding to a trust evaluation result is calculated; the trust evaluation result is obtained based on the preliminarily trained driver assistance system trust evaluation model and the subject's EEG signal data sample; the mathematical expression of the first activation model is: in, represents the activation value of the corresponding trust category c determined according to the output of the fully connected layer; P1 represents the number of point convolution nodes in the point convolution layer; Represents the weight parameter corresponding to channel i and evaluation category c in the fully connected layer; represents the activation value of the corresponding channel i determined according to the output of the global average pooling layer; n represents the number of sampling points of the EEG signal; l represents the size of the kernel of the separation convolution node in the separation convolution layer; represents the j-th activation value of the corresponding i channel determined according to the output of the batch normalization layer; is the first activation value corresponding to trust category c.

6. The trust evaluation method for a driving assistance system according to claim 4, characterized in that: Determining a second discrimination position of the subject's EEG signal data sample based on the first discrimination position specifically includes: The second discriminant position of the subject's EEG signal data sample is calculated according to the second discriminant position model; the mathematical expression of the second discriminant position model is as follows: q k =j k +(l-1) / 2; Among them, (i k ,j k ) indicates the first position; (p k ,q k ) corresponds to the first position (i k ,j k )'s second determination position; The jth of the pth channel in the EEG signal data sample of the driver assistance system trust evaluation model preliminarily trained in the input model k +r-1 sampling point data; Represents the i-th layer in the separation convolution layer k The rth weight parameter of the separated convolution node; Indicates the point convolution layer The point convolution nodes correspond to the weight parameters of p channels; Indicates the point convolution layer The point convolution node corresponds to the weight parameter of the p channel; l represents the size of the kernel of the separation convolution node in the separation convolution layer.

7. The trust evaluation method for a driving assistance system according to claim 6, characterized in that: The mathematical model of the Gaussian function is: in, It represents the heat map value corresponding to the position (p,q) in the input EEG signal data when the confidence category is c; σ is a constant.

8. The trust evaluation method for a driving assistance system according to claim 1, characterized in that: The separation convolution module includes a point convolution layer and a separation convolution layer; wherein the input of the point convolution layer is the input of the driver assistance system trust evaluation model, the output of the point convolution layer is the input of the separation convolution layer, and the output of the separation convolution layer is the input of the temporal convolution module; The temporal convolution module includes several temporal convolution layers; wherein, the input of the first temporal convolution layer is the output of the separation convolution module, the input of the remaining temporal convolution layers are the output of the previous temporal convolution layer, and the output of the last temporal convolution layer is the input of the classification module; The classification module includes a batch normalization layer, a global average pooling layer, a fully connected layer and an activation layer; the input of the batch normalization layer is the output of the temporal convolution module, the output of the batch normalization layer is the input of the global average pooling layer, the output of the global average pooling layer is the input of the fully connected layer, the output of the fully connected layer is the input of the activation layer, and the output of the activation layer is the trust evaluation result of the driving assistance system.

9. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the trust evaluation method for a driving assistance system according to any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for evaluating the trustworthiness of the driving assistance system according to any one of claims 1 to 8 is implemented.

Images