Vehicle brain-computer interface expected function interaction test evaluation method
Through the four-layer evaluation system and entropy value method, the gap in the interaction quality analysis, testing and evaluation of brain-vehicle interface functions in brain-controlled vehicles was solved, and a comprehensive and accurate evaluation of brain-controlled vehicle technology was achieved, which promoted its R&D and deployment.
Patent Information
- Application Number
- CN202410818767.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-06-24
- Publication Date
- 2024-12-06
AI Technical Summary
The lack of effective brain-controlled vehicle-brain interface interaction quality analysis, testing and evaluation methods in the prior art have hindered the development of brain-controlled vehicle technology.
A method for the expected function interaction test and evaluation of vehicle brain-computer interfaces is proposed. The interaction effect of brain-computer interfaces is measured and evaluated in detail through the four-layer evaluation system (target layer, functional layer, performance layer, index layer), including security and anti-interference and use the entropy method to calculate the index weight to achieve objective evaluation and optimization.
It provides a comprehensive and accurate evaluation method for the interactive performance of the vehicle brain-computer interface function, improves the R&D efficiency and deployment application of brain-controlled vehicle technology, reduces R&D costs, and promotes the practical application of brain-controlled vehicles.
Smart Images

Figure CN119090321A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of brain-computer interface and automobile intelligence, and in particular to a method for testing and evaluating the expected functional interaction of a vehicle brain-computer interface. Background Art
[0002] In the fields of cutting-edge science and technology and industrial transformation such as brain-like intelligence, brain-computer technology has become one of the key technologies that need to be tackled worldwide. The technological innovation, industrial ecology, infrastructure, regulatory standards, product supervision and network security systems of smart cars are gradually improving. The intelligence of cars has become the direction of future technological development. Brain-computer interface is a cutting-edge research field for smart vehicles. Therefore, research related to brain-controlled vehicles that apply brain-computer interface technology has a very broad development prospect. Since the brain-computer interface serves as an information exchange channel for the driver to control the vehicle, its performance plays a key role in the interaction of brain-controlled vehicles. Therefore, testing and evaluating the performance of the vehicle brain-computer interface system is the key to accelerating the development, application and deployment of brain-controlled vehicles.
[0003] However, the current research on brain-controlled vehicles at home and abroad is still in its infancy, and there are still many problems to be solved, especially in the analysis, testing and evaluation of the interactive quality of brain-vehicle interface functions. The lack of an objective and reasonable evaluation method will hinder the subsequent development of brain-controlled vehicle technology. Summary of the invention
[0004] In order to solve the above technical problems, the present invention proposes a vehicle brain-computer interface expected function interaction test and evaluation method, the evaluation system architecture includes four layers:
[0005] The first layer is the target layer: the core of the target layer is to evaluate the expected functional interaction level of the brain-computer interface (BMI), measure the interaction effect and user experience of the brain-computer interface in actual car control, and record the performance evaluation results. The target layer is the basis for guiding the evaluation of the functional layer;
[0006] The second layer is the functional layer: the functional layer evaluates the expected functions achieved by the brain-computer interface, including one or more functions such as braking control, fatigue monitoring, window lifting and lowering control, air conditioning control, and audio control; under each expected function, the performance requirements of the brain-computer interface and the user's operating habits may be different. In this layer, the characteristics of each function and the specific requirements for the brain-computer interface performance should be described in detail, which will help with subsequent targeted evaluation and optimization.
[0007] The third layer is the performance layer: the performance layer further subdivides the evaluation of brain-computer interface functions under each expected function into different performance dimensions, including one or more dimensions of safety, anti-interference, accuracy, timeliness, and comfort;
[0008] The safety mentioned above is used to evaluate the safety assurance capability of the system under various operating conditions, such as the prevention of erroneous operation and the response in emergency situations;
[0009] The anti-interference performance is used to analyze the system's ability to resist external interference in a complex environment and ensure stable signal transmission;
[0010] The accuracy is used to measure the accuracy of user intention recognition and the precision of instruction execution;
[0011] The timeliness described above is used to evaluate the time delay from the user issuing a command to the system responding, which is crucial to improving the smoothness of interaction;
[0012] The comfort level is used to examine the user's comfort and fatigue when using the brain-computer interface, which directly affects the user experience;
[0013] The fourth layer is the indicator layer: the indicator layer concretizes the performance dimension into quantifiable evaluation indicators. Each evaluation indicator should have a clear quantification method and evaluation standard to facilitate objective evaluation and comparison.
[0014] The security evaluation index includes recognition accuracy;
[0015] The anti-interference evaluation index includes one or more of the following indicators: the number of selectable targets, false trigger rate, and signal-to-noise ratio;
[0016] The accuracy evaluation index includes one or more of the following indicators: false alarm rate, Kappa coefficient, and mutual information;
[0017] The evaluation index of timeliness includes one or more of information transmission rate, response time and maximum kurtosis;
[0018] The evaluation index of comfort includes one or two of the following indexes: mental workload and driver satisfaction;
[0019] All indicators are objective numerical quantitative indicators;
[0020] The evaluation method includes the following steps:
[0021] (1) Data standardization
[0022] The original data is standardized to eliminate the dimensional differences between different indicators and make them comparable. There are l functions, each function has n performance dimensions, each performance dimension has m indicators, and the original data matrix is X = (x ij ) n×m ; where x ij Represents the original value of the jth indicator of the i-th performance dimension of a certain function, i=0,1,…n; j=0,1,…m;
[0023] For positive indicators (larger values indicate better performance):
[0024]
[0025] For negative indicators (smaller values indicate better performance):
[0026]
[0027] Among them, Z ij For x ij The standardized score, min(x ij ) is the minimum value of all sample values of the jth indicator of the i-th performance dimension, max(x ij ) is the maximum value among all sample values of the jth indicator of the ith performance dimension.
[0028] (2) Calculate the entropy value of the indicator
[0029] Calculate the entropy value e of the jth indicator j :
[0030]
[0031] in:
[0032] e j represents the entropy value of the jth indicator;
[0033] m is the total number of samples;
[0034] k is a constant, and the calculation formula is: Used to normalize the entropy value to ensure that the entropy value is within a reasonable range;
[0035] p ij It represents the proportion of the i-th sample on the j-th index. The calculation formula is: Where Z ij For x ij Standardized scores;
[0036] The entropy value reflects the amount of information of the indicator. The larger the entropy value, the more dispersed the information of the indicator and the smaller the weight. Conversely, the smaller the entropy value, the more concentrated the information of the indicator and the greater the weight.
[0037] (3) Calculate the weight of the indicator
[0038] Calculate the weight Wz of the jth indicator j :
[0039]
[0040] The purpose of calculating weights is to determine the importance of each indicator in the comprehensive evaluation. The larger the weight, the greater the impact of the indicator on the comprehensive evaluation.
[0041] (4) Calculate the weight of the performance dimension level
[0042] The analytic hierarchy process (AHP) was used to determine the weights of different performance dimensions. The steps are as follows:
[0043] (4.1) Constructing the judgment matrix
[0044] Suppose there are n performance dimensions, construct an n×n judgment matrix A, where a ij Indicates the importance of performance dimension i relative to performance dimension j; the judgment matrix A satisfies the following conditions:
[0045] a ij >0
[0046]
[0047] a ii =1
[0048] (4.2) Calculate the eigenvector and maximum eigenvalue of the judgment matrix
[0049] Calculate the maximum eigenvalue λ of the judgment matrix A max and its corresponding eigenvector V = (v1, v2, ..., v n ) T , where v n represents the nth eigenvector in the eigenvectors.
[0050] (4.3) Normalized eigenvector
[0051] Normalize the feature vector V to obtain the weight Wxi of each performance dimension level:
[0052]
[0053] (4.4) Consistency check
[0054] In order to ensure the consistency of the judgment matrix, a consistency test is required; the consistency index CI and consistency ratio CR are calculated:
[0055]
[0056]
[0057] Among them, RI is the random consistency index, which depends on the order n of the matrix;
[0058] If CR < 0.1, the judgment matrix has satisfactory consistency; otherwise, the judgment matrix needs to be adjusted;
[0059] (5) Calculation function weight
[0060] Under the target level, the weight Wg of the i-th function is calculated by the hierarchical analysis method i , the specific steps are the same as above;
[0061] (6) Calculate the overall evaluation score
[0062] The weight of each indicator is multiplied by its standardized value to obtain the comprehensive score of each sample. The score Sk of the kth function is:
[0063]
[0064] The final total performance score S of l functions is:
[0065]
[0066] The comprehensive score reflects the comprehensive performance of each sample on all indicators. The higher the score, the better the comprehensive performance.
[0067] Furthermore, in the indicator layer, the safety of brain-computer interaction is evaluated by the recognition accuracy; accurately identifying the driver's intention is the primary task of brain-controlled cars, and catastrophic accidents caused by incorrect recognition of the driver's intention should be avoided; the recognition accuracy is defined as follows: for the same set of test data, the recognition result is RR (Recognition Result), where the correct result is represented by TR (Ture Recognition) and the incorrect recognition result is represented by FR (False Recognition), then the recognition accuracy ERA (EEG Recognition Accuracy) is calculated by the following formula:
[0068]
[0069] Furthermore, in the indicator layer, the anti-interference indicators of brain-computer interaction include the number of selectable targets, false trigger rate and signal-to-noise ratio. All three indicators are objective numerical quantitative indicators.
[0070] The optional target number indicates the number of command categories that the system can output. The optional target number is usually set as a constant. It is an important criterion for measuring whether the system can complete the control task, and is directly related to the completeness and execution efficiency of the system. If the optional target number is less than the number of commands of the control task, the system is incomplete. If it is greater than the optional command number of the control task, the system resources will be occupied, reducing the communication efficiency of the system. The optional target number is the basis of the design of the brain-computer interface system and has a significant impact on the system performance evaluation.
[0071] The false trigger rate, also known as the false positive rate (FPR), in the field of brain-computer interface, mainly refers to the probability of the system being falsely triggered in an idle state. The calculation formula is:
[0072]
[0073] Among them, FP represents the number of samples predicted as positive samples but actually are negative samples, which specifically corresponds to the number of control instructions issued incorrectly in the idle state; TN represents the number of samples predicted as negative samples but actually are negative samples, which corresponds to the situation where instructions are avoided in the idle state;
[0074] The signal-to-noise ratio (SNR) refers to the ratio of energy consumption between signal and noise in a system composed of electronic devices; the signal is the effective signal generated by the electronic device, and the noise refers to the irregular interference signal generated after the effective signal passes through the electronic device. This signal is not related to the effective signal and does not carry useful information. In the field of brain-computer interface, the signal is defined as the cerebral cortex EEG signal collected by the EEG instrument, and the noise is defined as the noise such as the electromyographic signal that interferes with the EEG signal. Since the energy consumption of the signal and the noise is not easy to calculate in the field of brain-computer interface, the amplitude of the signal and the noise is used to calculate the signal-to-noise ratio of the EEG signal. The calculation formula is as follows:
[0075]
[0076] Among them, V s Represents the effective value of the signal amplitude, V n Represents the effective value of the noise replica of the signal.
[0077] Furthermore, in the indicator layer, the accuracy requirements for brain-computer interaction are evaluated by false alarm rate, Kappa coefficient and mutual information, all of which are objective numerical quantitative indicators.
[0078] The false positive rate (FPR) refers to the ratio of false positives generated by the system when the user ensures an idle state.
[0079] The Kappa coefficient is an evaluation index of classification accuracy based on the confusion matrix. This index can also be used for consistency testing. The calculation of the Kappa value is to multiply the total number of elements in the actual classification by the diagonal elements of the confusion matrix and then sum them up. Then, the sum of all categories is subtracted from the value by the product of the total number of real elements in a certain category and the total number of classified elements in that category. The sum of all categories is divided by the square of the total number of elements minus the product of the total number of real elements in a certain category and the total number of classified elements in that category. The calculation formula of the Kappa value is as follows:
[0080]
[0081] Among them, p0 represents the observed consistency (correct model prediction) proportion, that is, the proportion of evaluators who actually agree; p e represents the expected agreement proportion, that is, the proportion of raters who would agree under random circumstances.
[0082] Assumed distance is the evaluation value of the BCI system for the identified object. When the test data is classified correctly, such as when the left hand is imagined for the i-th time, the evaluation distance D obtained by analyzing the EEG signal is (i) > 0, otherwise, if the i-th trial is in the right hand imagination, then the estimated distance D obtained by analyzing the EEG signal (i) <0; however, in the actual classification process, the EEG signal is interfered by other factors, which cannot make each classification task executed correctly. For the motor imagery classification task, its recognition rate reaches 100%; therefore, the evaluation value of each classification can be regarded as two categories, one is the correct classification related to the motor imagery task, containing effective information output, and the other is the wrong classification, which is the interference or noise signal irrelevant to the motor imagery task; from the perspective of the signal-to-noise ratio, the entire motor imagery recognition task is represented in the form of:
[0083]
[0084] Among them, SNR t is the signal-to-noise ratio (SNR), is the i-th data point at time t. {L}, {R} represent two different data sets or condition sets, such as the evaluation value sets of left-hand imagination and right-hand imagination, and var represents the variance of the evaluation value set. In order to obtain relevant information about motor imagination from the classifier evaluation results, according to Shannon's theorem, the mutual information I can be expressed as entropy difference, and the entropy H for a Gaussian process x is defined as follows:
[0085]
[0086] Where, e is the base of natural logarithm, δ x represents the standard deviation of the random variable x.
[0087] According to the entropy value definition, the output signal-to-noise ratio of the evaluation system is calculated, and the calculation formula of the mutual information of the motor imagery recognition task based on binary classification is as follows:
[0088] I t =0.5×log2(SNR t +1)
[0089] The larger the mutual information value, the better the classification effect of the BCI system and the higher the confidence of the classifier's output; the mutual information value reflects a better recognition rate to some extent, while a higher recognition rate makes it difficult to judge the value of the information.
[0090] Furthermore, in the indicator layer, the information transmission rate is a commonly used indicator for measuring information transmission. It estimates the number of bits of information transmitted per unit time from the perspective of information theory, taking into account the number of optional targets, accuracy and transmission speed. ITR is one of the most commonly used comprehensive evaluation indicators for evaluating the communication rate of the system in brain-computer interface research. Its unit is often expressed in bit / min, and its calculation process is as follows:
[0091]
[0092] T=T s +T I
[0093]
[0094] Where N is the number of target options in the system, P is the recognition accuracy of each target option, B is the amount of information transmitted by a single target selection, T is the time for a single target selection, and T s represents the single target selection stimulus time, T I Indicates the interval between two selections.
[0095] The response time indicates the time required for the user to issue a BCI command, reflecting how quickly the BCI system executes the control intention. The calculation of a complete response time should start from the time the user issues the control intention as the starting point (which can be the time when the BCI system gives an output prompt or the time when the user independently chooses the output command), to the time when the system collects data on the length of time required for signal analysis, and then to the time when the classification result is generated and output as the end point.
[0096] The maximum steepness of MI is an evaluation index based on mutual information. It not only takes into account the quality of mutual information, but also evaluates the response time of the entire system. The faster the response time of the system, the faster the expected classification output is obtained, and the higher the maximum steepness is. The maximum steepness can better evaluate the quality of the online BCI system, while evaluating the output confidence and system response time. It is an evaluation scheme for online BCI. The calculation formula of the maximum steepness is as follows:
[0097] MI_maxsteepness = MI(t) / (t-3)
[0098] Where MI(t) is the mutual information value, and t>3.5s is required to avoid stress response.
[0099] Furthermore, in the indicator layer, the mental workload refers to the mental resources that the user expends to complete a task, that is, the degree of brain fatigue; the evaluation result of this indicator directly affects the practicality and convenience of the corresponding BCI system, especially for users with severe paralysis, their degree of brain fatigue is an important factor that cannot be ignored. Subjective workload assessment technology or NASA workload index questionnaire is used to investigate users' subjective feelings about the relevant system. Among them, NASA-TLX covers six dimensions, including spirit, performance, physical strength, time requirements, effort and frustration. Each dimension has 20 levels, and the corresponding score distribution is 0-100. The smaller the value, the less workload, and vice versa. This method has been widely used in the performance evaluation of BCI systems.
[0100] Driver satisfaction refers to the driver's satisfaction with the brain-computer interaction function during driving. This indicator reflects the driver's evaluation of the comfort, safety and overall user experience of the brain-controlled car function. Through the driver satisfaction survey, relevant research institutions of brain-computer interaction functions can obtain feedback and suggestions from drivers, thereby improving the brain-computer interaction function and enhancing the driver's driving experience. This plays a very important role in improving the quality of human-computer interaction of brain-controlled car functions. Driver satisfaction surveys are usually conducted through questionnaires, interviews, and interviews to collect drivers' opinions and evaluations. These survey results can help researchers formulate improvement measures and provide drivers with a better brain-computer interaction experience.
[0101] Questionnaire survey is an effective method to collect people's evaluation of the comfort of using the expected functions of brain-computer interface. By using scoring or satisfaction scale methods, the subjects are asked to evaluate their satisfaction with the comfort of the expected functions of brain-computer interface. The questionnaire adopts a scoring form of 1 to 5 to evaluate various indicators related to comfort; finally, based on the collected questionnaires, the data is analyzed and summarized to obtain specific data for the evaluation of the comfort of the expected functions of brain-computer interface.
[0102] Beneficial effects of the present invention:
[0103] The present invention is based on the field of vehicle brain-computer interface testing and evaluation, and focuses on the problem that the current evaluation methods cannot meet the needs of vehicle brain-computer interface function performance evaluation. It innovatively proposes a vehicle-brain-computer interface-function interaction performance evaluation method, which evaluates the measured performance of different brain-computer interface functions, considering the five dimensions of safety, anti-interference, accuracy, timeliness, and comfort of the brain-computer interface function in various scenarios, and matching the indicators in each dimension with the evaluation needs of the brain-computer interface function, setting the indicators of each dimension required for the evaluation, thereby forming an evaluation method with different evaluation emphases, using the improved entropy method to calculate the single-level weight values of each component of the indicator layer, performance layer, and function layer, and assigning values to the indicator layer according to the measured performance of the brain-computer interface function, and finally realizing the vehicle-brain-computer interface-function interaction performance evaluation.
[0104] The vehicle-brain-computer interface-function interaction performance evaluation method provided by the present invention can construct a relatively comprehensive interaction performance evaluation method with different focuses for different vehicle brain-computer interface functions, thereby improving the comprehensiveness and accuracy of the vehicle-brain-computer interface-function interaction performance evaluation, providing a theoretical basis and technical support for the vehicle-brain-computer interface-function interaction performance evaluation, improving the research and development efficiency of the vehicle brain-computer interface system function and reducing the research and development cost of the brain-controlled vehicle system, thereby promoting the deployment and application of brain-controlled vehicles on actual roads. BRIEF DESCRIPTION OF THE DRAWINGS
[0105] Figure 1 A schematic diagram of the evaluation system architecture of the present invention;
[0106] Figure 2 This is a schematic diagram of the evaluation score calculation process of the present invention;
[0107] Figure 3 It is a schematic diagram of the weight calculation structure of the present invention. DETAILED DESCRIPTION
[0108] The present invention proposes a method for testing and evaluating the expected functional interaction of a vehicle brain-computer interface. Figure 1 , 3 As shown, the evaluation architecture consists of four layers:
[0109] The first layer is the target layer: recording the performance evaluation results, S is the total performance score;
[0110] The second layer is the functional layer: including brake control, fatigue monitoring, window lifting control, air conditioning control, and audio control. S1-S4 are the scores of the four functions, and Wg1-Wg4 are the weights of the four functions.
[0111] The third layer is the performance layer: including five dimensions: security, anti-interference, accuracy, timeliness, and comfort. Wx1-Wx5 are the weights of the five dimensions.
[0112] The fourth layer is the indicator layer:
[0113] The security evaluation index includes recognition accuracy; the recognition accuracy is defined as follows: for the same set of test data, the recognition result is RR (Recognition Result), where the correct result is represented by TR (Ture Recognition), and the wrong recognition result is represented by FR (False Recognition), then the recognition accuracy ERA (EEGRecognition Accuracy) is calculated by the following formula:
[0114]
[0115] The evaluation index of anti-interference performance includes the number of selectable targets, false trigger rate, and signal-to-noise ratio; the number of selectable targets indicates the number of command categories that the system can output, and the number of selectable targets is usually set to a constant. If the number of selectable targets is less than the number of commands of the control task, the system is incomplete; if it is greater than the number of selectable commands of the control task, the system resources will be occupied, reducing the communication efficiency of the system;
[0116] The false trigger rate, also known as the false positive rate (FPR), in the field of brain-computer interface, mainly refers to the probability of the system being falsely triggered in an idle state. The calculation formula is:
[0117]
[0118] Among them, FP represents the number of samples predicted as positive samples but actually are negative samples, which specifically corresponds to the number of control instructions issued incorrectly in the idle state; TN represents the number of samples predicted as negative samples but actually are negative samples, which corresponds to the situation where instructions are avoided in the idle state;
[0119] The signal-to-noise ratio (SNR) refers to the ratio of energy consumption between signal and noise in a system composed of electronic devices. In the field of brain-computer interface, the signal is defined as the EEG signal of the cerebral cortex collected by the EEG instrument, and the noise is defined as the noise such as the electromyographic signal that interferes with the EEG signal. Since the energy consumption of the signal and the noise is not easy to calculate in the field of brain-computer interface, the amplitude of the signal and the noise is used to calculate the signal-to-noise ratio of the EEG signal. The calculation formula is as follows:
[0120]
[0121] Among them, V s Represents the effective value of the signal amplitude, V n Represents the effective value of the noise replica of the signal.
[0122] The accuracy evaluation index includes false alarm rate, Kappa coefficient, and mutual information; the false alarm rate (i.e., false positive rate, FPR) indicates the ratio of false alarms generated by the system when the user ensures an idle state;
[0123] The Kappa coefficient is a classification accuracy evaluation index based on the confusion matrix. The calculation formula of the Kappa value is as follows:
[0124]
[0125] Among them, p0 represents the observed consistency (correct model prediction) proportion, that is, the proportion of evaluators who actually agree; p e represents the expected agreement proportion, that is, the proportion of raters who would agree under random circumstances.
[0126] Assumed distance is the evaluation value of the BCI system for the identified object. When the test data is classified correctly, such as when the left hand is imagined for the i-th time, the evaluation distance D obtained by analyzing the EEG signal is (i) > 0, otherwise, if the i-th trial is in the right hand imagination, then the estimated distance D obtained by analyzing the EEG signal (i) <0; however, in the actual classification process, the EEG signal is interfered by other factors, which cannot make each classification task executed correctly. For the motor imagery classification task, its recognition rate reaches 100%; therefore, the evaluation value of each classification can be regarded as two categories, one is the correct classification and is related to the motor imagery task, containing effective information output, and the other is the wrong classification, which is the interference or noise signal that is irrelevant to the motor imagery task; from the perspective of the signal-to-noise ratio, the entire motor imagery recognition task is represented in the form of:
[0127]
[0128] Among them, SNR t is the signal-to-noise ratio (SNR), is the i-th data point at time t. {L}, {R} represent two different data sets or condition sets, such as the evaluation value sets of left-hand imagination and right-hand imagination, and var represents the variance of the evaluation value set. In order to obtain relevant information about motor imagination from the classifier evaluation results, according to Shannon's theorem, the mutual information I can be expressed as entropy difference, and the entropy H for a Gaussian process x is defined as follows:
[0129]
[0130] Where, e is the base of natural logarithm, δ x represents the standard deviation of the random variable x.
[0131] According to the entropy value definition, the output signal-to-noise ratio of the evaluation system is calculated, and the calculation formula of the mutual information of the motor imagery recognition task based on binary classification is as follows:
[0132] I t =0.5×log2(SNR t +1)
[0133] The larger the mutual information value, the better the classification effect of the BCI system and the higher the confidence of the classifier's output; the mutual information value reflects a better recognition rate to some extent, while a higher recognition rate makes it difficult to judge the value of the information.
[0134] The evaluation indexes of timeliness include information transmission rate, response time, and maximum kurtosis; the calculation process of information transmission rate is as follows:
[0135]
[0136] T=T s +T I
[0137]
[0138] Where N is the number of target options in the system, P is the recognition accuracy of each target option, B is the amount of information transmitted by a single target selection, T is the time for a single target selection, and T s represents the single target selection stimulus time, T I Indicates the interval between two selections.
[0139] The response time refers to the time required for the user to issue a BCI command. The calculation of a complete response time should be from the user issuing the control intention as the starting point, to the system collecting data on the length of time required for signal analysis, to the moment when the classification result is generated and output as the end point.
[0140] The maximum steepness of MI is an evaluation index based on mutual information. The calculation formula of the maximum steepness is as follows:
[0141] MI_maxsteepness = MI(t) / (t-3)
[0142] Where MI(t) is the mutual information value, and t>3.5s is required to avoid stress response.
[0143] The evaluation indexes of comfort include mental workload and driver satisfaction; the mental workload refers to the mental resources that the user spends to complete a task, that is, the degree of brain fatigue; the NASA workload index is used as a questionnaire to investigate the user's subjective feelings about the relevant system, among which NASA-TLX covers six dimensions, namely, spirit, performance, physical strength, time requirements, effort and frustration, each dimension has 20 levels, and the corresponding score distribution is 0-100, the smaller the value, the less workload, and vice versa, the more workload;
[0144] Driver satisfaction refers to the driver's satisfaction with the brain-computer interaction function during driving. This indicator reflects the driver's evaluation of the comfort, safety and overall experience of the brain-controlled car function. Driver satisfaction surveys are usually conducted through questionnaires, interviews, and other methods to collect drivers' opinions and evaluations. These survey results can help researchers develop improvement measures and provide drivers with a better brain-computer interaction experience.
[0145] Through questionnaire survey, various indicators related to comfort were evaluated using a score of 1 to 5; finally, based on the collected questionnaires, the data were analyzed and summarized to obtain specific data for the evaluation of the comfort of the expected functions of the brain-computer interface.
[0146] W 11 -Wz 52 is the weight of each indicator, where Wz 11 The weight of the first indicator of the first performance; the standardized value of each indicator score is Z ij , represents the calculated value of the jth indicator of the i-th performance;
[0147] like Figure 2 As shown, the evaluation method includes the following steps:
[0148] (1) Data standardization
[0149] After conducting the experiment and collecting the EEG signals, the indicators are calculated, and each indicator is standardized to obtain a standardized data matrix;
[0150] For positive indicators (larger values are better):
[0151]
[0152] For negative indicators (smaller values are better):
[0153]
[0154] Among them, Z ij For x ij The standardized score, min(x ij ) is the minimum value of all sample values of the jth indicator of the i-th performance dimension. max(x ij ) is the maximum value among all sample values of the jth indicator of the ith performance dimension.
[0155] (2) Calculate the entropy value of the indicator
[0156] Calculate the entropy value e of the jth indicator j :
[0157]
[0158] in:
[0159] e j represents the entropy value of the jth indicator;
[0160] m is the total number of samples;
[0161] k is a constant, and the calculation formula is: Used to normalize the entropy value to ensure that the entropy value is within a reasonable range;
[0162] p ij It represents the proportion of the i-th sample on the j-th index. The calculation formula is: Where Z ij For x ij The standardized score.
[0163] (3) Calculate the weight of the indicator
[0164] Calculate the weight Wz of the jth indicator j :
[0165]
[0166] (4) Calculate the weight of the performance dimension level
[0167] The analytic hierarchy process (AHP) was used to determine the weights of different performance dimensions. The steps are as follows:
[0168] (4.1) Constructing the judgment matrix
[0169] There are five performance dimensions. We construct a 5×5 judgment matrix A, where a ij Indicates the importance of performance dimension i relative to performance dimension j; the judgment matrix A satisfies the following conditions:
[0170] a ij >0
[0171]
[0172] a ii =1
[0173] (4.2) Calculate the eigenvector and maximum eigenvalue of the judgment matrix
[0174] Calculate the maximum eigenvalue λ of the judgment matrix A max and its corresponding eigenvector V = (v1, v2, ..., v n ) T , where v n represents the nth eigenvector in the eigenvectors.
[0175] (4.3) Normalized eigenvector
[0176] Normalize the feature vector V, and determine the weights Wx1, Wx2, Wx3, Wx4, and Wx5 of the performance dimension hierarchy according to the improved entropy method;
[0177] (4.4) Consistency check
[0178] In order to ensure the consistency of the judgment matrix, a consistency test is required; the consistency index CI and consistency ratio CR are calculated:
[0179]
[0180]
[0181] Among them, RI is the random consistency index, which depends on the order n of the matrix;
[0182] (5) Calculation function weight
[0183] The weights of each functional dimension Wg1, Wg2, Wg3, and Wg4 are calculated by analytic hierarchy process;
[0184] (6) Calculate the overall evaluation score
[0185] Calculate the function score Sk (k = 1, 2, 3, 4):
[0186]
[0187] Calculate the overall evaluation score:
[0188]
[0189] Among them, l=4.
[0190] Through the above steps, the brain-computer interface control performance can be comprehensively evaluated and a comprehensive score for each sample can be obtained.
Claims
1. A method for testing and evaluating the expected functional interaction of a vehicle brain-computer interface, characterized in that: The evaluation system architecture consists of four layers: The first layer is the target layer: the core of the target layer is to evaluate the expected functional interaction level of the brain-computer interface (BMI), measure the interaction effect and user experience of the brain-computer interface in actual car control, and record the performance evaluation results. The target layer is the basis for guiding the evaluation of the functional layer; The second layer is the functional layer: the functional layer evaluates the expected functions achieved by the brain-computer interface, including one or more functions of controlling brakes, fatigue monitoring, controlling window lifting, controlling air conditioning, and controlling audio; The third layer is the performance layer: the performance layer further subdivides the evaluation of brain-computer interface functions under each expected function into different performance dimensions, including one or more dimensions of safety, anti-interference, accuracy, timeliness, and comfort; The safety described is used to evaluate the system's safety assurance capabilities under various operating conditions; The anti-interference performance is used to analyze the system's ability to resist external interference in a complex environment and ensure stable signal transmission; The accuracy is used to measure the accuracy of user intention recognition and the precision of instruction execution; The timeliness is used to evaluate the time delay from the user issuing a command to the system responding; The comfort level is used to examine the user's comfort and fatigue when using the brain-computer interface; The fourth layer is the indicator layer: the indicator layer concretizes the performance dimensions into quantifiable evaluation indicators; The security evaluation index includes recognition accuracy; The anti-interference evaluation index includes one or more of the following indicators: the number of selectable targets, false trigger rate, and signal-to-noise ratio; The accuracy evaluation index includes one or more of the following indicators: false alarm rate, Kappa coefficient, and mutual information; The evaluation index of timeliness includes one or more of information transmission rate, response time and maximum kurtosis; The evaluation index of comfort includes one or two of the following indexes: mental workload and driver satisfaction; All indicators are objective numerical quantitative indicators; The evaluation method includes the following steps: (1) Data standardization The original data is standardized. There are l functions, each function has n performance dimensions, each performance dimension has m indicators, and the original data matrix is X = (x ij ) n×m ; where x ij Represents the original value of the jth indicator of the i-th performance dimension of a certain function, i=0,1,…n; j=0,1,…m; For positive indicators: Larger values indicate better performance; For negative indicators: Smaller values indicate better performance; Among them, Z ij For x ij The standardized score, min(x ij ) is the minimum value of all sample values of the jth indicator of the i-th performance dimension, max(x ij ) is the maximum value among all sample values of the jth indicator of the i-th performance dimension; (2) Calculate the entropy value of the indicator Calculate the entropy value e of the jth indicator j : in: e j represents the entropy value of the jth indicator; m is the total number of samples; k is a constant, and the calculation formula is: Used to normalize the entropy value to ensure that the entropy value is within a reasonable range; p ij It represents the proportion of the i-th sample on the j-th index. The calculation formula is: Where Z ij For x ij Standardized scores; The entropy value reflects the amount of information of the indicator. The larger the entropy value, the more dispersed the information of the indicator and the smaller the weight. On the contrary, the smaller the entropy value, the more concentrated the information of the indicator and the greater the weight. (3) Calculate the weight of the indicator Calculate the weight Wz of the jth indicator j : The purpose of calculating weights is to determine the importance of each indicator in the comprehensive evaluation. The larger the weight, the greater the impact of the indicator on the comprehensive evaluation; (4) Calculate the weight of the performance dimension level The analytic hierarchy process (AHP) was used to determine the weights of different performance dimensions. The steps are as follows: (4.1) Constructing the judgment matrix Suppose there are n performance dimensions, construct an n×n judgment matrix A, where a ij Indicates the importance of performance dimension i relative to performance dimension j; the judgment matrix A satisfies the following conditions: a ij >0 a ii =1 (4.2) Calculate the eigenvector and maximum eigenvalue of the judgment matrix Calculate the maximum eigenvalue λ of the judgment matrix A max and its corresponding eigenvector V = (v1, v2, ..., v n ) T , where v n represents the nth eigenvector in the eigenvector; (4.3) Normalized eigenvector Normalize the feature vector V to obtain the weight Wx of each performance dimension level i : (4.4) Consistency check In order to ensure the consistency of the judgment matrix, a consistency test is required; the consistency index CI and consistency ratio CR are calculated: Among them, RI is the random consistency index, which depends on the order n of the matrix; If CR < 0.1, the judgment matrix has satisfactory consistency; otherwise, the judgment matrix needs to be adjusted; (5) Calculate the weight of the function Under the target level, the weight Wg of the i-th function is calculated by the hierarchical analysis method i , the specific steps are the same as above; (6) Calculate the overall evaluation score The weight of each indicator is multiplied by its standardized value to obtain the comprehensive score of each sample. The score Sk of the kth function is: The final total performance score S of l functions is: The comprehensive score reflects the comprehensive performance of each sample on all indicators. The higher the score, the better the comprehensive performance.
2. A vehicle brain-computer interface expected function interaction test and evaluation method according to claim 1, characterized in that: The safety of brain-computer interaction is evaluated by the recognition accuracy; the recognition accuracy is defined as follows: for the same set of test data, the recognition result is RR (Recognition Result), where the correct result is represented by TR (Ture Recognition) and the wrong recognition result is represented by FR (False Recognition). The recognition accuracy ERA (EEG Recognition Accuracy) is calculated by the following formula:
3. A vehicle brain-computer interface expected function interaction test and evaluation method according to claim 1, characterized in that: The anti-interference index of brain-computer interaction includes three indicators: the number of selectable targets, false trigger rate, and signal-to-noise ratio; The optional target number indicates the number of command categories that the system can output; if the optional target number is less than the number of commands of the control task, the system is incomplete; if it is greater than the number of optional commands of the control task, the system resources will be occupied, reducing the communication efficiency of the system; The false trigger rate refers to the probability of the system being falsely triggered in an idle state, and the calculation formula is: Among them, FP represents the number of samples predicted as positive samples but actually are negative samples, which specifically corresponds to the number of control instructions issued incorrectly in the idle state; TN represents the number of samples predicted as negative samples but actually are negative samples, which corresponds to the situation where instructions are avoided in the idle state; The signal-noise ratio (SNR) refers to the ratio of energy consumption between signal and noise in a system composed of electronic devices. The signal is defined as the EEG signal of the cerebral cortex collected by the EEG instrument, and the noise is defined as the noise such as the electromyographic signal that interferes with the EEG signal. The signal-noise ratio of the EEG signal is calculated using the amplitude of the signal and the noise. The calculation formula is as follows: Among them, V s Represents the effective value of the signal amplitude, V n Represents the effective value of the noise replica of the signal.
4. A vehicle brain-computer interface expected function interaction test and evaluation method according to claim 1, characterized in that: The accuracy requirements of brain-computer interaction are evaluated by false alarm rate, Kappa coefficient and mutual information; The false alarm rate refers to the ratio of false alarms generated by the system when the user ensures an idle state; The Kappa coefficient is a classification accuracy evaluation index based on the confusion matrix. The calculation formula of the Kappa value is as follows: Among them, p0 represents the observed consistency (correct model prediction) proportion, that is, the proportion of evaluators who actually agree; p e represents the expected agreement proportion, i.e., the proportion of raters who agree on the same subject under random circumstances; Assumed distance It is the evaluation value of the BCI system for the identified object. When the test data is correctly classified, the evaluation distance D obtained by analyzing the EEG signal (i) > 0, otherwise, the estimated distance D obtained by analyzing the EEG signal (i) <0; the evaluation values of the classification include two categories, one is the correct classification and is related to the motor imagery task, containing effective information output, and the other is the wrong classification, which is interference or noise signal irrelevant to the motor imagery task; from the perspective of the signal-to-noise ratio, the entire motor imagery recognition task, its representation is defined as: Among them, SNR t is the signal-to-noise ratio (SNR), is the i-th data point at time t, {L}, {R} represent two different data groups or condition sets, such as the evaluation value sets of left-hand imagination and right-hand imagination, var represents the variance of the evaluation value set, in order to obtain relevant information about motor imagination from the classifier evaluation results, according to Shannon's theorem, the mutual information I is expressed as entropy difference, and the entropy H for a Gaussian process x is defined as follows: Where, e is the base of natural logarithm, δ x represents the standard deviation of the random variable x; According to the entropy value definition, the output signal-to-noise ratio of the evaluation system is calculated, and the calculation formula of the mutual information of the motor imagery recognition task based on binary classification is as follows: IN t =0.5×log2(SNR t +1) The larger the value of mutual information, the better the classification effect of the BCI system and the higher the confidence of the output of the classifier.
5. A vehicle brain-computer interface expected function interaction test and evaluation method according to claim 1, characterized in that: The information transmission rate is a commonly used indicator for measuring information transmission. Its unit is usually expressed in bit / min. Its calculation process is as follows: T=T s +T I Where N is the number of target options in the system, P is the recognition accuracy of each target option, B is the amount of information transmitted by a single target selection, T is the time for a single target selection, and T s represents the single target selection stimulus time, T I Indicates the interval between two selections; The response time refers to the time required for the user to issue a BCI command. The calculation of a complete response time should be from the user issuing a control intention as the starting point, to the system collecting data on the length of time required for signal analysis, to the moment when the classification result is generated and output as the end point; The maximum steepness of MI is an evaluation index based on mutual information. The calculation formula of the maximum steepness is as follows: MI_maxsteepness = MI(t) / (t-3) Where MI(t) is the mutual information value, and t>3.5s is required to avoid stress response.
6. A vehicle brain-computer interface expected function interaction test and evaluation method according to claim 1, characterized in that: The mental workload refers to the mental resources that the user expends in completing a task, i.e., the degree of mental fatigue. The subjective workload assessment technique or the NASA workload index questionnaire is used to investigate the user's subjective feelings about the relevant system. Driver satisfaction refers to the driver's satisfaction with the brain-computer interaction function during the driving process. This indicator reflects the driver's evaluation of the comfort, safety and overall user experience of the brain-controlled car function.
