Application efficiency evaluation method and device for brain-computer interface system

Abstract

The invention provides an application efficiency evaluation method and device for a brain-computer interface system. The method comprises the following steps: determining a typical mission profile; establishing a preset evaluation index system considering the system environment performance and the ability of the tested person; testing the electroencephalogram interface system based on the typical task profile, and obtaining objective test data related to the system environment performance and subjective test data related to the ability of the testee; determining an evaluation value of each preset evaluation index in a preset evaluation index system based on the objective test data and the subjective test data; determining the weight of each preset evaluation index by using an analytic hierarchy process; and calculating a comprehensive efficiency evaluation value of the brain-computer interface system based on the evaluation value of each preset evaluation index and the corresponding weight. Therefore, the problem that quantitative evaluation of the system efficiency is difficult due to the fact that a traditional method is poor in adaptability, high in expert dependency degree and the like in a brain-computer interface scene is solved, and a reference basis and technical support are provided for a user to evaluate the comprehensive efficiency of the brain-computer interface system.

Description

Technical Field

[0001] The present invention relates to the field of front-end development technology, and in particular to a method and device for evaluating the application efficiency of a brain-computer interface system. Background Art

[0002] With the continuous development of artificial intelligence and brain-computer interface technology, the application scenarios of brain-computer interface systems have become more diverse. In recent years, brain-computer interface systems have made progress in paradigm coding and EEG decoding research. However, there is still a lack of qualitative evaluation of the impact of brain-computer interface system parameters on the performance of the usage scenario in terms of actual system application effects. In addition, the existing technology for evaluating the effectiveness of collaborative tasks of human-computer systems uses task performance indicators to single-handedly evaluate the performance of human-computer system tasks, without considering that human factors, automation factors, and situational factors will affect the performance and safety of the human-computer system by affecting human processes or states. Summary of the Invention

[0003] To this end, the present invention provides a method and device for evaluating the application efficiency of a brain-computer interface system, aiming to solve the technical problems that the existing technology is difficult to qualitatively evaluate the impact of brain-computer interface system parameters on the performance of usage scenarios and does not consider the impact of human factors on the execution of human-computer system tasks.

[0004] To achieve the above objectives, the present invention adopts the following technical solutions:

[0005] According to a first aspect of the present invention, the present invention provides a method for evaluating the performance of a brain-computer interface system, the method comprising:

[0006] Determine typical mission profiles; and establish a pre-set evaluation indicator system that takes into account the system environment performance and the capabilities of the test subjects;

[0007] Performing a test on the EEG interface system based on the typical task profile to obtain objective test data related to the system environmental performance and subjective test data related to the test subject's ability;

[0008] Determining the evaluation value of each preset evaluation indicator in the preset evaluation indicator system based on the objective test data and the subjective test data; and determining the weight of each preset evaluation indicator using the hierarchical analysis method;

[0009] The comprehensive performance evaluation value of the brain-computer interface system is calculated based on the evaluation value of each of the preset evaluation indicators and the corresponding weight.

[0010] Furthermore, determining the typical mission profile includes:

[0011] Obtaining task-related information; the task-related information includes typical application scenarios and task assumptions and / or task objectives corresponding to the typical operation scenarios;

[0012] Performing multi-dimensional task analysis on the task-related information using a spatiotemporal constraint model to establish a typical task profile including at least one task phase;

[0013] The mission phases include a preparation phase, an activation phase, a preparation phase, a maintenance phase and / or a termination phase.

[0014] Furthermore, the preset evaluation indicators include objective performance indicators and subjective performance indicators;

[0015] The objective performance indicators include at least one of the following: operating environment, system weight, EEG signal interference factors, EEG signal acquisition influencing factors, EEG signal transmission influencing factors, EEG signal decoding influencing factors, total success rate, total task adaptation rate, target recognition performance, total task completion time, and delay time;

[0016] The subjective performance index includes at least one of subjective feeling factors, personnel fatigue, personnel situational awareness, personnel emotional arousal, and personnel subjective load.

[0017] Furthermore, the obtaining of objective test data related to the system environment performance and subjective test data related to the test subject's ability includes:

[0018] Using a preset device state detection model to collect operating data of the brain-computer interface system during the test as objective test data; the device state detection model includes a customized evaluation scale and / or a counter; the objective test data includes performance values ​​corresponding to each of the objective performance indicators;

[0019] The subjective test data of the test subjects during the test process are obtained using a preset subjective questionnaire scale; the subjective test data include the performance values ​​corresponding to each of the subjective performance indicators.

[0020] Furthermore, before performing the test on the EEG interface system based on the typical task profile, the method further includes:

[0021] Establish a pre-experimental training mechanism for the test subjects, specifically including:

[0022] By simulating tasks for the brain-computer interface system, the tested person is given standardized operation training until the operator reaches a preset operation proficiency threshold to ensure the comparability and consistency of the test results.

[0023] Furthermore, determining the evaluation value of each preset evaluation indicator in the preset evaluation indicator system based on the objective test data and the subjective test data includes:

[0024] The objective test data and the subjective test data of different dimensions and / or types are normalized using hierarchical normalization rules that match each of the preset evaluation indicators, and the performance values ​​corresponding to each of the preset evaluation indicators are unified into the [0, 1] interval as the evaluation values ​​of each of the preset evaluation indicators.

[0025] Furthermore, the method of determining the weight of each of the preset evaluation indicators by using the analytic hierarchy process includes:

[0026] Obtain statistical results of the weighted scores of the experts for each of the preset evaluation indicators, and construct a judgment matrix using the statistical results of the weighted scores;

[0027] Calculating the weights of multiple elements in the judgment matrix by using the eigenvalue method and the eigenvector method;

[0028] A consistency check is performed on the judgment matrix after weight calculation, and the weights of the elements that pass the consistency check are used as the weights of each of the preset evaluation indicators.

[0029] Furthermore, the consistency check of the judgment matrix after weight calculation includes:

[0030] The consistency index is calculated based on the eigenvalue and order of the judgment matrix. The mathematical expression is:

[0031]

[0032] Where CI represents the consistency index; λ max represents the eigenvalue; n represents the order;

[0033] And, searching for a corresponding random consistency ratio value in a consistency ratio random indicator table according to the order of the judgment matrix;

[0034] The consistency ratio is calculated according to the consistency index and the random consistency ratio value, and the mathematical expression is:

[0035]

[0036] Wherein, CR represents the consistency ratio; RCR represents the random consistency ratio value;

[0037] Determine whether the consistency ratio is less than a preset consistency threshold; if the consistency ratio is less than the preset consistency threshold, consider that the judgment matrix passes the consistency check; if the consistency ratio is greater than or equal to the preset consistency threshold, readjust the judgment matrix.

[0038] Furthermore, the calculation of the comprehensive effectiveness evaluation value of the brain-computer interface system based on the evaluation value of each of the preset evaluation indicators and the corresponding weights includes:

[0039] A comprehensive performance evaluation function for the brain-computer interface system is established, and the mathematical expression is:

[0040]

[0041] Wherein, D represents the comprehensive performance evaluation value of the brain-computer interface system; CR i Represents the evaluation value of the i-th preset evaluation indicator; α i represents the weight of the i-th preset evaluation indicator; N represents the total number of preset evaluation indicators;

[0042] The evaluation value and corresponding weight of each of the preset evaluation indicators are input into the comprehensive performance evaluation function to obtain the comprehensive performance evaluation value of the brain-computer interface system.

[0043] According to a second aspect of the present invention, the present invention provides an apparatus for evaluating the performance of a brain-computer interface system, the apparatus comprising:

[0044] System mission determination module, used to determine typical mission profiles;

[0045] Evaluation indicator construction module, used to establish a preset evaluation indicator system that takes into account the system environment performance and the test subject's capabilities;

[0046] A test data acquisition module, configured to perform a test on the EEG interface system based on the typical task profile, and obtain objective test data related to the system environmental performance and subjective test data related to the test subject's ability;

[0047] an intermediate data calculation module, configured to determine, based on the objective test data and the subjective test data, an evaluation value of each preset evaluation indicator in the preset evaluation indicator system; and to determine a weight of each preset evaluation indicator using a hierarchical analysis method;

[0048] The comprehensive performance evaluation module is used to calculate the comprehensive performance evaluation value of the brain-computer interface system based on the evaluation value of each preset evaluation indicator and the corresponding weight.

[0049] The present invention adopts the above technical solution and has at least the following beneficial effects:

[0050] Through the solution of the present invention, a typical task profile is determined; and a preset evaluation index system is established that takes into account the system environment performance and the test subject's ability; based on the typical task profile, the EEG interface system is tested to obtain objective test data related to the system environment performance and subjective test data related to the test subject's ability; based on the objective test data and the subjective test data, the evaluation value of each preset evaluation index in the preset evaluation index system is determined; and the weight of each preset evaluation index is determined using the hierarchical analysis method; the comprehensive performance evaluation value of the brain-computer interface system is calculated based on the evaluation value of each preset evaluation index and the corresponding weight. In this way, the problem of the difficulty in quantitatively evaluating system performance due to the poor adaptability of traditional methods in brain-computer interface scenarios and high dependence on experts is solved, and a reference basis and technical support for users to evaluate the comprehensive performance of brain-computer interface systems is provided.

[0051] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0053] Figure 1 A flow chart of a method for evaluating the application effectiveness of a brain-computer interface system according to an embodiment of the present invention is shown;

[0054] Figure 2 A schematic diagram of a process for constructing a preset evaluation index system according to an embodiment of the present invention is shown;

[0055] Figure 3 A schematic diagram showing the structure of a preset evaluation index system provided by an embodiment of the present invention is shown;

[0056] Figure 4 A schematic flow chart of a method for evaluating the effectiveness of a brain-computer interface system according to another embodiment of the present invention is shown;

[0057] Figure 5 A schematic diagram of the process of expert scoring statistics provided by one embodiment of the present invention is shown;

[0058] Figure 6 A schematic structural diagram of an apparatus for evaluating the application effectiveness of a brain-computer interface system provided by one embodiment of the present invention is shown. DETAILED DESCRIPTION

[0059] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0060] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, the elements defined by the phrase "comprising..." do not exclude the presence of other identical elements in the process, method, article, or device comprising the elements.

[0061] The embodiment of the present invention provides a method for evaluating the application efficiency of a brain-computer interface system. Figure 1 As shown, it may at least include the following steps S101 to S104:

[0062] Step S101 , determining a typical task profile; and establishing a preset evaluation indicator system that takes into account system environment performance and test subject capabilities.

[0063] In an embodiment of the present invention, a typical task profile is used to determine the specific tests to be performed on the brain-computer interface system. To determine the typical task profile, task-related information can be obtained first. Then, a spatiotemporal constraint model is used to perform multi-dimensional task analysis on the task-related information to establish a typical task profile that includes multiple task phases.

[0064] Task-related information includes typical application scenarios and the corresponding task assumptions and objectives for these scenarios. For example, in typical application scenarios such as forests, mountains, and oceans, the BCI system's application task assumptions and objectives are clearly defined, such as rapid and accurate identification of weakly hidden targets in the scene and user satisfaction with the system's performance. Furthermore, a spatiotemporal constraint model is used to perform multi-dimensional task decomposition of task-related information, such as preparation, activation, preparation, maintenance, and termination, to establish a typical task profile corresponding to typical application scenarios for BCI systems in both indoor and outdoor conditions.

[0065] The construction of a spatiotemporal constraint model involves spatiotemporal modeling of multi-dimensional parameters. First, task phases are divided based on task-related information, such as medical rehabilitation and environmental control, and spatiotemporal anchor points are defined for each phase. These multi-dimensional parameters are then spatiotemporally mapped, and corresponding test data is collected. For example, time-varying environmental parameters, including temperature, light, and noise, are collected in real time by sensors and mapped to spatiotemporal units. In practice, light intensity in outdoor scenes varies over time, and a time series model of light intensity can be built within spatiotemporal units. Another example is device dynamic parameters, including electrode impedance and signal-to-noise ratio, which are acquired through the device status monitoring module. For example, poor electrode contact leading to a decrease in signal quality triggers a spatiotemporal update of the device status parameters. Other parameters include user status, such as attention level and fatigue, which are comprehensively assessed through physiological signals (such as EEG and heart rate) and behavioral data (such as reaction time). In practice, the user's fatigue state is analyzed using the alpha wave power spectrum of the EEG signal and mapped to the corresponding spatiotemporal units.

[0066] Furthermore, if Figure 2 As shown, the embodiment of the present invention also establishes a preset evaluation index system that takes into account the system environment performance and the test subject's ability.

[0067] The embodiments of the present invention take into account that human, automation, and situational factors all affect the performance and safety of the BCI system by influencing human processes or states. Task efficiency, automation behavior efficiency, human behavior efficiency, and cognitive and physiological precursors of human behavior all affect the execution of tasks in the human-computer system. In other words, the evaluation of BCI system energy efficiency in the embodiments of the present invention not only focuses on task execution, but also needs to pay attention to behavioral performance and human status that affect task performance and system safety.

[0068] Specifically, a comprehensive evaluation index system for the application effectiveness of the brain-computer interface system is established based on the functional objectives and abstract functional layers. The functional objectives are mainly considered from two aspects: recognition results and system status. Indicators such as task success rate in different scenarios are required. The system status field requires ensuring the safety and adaptability of the brain-computer system. The abstract functional layer divides the entire task information system test content into horizontal layers according to the mission tasks, and then divides it vertically based on the task profile within each layer. Figure 3 In this embodiment, a typical task profile of a brain-computer interface system is combined with a full-factor modeling of environment, task, and personnel. This model establishes a preset evaluation indicator system for the brain-computer interface system under typical task profile conditions, focusing on the system environment, the task capabilities of the brain-computer interface system, and the capabilities of the subjects. The preset evaluation indicator system includes multiple preset evaluation indicators for each of these three aspects: the system environment, the task capabilities of the brain-computer interface system, and the capabilities of the subjects.

[0069] The embodiments of the present invention can be divided into objective performance indicators and subjective performance indicators based on the different collection methods of different preset evaluation indicators. Generally speaking, the preset evaluation indicators of the system environment and the task capability of the brain-computer interface system can be used as the objective performance indicators. The objective performance indicators may include the application environment, system weight, EEG signal interference factors, EEG signal collection influencing factors, EEG signal transmission influencing factors, EEG signal decoding influencing factors, total success rate, total task adaptation rate, target recognition performance, total task completion time and delay time; subjective performance indicators may include subjective feeling factors, personnel fatigue, personnel situational awareness, personnel emotional arousal and personnel subjective load.

[0070] It can be understood that the preset evaluation index system is used to conduct comprehensive performance evaluation of the brain-computer structure system, and the brain-computer interface system can be feedback optimized based on the comprehensive performance evaluation structure.

[0071] Step S102 : Testing the EEG interface system based on typical task profiles to obtain objective test data related to the system environment performance and subjective test data related to the test subject's ability.

[0072] like Figure 4 As shown in the figure, the EEG interface system is tested in typical task scenarios based on typical task profiles, and objective and subjective test data generated during the test are obtained. Specifically, the BCI system's operating data during the test can be collected using a preset device status detection model as objective test data; and the subject's subjective test data during the test can be obtained using a preset subjective questionnaire scale. The subjective test data includes the performance values ​​corresponding to each subjective performance indicator.

[0073] The device status detection model can include customized evaluation scales, counters, stopwatches, etc., which are used to detect the performance values ​​corresponding to objective performance indicators, forming an objective evaluation data set that reflects environmental adaptability and task execution capabilities, namely objective test data. For subjective evaluation indicators, a subject subjective scale is used to quantitatively evaluate the subjective experience of the subjects under multi-dimensional task loads, namely subjective test data. As shown in Table 1, an example of a subject subjective scale is shown:

[0074] Table 1 (Example of subjective quantity expression of subjects)

[0075]

[0076]

[0077] It should be noted that before performing tests on the EEG interface system based on typical task profiles, the embodiment of the present invention also establishes a pre-experimental training mechanism for the persons being tested, that is, through task simulation of the brain-computer interface system, the persons being tested are trained in standardized operations until the operators reach a preset operation proficiency threshold, so as to ensure the comparability and consistency of the test results.

[0078] Step S103 : determining the evaluation value of each preset evaluation indicator in the preset evaluation indicator system based on the objective test data and the subjective test data; and determining the weight of each preset evaluation indicator using the analytic hierarchy process.

[0079] It is understandable that, based on the existence of various types and dimensions of data in the objective test data and subjective test data, these test data need to be normalized for the convenience of subsequent calculations. Specifically, the hierarchical normalization rules that match the preset evaluation indicators can be used to normalize the objective test data and subjective test data of different dimensions and / or types, and the performance values ​​corresponding to the preset evaluation indicators are unified to the [0, 1] interval as the evaluation values ​​of the preset evaluation indicators. Thus, the performance values ​​of the objective test data and subjective test data obtained by the brain-computer interface system in typical task scenarios are integrated, solving the problem of dimensional unification in the multi-dimensional evaluation of the brain-computer interface system.

[0080] Furthermore, the embodiment of the present invention uses the Analytic Hierarchy Process to determine the weights of each preset evaluation indicator. The Analytic Hierarchy Process (AHP) is a decision analysis method that combines qualitative and quantitative methods. It can decompose the decision problem layer by layer into a series of independent decision indicators, and quantitatively analyze the indicator importance information given by experts to obtain the decision result. It should be noted that there is no universal fixed weight value for each preset evaluation indicator in the embodiment of the present invention. The weight selection needs to be combined with specific scenarios, data characteristics and evaluation objectives, and dynamically determined by scientific methods. Therefore, by specifically designing a preset evaluation indicator system for the brain-computer interface system, and determining the weight for each preset evaluation indicator to conduct a comprehensive application evaluation, the use value of the brain-computer interface system can be enhanced, and theoretical guidance and support can be provided for the development and application of the brain-computer interface system.

[0081] Specifically, the analytic hierarchy process includes the following steps S1 to S4:

[0082] Step S1: Establishing a hierarchical structure: Decompose the comprehensive effectiveness evaluation problem of the brain-computer interface system into different levels, generally including the target level, the criterion level, and the solution level. In this embodiment of the present invention, a hierarchical analysis is performed with the goal of determining the weights of different preset evaluation indicators.

[0083] Step S2: Constructing a judgment matrix: Obtain the statistical results of the experts' weighted scores for each preset evaluation indicator, and use the statistical results of the weighted scores to construct a judgment matrix.

[0084] In actual operation, the statistical data of weighted expert scores can be obtained through anonymous letter inquiries, summarized feedback, survey scoring, etc., and the weight determination process of expert scoring indicators can be completed, such as Figure 5 As shown, you can first collect and organize the data of the preset evaluation indicators to form an expert inquiry form, provide the information to the experts, and the experts will score and submit their opinions for summary. If the opinions are consistent, the expert scores for each preset evaluation indicator will be statistically calculated; if the opinions are inconsistent, the experts will re-score. Generally, if you think it is important, you will be assigned 5 points, if you think it is relatively important, you will be assigned 4 points, if you think it is generally important, you will be assigned 3 points, if you think it is not important, you will be assigned 2 points, and if you think it is not important, you will be assigned 1 point, as shown in Table 2 below:

[0085] Table 2 (Example of expert scoring table)

[0086]

[0087] Based on the statistical results of expert scoring, elements at the same level are compared pairwise to construct a judgment matrix.

[0088] Step S3: Calculate weights: Calculate the weights of multiple elements in the judgment matrix using the eigenvalue method and the eigenvector method.

[0089] Specifically, for the judgment matrix of each level, the eigenvector is calculated, each column of the judgment matrix is ​​normalized so that the sum of the elements of each column is 1, and the average value of each row of the normalized judgment matrix is ​​taken to obtain an eigenvector. Each element of the eigenvector is normalized so that the sum of all elements is 1. Each element of the normalized eigenvector is the weight of the corresponding element, and finally the weight of each element in each level can be obtained.

[0090] Step S4: consistency check: perform consistency check on the judgment matrix after weight calculation, and use the weights of the elements that pass the consistency check as the weights of each preset evaluation indicator.

[0091] As you can understand, checking the consistency of the judgment matrix ensures the rationality of the decision-making process. A consistency check is performed to ensure the rationality of the judgment matrix. The consistency index (CI) and consistency ratio (CR) are used to assess the consistency level of the judgment matrix. If the CR is less than a preset consistency threshold, the judgment matrix is ​​considered consistent.

[0092] Specifically, the judgment matrix after weight calculation is subjected to consistency check, including: calculating the consistency index according to the eigenvalue and order of the judgment matrix; and searching for the corresponding random consistency ratio value in the consistency ratio random index table according to the order of the judgment matrix; and then calculating the consistency ratio according to the consistency index and the random consistency ratio value.

[0093] Among them, the mathematical expression of the consistency index is:

[0094]

[0095] Where CI represents the consistency index; λ max represents the eigenvalue; n represents the order.

[0096] The mathematical expression of the consistency ratio is:

[0097]

[0098] Where CR represents the consistency ratio and RCR represents the random consistency ratio value.

[0099] In practice, the Random Index (RI) can be calculated based on the order n of the judgment matrix. The RI value can be found in the consistency ratio random index table. The corresponding RI value is selected based on the size of n. Then, based on the order n of the judgment matrix, the corresponding random consistency ratio value RCR is found in the consistency ratio random index table.

[0100] Then, it is determined whether the consistency ratio is less than a preset consistency threshold; if the consistency ratio is less than the preset consistency threshold, the judgment matrix is ​​considered to have passed the consistency check; if the consistency ratio is greater than or equal to the preset consistency threshold, the judgment matrix is ​​readjusted. Preferably, the preset consistency threshold is 0.1.

[0101] In actual operation, the embodiment of the present invention uses the environment, system weight, EEG signal interference factors, EEG signal acquisition influencing factors, EEG signal transmission influencing factors, EEG signal decoding influencing factors, total success rate, total task adaptation rate, target recognition performance, total task completion time, delay time, as well as subjective perception factors, personnel fatigue, personnel situational awareness, personnel emotional arousal, and personnel subjective load as factors to construct a multi-order judgment matrix for hierarchical analysis, and analyzes to obtain the eigenvector and the corresponding weight value of each item. Combined with the eigenvector, the maximum eigenroot λ can be calculated. max , and then the CI value and CR value can be calculated using the maximum eigenvalue to perform consistency test.

[0102] It should be noted that embodiments of the present invention can also incorporate a real-time weight update mechanism. Its core objective is to enable the evaluation system to adapt to the environment, task, and subject status through dynamic feedback. Based on data dynamics, task phases, environmental feedback, and algorithm optimization, dynamic weight calibration is achieved through feedback mechanisms and adaptive models, ensuring the timeliness and accuracy of the evaluation system.

[0103] Step S104: Calculate the comprehensive performance evaluation value of the brain-computer interface system based on the evaluation values ​​of each preset evaluation indicator and the corresponding weights.

[0104] Based on the capability analysis of typical task profiles, a comprehensive effectiveness evaluation function for the brain-computer interface system is established. The mathematical expression is:

[0105]

[0106] Where D represents the comprehensive performance evaluation value of the brain-computer interface system; CR i Represents the evaluation value of the i-th preset evaluation indicator; α i represents the weight of the i-th preset evaluation indicator; N represents the total number of preset evaluation indicators;

[0107] Furthermore, the evaluation values ​​and corresponding weights of each preset evaluation indicator are input into the comprehensive performance evaluation function to obtain the comprehensive performance evaluation value of the brain-computer interface system. The comprehensive performance evaluation value can be mapped to energy efficiency levels, such as excellent / good / medium / poor, to verify the effectiveness of the performance evaluation of the brain-computer interface system.

[0108] An embodiment of the present invention provides an application efficiency evaluation method for a brain-computer interface system. Through the task profile of the brain-computer interface system application scenario, a collaborative task efficiency index evaluation framework of the brain-computer interface system is constructed; based on the hierarchical analysis and dynamic weight fusion mechanism, performance parameters and preset evaluation index data during the system operation are collected, a hierarchical analysis method is established to establish an efficiency model, and expert scoring is performed to obtain various test coefficients, and an application efficiency evaluation system for the brain-computer interface system is constructed. This solves the problem that the traditional hierarchical analysis method has poor adaptability in brain-computer interface scenarios and high expert dependence, which makes it difficult to quantitatively evaluate system efficiency. It provides a reference basis and technical support for users to conduct comprehensive efficiency evaluation of brain-computer interface systems.

[0109] Further, as Figure 1 The embodiment of the present invention provides a device for evaluating the application efficiency of a brain-computer interface system, such as Figure 6 As shown, the apparatus may include: a system task determination module 610 , an evaluation index construction module 620 , a test data collection module 630 , an intermediate data calculation module 640 and a comprehensive effectiveness evaluation module 650 .

[0110] The system mission determination module 610 may be used to determine a typical mission profile;

[0111] The evaluation index construction module 620 can be used to establish a preset evaluation index system that takes into account the system environment performance and the test subject's ability;

[0112] The test data acquisition module 630 can be used to perform tests on the EEG interface system based on typical task profiles to obtain objective test data related to the system environment performance and subjective test data related to the test subject's ability;

[0113] The intermediate data calculation module 640 can be used to determine the evaluation value of each preset evaluation indicator in the preset evaluation indicator system based on the objective test data and the subjective test data; and to determine the weight of each preset evaluation indicator using the hierarchical analysis method;

[0114] The comprehensive performance evaluation module 650 can be used to calculate the comprehensive performance evaluation value of the brain-computer interface system based on the evaluation values ​​of each preset evaluation indicator and the corresponding weights.

[0115] It should be noted that for other corresponding descriptions of the functional modules involved in the device for evaluating the application effectiveness of a brain-computer interface system provided by the embodiment of the present invention, please refer to Figure 1 The corresponding description of the method shown will not be repeated here.

[0116] Those skilled in the art will clearly understand that the specific working processes of the systems, devices, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and for the sake of brevity, they will not be further described here.

[0117] In addition, the functional units in various embodiments of the present invention may be physically independent of each other, or two or more functional units may be integrated together, or all functional units may be integrated into a single processing unit. The above-mentioned integrated functional units may be implemented in the form of hardware, software, or firmware.

[0118] Those skilled in the art will understand that if the integrated functional unit is implemented in the form of software and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention can essentially or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, which includes a number of instructions for enabling a computing device (such as a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention when running the instructions. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0119] Alternatively, all or part of the steps of implementing the aforementioned method embodiments may be accomplished by hardware associated with program instructions (such as a computing device such as a personal computer, a server, or a network device), and the program instructions may be stored in a computer-readable storage medium. When the program instructions are executed by a processor of a computing device, the computing device executes all or part of the steps of the method described in each embodiment of the present invention.

[0120] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that within the spirit and principles of the present invention, they can still modify the technical solutions described in the above embodiments, or make equivalent replacements for some or all of the technical features therein. However, these modifications or replacements do not deviate from the scope of protection of the present invention.

Claims

1. A method for evaluating the performance of a brain-computer interface system, characterized in that: The method comprises: Determine typical mission profiles; and establish a pre-set evaluation indicator system that takes into account the system environment performance and the capabilities of the test subjects; Performing a test on the EEG interface system based on the typical task profile to obtain objective test data related to the system environmental performance and subjective test data related to the test subject's ability; Determining the evaluation value of each preset evaluation indicator in the preset evaluation indicator system based on the objective test data and the subjective test data; and determining the weight of each preset evaluation indicator using the hierarchical analysis method; The comprehensive performance evaluation value of the brain-computer interface system is calculated based on the evaluation value of each of the preset evaluation indicators and the corresponding weight.

2. The method according to claim 1, characterized in that Determining a typical mission profile includes: Obtaining task-related information; the task-related information includes typical application scenarios and task assumptions and / or task objectives corresponding to the typical operation scenarios; Performing multi-dimensional task analysis on the task-related information using a spatiotemporal constraint model to establish a typical task profile including at least one task phase; The mission phases include a preparation phase, an activation phase, a preparation phase, a maintenance phase and / or a termination phase.

3. The method according to claim 1, characterized in that The preset evaluation indicators include objective performance indicators and subjective performance indicators; The objective performance indicators include at least one of the following: operating environment, system weight, EEG signal interference factors, EEG signal acquisition influencing factors, EEG signal transmission influencing factors, EEG signal decoding influencing factors, total success rate, total task adaptation rate, target recognition performance, total task completion time, and delay time; The subjective performance index includes at least one of subjective feeling factors, personnel fatigue, personnel situational awareness, personnel emotional arousal, and personnel subjective load.

4. The method according to claim 3, characterized in that The obtaining of objective test data related to the system environment performance and subjective test data related to the test subject's ability includes: Using a preset device state detection model to collect operating data of the brain-computer interface system during the test as objective test data; the device state detection model includes a customized evaluation scale and / or a counter; the objective test data includes performance values ​​corresponding to each of the objective performance indicators; The subjective test data of the test subjects during the test process are obtained using a preset subjective questionnaire scale; the subjective test data include the performance values ​​corresponding to each of the subjective performance indicators.

5. The method according to claim 3, characterized in that Before performing the test on the EEG interface system based on the typical task profile, the method further includes: Establish a pre-experimental training mechanism for the test subjects, specifically including: By simulating tasks for the brain-computer interface system, the tested person is given standardized operation training until the operator reaches a preset operation proficiency threshold to ensure the comparability and consistency of the test results.

6. The method according to claim 4, characterized in that The step of determining the evaluation value of each preset evaluation indicator in the preset evaluation indicator system based on the objective test data and the subjective test data includes: The objective test data and the subjective test data of different dimensions and / or types are normalized using hierarchical normalization rules that match each of the preset evaluation indicators, and the performance values ​​corresponding to each of the preset evaluation indicators are unified into the [0, 1] interval as the evaluation values ​​of each of the preset evaluation indicators.

7. The method according to claim 1, characterized in that The method of determining the weight of each of the preset evaluation indicators by using the hierarchical analysis method includes: Obtain statistical results of the weighted scores of the experts for each of the preset evaluation indicators, and construct a judgment matrix using the statistical results of the weighted scores; Calculating the weights of multiple elements in the judgment matrix by using the eigenvalue method and the eigenvector method; A consistency check is performed on the judgment matrix after weight calculation, and the weights of the elements that pass the consistency check are used as the weights of each of the preset evaluation indicators.

8. The method according to claim 7, characterized in that The consistency check of the judgment matrix after weight calculation includes: The consistency index is calculated based on the eigenvalue and order of the judgment matrix. The mathematical expression is: Where CI represents the consistency index; λ max represents the eigenvalue; n represents the order; And, searching for a corresponding random consistency ratio value in a consistency ratio random indicator table according to the order of the judgment matrix; The consistency ratio is calculated according to the consistency index and the random consistency ratio value, and the mathematical expression is: Wherein, CR represents the consistency ratio; RCR represents the random consistency ratio value; Determine whether the consistency ratio is less than a preset consistency threshold; if the consistency ratio is less than the preset consistency threshold, consider that the judgment matrix passes the consistency check; if the consistency ratio is greater than or equal to the preset consistency threshold, readjust the judgment matrix.

9. The method according to any one of claims 1 to 8, characterized in that The calculating of the comprehensive effectiveness evaluation value of the brain-computer interface system based on the evaluation value of each of the preset evaluation indicators and the corresponding weights includes: A comprehensive performance evaluation function for the brain-computer interface system is established, and the mathematical expression is: Wherein, D represents the comprehensive performance evaluation value of the brain-computer interface system; CR i Represents the evaluation value of the i-th preset evaluation indicator; α i represents the weight of the i-th preset evaluation indicator; N represents the total number of preset evaluation indicators; The evaluation value and corresponding weight of each of the preset evaluation indicators are input into the comprehensive performance evaluation function to obtain the comprehensive performance evaluation value of the brain-computer interface system.

10. A device for evaluating the performance of a brain-computer interface system, characterized in that: The device comprises: System mission determination module, used to determine typical mission profiles; Evaluation indicator construction module, used to establish a preset evaluation indicator system that takes into account the system environment performance and the test subject's capabilities; A test data acquisition module, configured to perform a test on the EEG interface system based on the typical task profile, and obtain objective test data related to the system environmental performance and subjective test data related to the test subject's ability; an intermediate data calculation module, configured to determine, based on the objective test data and the subjective test data, an evaluation value of each preset evaluation indicator in the preset evaluation indicator system; and to determine a weight of each preset evaluation indicator using a hierarchical analysis method; The comprehensive performance evaluation module is used to calculate the comprehensive performance evaluation value of the brain-computer interface system based on the evaluation value of each preset evaluation indicator and the corresponding weight.

Images