Usability testing and evaluation methods for brain-computer interface systems, edge computing devices and media

Abstract

This disclosure provides a usability testing and evaluation method, edge computing device, and medium for a brain-computer interface (BCI) system. The method includes: acquiring the subject's EEG data based on a first SSVEP paradigm set in the BCI system; inputting the parameters of the first SSVEP paradigm and the EEG data into an evaluation network to generate usability labels corresponding to the first SSVEP paradigm; wherein the evaluation network is trained from EEG data samples, parameter samples, and label samples, the EEG data samples are the EEG data corresponding to a second SSVEP paradigm of the BCI system, the parameter samples are the parameters of the second SSVEP paradigm, and the label samples are the usability labels of the second SSVEP paradigm, determined by the EEG data and the subject's adaptability score for the second SSVEP paradigm. This technical solution improves the accuracy of usability evaluation for BCI systems.

Description

Technical Field

[0001] This disclosure relates to the field of brain-computer interface (BCI) technology, and in particular to a usability testing and evaluation method for a brain-computer interface system, an edge computing device, and a computer-readable storage medium. Background Technology

[0002] When a BCI system based on Steady-State Visual Evoked Potentials (SSVEP) is applied to subjects, the subjects continuously gaze at a stimulation unit displayed on a visual stimulator with a fixed frequency. This induces an electrophysiological signal in the frontal lobe region of the brain that is related to the flashing frequency (i.e., the stimulation frequency) of the stimulation unit. Because this electrophysiological signal has a higher signal-to-noise ratio (SNR), it can generate a higher information transmission rate (ITR). Therefore, the SSVEP paradigm may become a more advantageous BCI paradigm in the future.

[0003] However, current assessments of the usability of the SSVEP paradigm are based solely on electroencephalogram (EEG) data from SSVEP signals, which limits the accuracy of the assessment results. Summary of the Invention

[0004] One of the technical problems to be solved by this disclosure is to propose a usability testing and evaluation method, edge computing device and medium for a brain-computer interface system, so as to evaluate the usability of the SSVEP paradigm based on EEG data and in conjunction with user adaptability, thereby further improving the accuracy of usability evaluation results.

[0005] To address the aforementioned technical problems, this disclosure provides a usability testing and evaluation method for a brain-computer interface system, comprising: acquiring the subject's EEG data based on a first SSVEP paradigm set in the brain-computer interface system; inputting the parameters of the first SSVEP paradigm and the EEG data into an evaluation network to generate a usability label corresponding to the first SSVEP paradigm; The evaluation network is trained using EEG data samples, parameter samples, and label samples. The EEG data samples are the EEG data corresponding to the second SSVEP paradigm in the brain-computer interface system, the parameter samples are the parameters of the second SSVEP paradigm, and the label samples are the usability labels of the second SSVEP paradigm, which are determined by the EEG data and the subjects' fitness scores for the second SSVEP paradigm.

[0006] In some embodiments, the process of determining the labeled samples includes: obtaining the subject's adaptability score to the second SSVEP paradigm; calculating the recognition accuracy of the EEG data samples; calculating a comprehensive evaluation index value by weighted summation based on the average signal-to-noise ratio, recognition accuracy, and adaptability score of the EEG data samples; and determining the labeled samples corresponding to the EEG data samples based on the comprehensive evaluation index value; wherein, the formula for calculating the comprehensive evaluation index value is: s = k1 × s1 + k2 × s2 + k3 × s3 In the formula, k1 is the first weight, k2 is the second weight, k3 is the third weight, s1 is the first evaluation index value, which is determined by the average signal-to-noise ratio of the EEG data samples based on the first preset piecewise function, s2 is the second evaluation index value, which is determined by the adaptability score, and s3 is the third evaluation index value, which is determined by the recognition accuracy based on the second preset piecewise function.

[0007] In some embodiments, the first evaluation index value is determined by the average signal-to-noise ratio and may include: When the average signal-to-noise ratio is less than a1, the value of the first evaluation index is b1. When the average signal-to-noise ratio is greater than or equal to a1 and less than a2, the value of the first evaluation index is b2. When the average signal-to-noise ratio is greater than or equal to a2 and less than a3, the value of the first evaluation index is b3. When the average signal-to-noise ratio is greater than or equal to a3 and less than a4, the value of the first evaluation index is b4. When the average signal-to-noise ratio is greater than or equal to a4 and less than a5, the value of the first evaluation index is b5. When the average signal-to-noise ratio is greater than or equal to a5, the value of the first evaluation index is b6. Among them, a1 <a2<a3<a4<a5,b1<b2<b3<b4<b5<b6。

[0008] In some embodiments, obtaining the subject's adaptation score to the second SSVEP paradigm includes at least: obtaining a visual comfort score and a visual fatigue score. The second evaluation index value s2 is determined by the adaptation score, including: determining the second evaluation index value using the following calculation formula: s2 = a × (Sc - Sf) + b, where a and b are integers, Sc is the visual comfort score, and Sf is the visual fatigue score.

[0009] In some embodiments, when the value of the third evaluation index is less than 80, k3 increases and k1 and / or k2 decreases.

[0010] In some embodiments, the evaluation network includes at least a feature extraction network and a fully connected layer. The feature extraction network includes at least a convolutional neural network layer, a pooling layer, a dual-stream long short-term memory network layer, and a feature fusion module. The parameters of the first SSVEP paradigm and EEG data are input into the evaluation network to generate usability labels corresponding to the first SSVEP paradigm. This includes: obtaining the EEG data and the number of stimulation units corresponding to the first SSVEP paradigm; extracting spatiotemporal features from the EEG data corresponding to the first SSVEP paradigm using a convolutional neural network layer based on the number of stimulation units; extracting salient features from the spatiotemporal features of the EEG data corresponding to the first SSVEP paradigm using a pooling layer based on the single test duration of the subject under the first SSVEP paradigm; and determining the first SSVEP paradigm. Using the EEG data corresponding to the VEP paradigm, along with the wavelet transform coefficients and frequency intervals of the stimulus units, temporal features are extracted from the EEG data corresponding to the first SSVEP paradigm using a temporal LSTM in a dual-stream long short-term memory network layer. Frequency features are then extracted from the EEG data corresponding to the first SSVEP paradigm using a frequency-stream LSTM in the same dual-stream long short-term memory network layer, based on the frequency intervals and wavelet transform coefficients of the stimulus units. Based on the spatiotemporal features, saliency features, temporal features, and frequency features of the frequency-stream LSTM, feature fusion is performed using a frequency-stream LSTM feature fusion module to generate dynamic evaluation features. Finally, based on these dynamic evaluation features, usability labels are output using a fully connected layer.

[0011] In some embodiments, spatiotemporal features are extracted from EEG data corresponding to the first SSVEP paradigm using a convolutional neural network layer based on the number of stimulating units, including: determining the kernel size of the convolutional neural network layer based on the number of stimulating units; and extracting spatiotemporal features from EEG data corresponding to the first SSVEP paradigm using the convolutional neural network layer with the kernel size determined.

[0012] In some embodiments, adjusting the size of the convolution kernel according to the number of stimulation units may include: when the number of stimulation units is less than or equal to 8, adjusting the size of the convolution kernel to 3; when the number of stimulation units is greater than 8 and less than or equal to 20, adjusting the size of the convolution kernel to k = 3 + 0.15 × (n - 8), where k is the size of the convolution kernel and n is the number of stimulation units; and when the number of stimulation units is greater than 20, adjusting the size of the convolution kernel to k = 5 + 0.1 × (n - 20).

[0013] In some embodiments, the pooling layer may include a compression engine and adaptive time-frequency pooling. The pooling layer receives spatiotemporal features from the convolutional neural network layer, receives a single test duration from the BCI system, and extracts salient features from the EEG data corresponding to the first SSVEP paradigm from the spatiotemporal features based on the single test duration. This may include: the compression engine receiving the single test duration from the BCI system and determining the output dimension of the pooling layer based on the single test duration; and the adaptive time-frequency pooling extracting salient features from the spatiotemporal features extracted from the convolutional neural network layer and outputting the salient features with the output dimension determined by the compression engine.

[0014] In some embodiments, the dual-stream long short-term memory network layer further includes a step size allocator, and the method further includes: determining a first step size using the step size allocator based on the duration of a single test under the first SSVEP paradigm; and determining a second step size using the step size allocator based on the frequency interval of the stimulus units; wherein the first step size is used for temporal-stream LSTM to extract temporal features, and the second step size is used for frequency-stream LSTM to extract frequency features.

[0015] In some embodiments, the fully connected layer described above can be used to determine and output availability labels of the first SSVEP paradigm based on the features extracted by the feature extraction network.

[0016] This disclosure also provides a usability testing and evaluation device for a brain-computer interface system. The device includes: a data acquisition module configured to acquire the subject's EEG data based on a first SSVEP paradigm set in the brain-computer interface system; and an evaluation module configured to input the parameters of the first SSVEP paradigm and the EEG data into an evaluation network to generate a usability label corresponding to the first SSVEP paradigm. The evaluation network is trained using EEG data samples, parameter samples, and label samples. The EEG data samples are the EEG data corresponding to the second SSVEP paradigm in the brain-computer interface system, the parameter samples are the parameters of the second SSVEP paradigm, and the label samples are the usability labels of the second SSVEP paradigm, which are determined by the EEG data and the subjects' fitness scores for the second SSVEP paradigm.

[0017] This disclosure also provides an edge computing device, including a processor and a memory, wherein the memory stores programs or instructions that can run on the processor, and when the programs or instructions are executed by the processor, they implement any of the brain-computer interface system usability testing and evaluation methods described above.

[0018] This disclosure also provides a computer-readable storage medium storing a computer program, which, when run on a computer, executes any of the aforementioned brain-computer interface system usability testing and evaluation methods.

[0019] Through the above technical solutions, the usability testing and evaluation method, equipment and medium of the brain-computer interface system provided in this disclosure evaluate the usability of the first SSVEP paradigm based on the brain-computer interface system by using an evaluation network trained from multimodal data such as EEG data and user adaptability scores. This enables the evaluation method to evaluate the SSVEP paradigm by coordinating signal reliability and user adaptability, thereby generating high-precision usability labels and improving the evaluation accuracy. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a schematic diagram of a usability testing and evaluation method for a brain-computer interface system disclosed in this embodiment. Figure 2 This is a schematic diagram of the structure of an evaluation network disclosed in an embodiment of this disclosure; Figure 3 This is a schematic diagram illustrating an application scenario of the usability testing and evaluation method for a brain-computer interface system disclosed in this embodiment. Figure 4 This is a schematic diagram of another evaluation network structure disclosed in an embodiment of this disclosure; Figure 5 This is a schematic diagram of the structure of an edge computing device disclosed in an embodiment of this disclosure. Detailed Implementation

[0022] The embodiments of this disclosure will be further described in detail below with reference to the accompanying drawings and examples. The detailed description of the embodiments and the accompanying drawings are used to illustrate the principles of this disclosure by way of example, but should not be used to limit the scope of this disclosure. This disclosure can be implemented in many different forms and is not limited to the specific embodiments disclosed herein, but includes all technical solutions falling within the scope of the claims.

[0023] These embodiments are provided to make the disclosure thorough and complete, and to fully express the scope of the disclosure to those skilled in the art. It should be noted that, unless otherwise specifically stated, the relative arrangement of components and steps, material composition, numerical expressions, and values ​​set forth in these embodiments should be interpreted as exemplary only and not as limiting.

[0024] All terms used in this disclosure have the same meaning as understood by one of ordinary skill in the art to which this disclosure pertains, unless otherwise specifically defined. It should also be understood that terms defined in general dictionaries should be interpreted as having meanings consistent with their meanings in the context of the relevant art, and not as having idealized or highly formalized meanings, unless expressly defined herein.

[0025] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, they should be considered part of the specification.

[0026] In the SSVEP paradigm, the user can first set parameters through the visual stimulator's control system, such as the frequency range of the SSVEP paradigm, the number of stimulation units, the frequency of each stimulation unit, the phase of each stimulation unit, the morphology of the stimulation units (e.g., size, shape), the duration of a single test, the total number of tests, and whether to use the BCI's real-time control system to pause or stop the test. Then, the visual stimulator displays the stimulation units used to visually stimulate the subject based on these parameters. While the subject, wearing the acquisition device, gazes at the stimulation units, the acquisition device extracts the subject's EEG data from the received SSVEP signals.

[0027] However, existing assessment systems rely solely on subjects' EEG data to determine the applicability of the SSVEP paradigm, neglecting the impact of user adaptability on BCI system usability. BCI system usability encompasses not only signal quality at the application level but also user adaptability. The fact that existing assessment systems evaluate BCI system usability based solely on EEG data leads to low accuracy in the assessment results.

[0028] To further improve the accuracy of evaluation results, this application provides a usability testing and evaluation method for a brain-computer interface system. This method evaluates the applicability of the SSVEP paradigm based on the brain-computer interface system by using an evaluation network trained with EEG data samples and labeled samples determined by user adaptability scores. Based on EEG data analysis, it considers both signal reliability and user adaptability, thereby improving the accuracy of the usability evaluation results.

[0029] Figure 1 A schematic diagram of a usability testing and evaluation method for a brain-computer interface system provided in this disclosure embodiment includes: Step 101: Obtain the subject's EEG data based on the first SSVEP paradigm set in the brain-computer interface system.

[0030] Step 102: Input the parameters of the first SSVEP paradigm and the EEG data into the evaluation network to generate usability labels corresponding to the first SSVEP paradigm; wherein, the evaluation network is trained from EEG data samples, parameter samples and label samples, the EEG data samples are the EEG data corresponding to the second SSVEP paradigm of the brain-computer interface system, the parameter samples are the parameters of the second SSVEP paradigm, and the label samples are the usability labels of the second SSVEP paradigm, which are determined by the EEG data and the subject's adaptability rating of the first SSVEP paradigm.

[0031] Specifically, the usability label of the training evaluation network is determined by EEG data and the subject's adaptation score for the first SSVEP paradigm. This enables the evaluation network to have the ability to coordinate the evaluation of EEG data and user adaptability (which can be reflected by the subject's adaptation score for the SSVEP paradigm). Therefore, by processing the parameters of the SSVEP paradigm and the corresponding EEG data through the evaluation network, the usability label of the brain-computer interface system obtained not only includes the signal quality evaluation at the application level of the BCI system, but also the user adaptability evaluation at the application level of the BCI system, thereby improving the accuracy of the usability evaluation results of the brain-computer interface system.

[0032] The parameters for both the first and second SSVEP paradigms can be found in the foregoing descriptions. The second SSVEP paradigm may be the same as or different from the first SSVEP paradigm. The second and first SSVEP paradigms may differ depending on the parameters set and / or the target users of the BCI system. The user who scores the second SSVEP paradigm is a target user of the BCI system. This user may or may not be the same person as the user who sets the paradigm parameters. If they are different, the scoring user may be referred to as the subject, to distinguish them from the user who sets the parameters.

[0033] Figure 2 This is a schematic diagram of the structure of an evaluation network provided in an embodiment of the present disclosure. The evaluation network includes a feature extraction network 21 and a fully connected layer 22.

[0034] The feature extraction network 21 and the fully connected layer 22 are trained from EEG data samples and label samples. The EEG data samples are from the SSVEP signals corresponding to the second SSVEP paradigm of the BCI system, and the label samples are obtained from the EEG data samples and the subjects' fitness scores for the second SSVEP paradigm.

[0035] The feature extraction network 21 can be used to receive the EEG data corresponding to the first SSVEP paradigm of the BCI system and the parameters of the first SSVEP paradigm, and extract features from the EEG data corresponding to the first SSVEP paradigm according to the parameters of the first SSVEP paradigm.

[0036] The fully connected layer 22 can be used to receive features, evaluate the availability of the first SSVEP paradigm based on the features, and output the availability label of the first SSVEP paradigm.

[0037] Specifically, the evaluation network is trained using EEG data samples and labeled samples based on user adaptability scores, enabling the evaluation network to collaboratively assess signal reliability and user adaptability, thereby improving the evaluation accuracy of usability testing and evaluation methods for brain-computer interface systems.

[0038] In some embodiments of this disclosure, in order to coordinate the usability testing and evaluation method of the brain-computer interface system with the user's subjective rating, this disclosure also provides a usability testing and evaluation method for the brain-computer interface system. The method differs from the above embodiments in that the above-mentioned adaptability rating includes one or more of visual comfort rating and visual fatigue rating.

[0039] Specifically, since the adaptability score includes one or more of the visual comfort score and the visual fatigue score, the label sample is based on both EEG data and the user's visual adaptability score. This enables the evaluation network to coordinate EEG data and user visual adaptability, thereby improving the accuracy of the evaluation network's assessment.

[0040] In some embodiments of this disclosure, the subject's adaptation rating to the second SSVEP paradigm may further include any one or more of the following ratings: preference rating and cognitive fatigue rating, etc. Exemplarily, the above ratings can be obtained through questionnaires or interviews. For example, visual comfort can be divided into five categories: "very uncomfortable," "uncomfortable," "neutral," "comfortable," and "very comfortable," corresponding to scores 1, 2, 3, 4, and 5 respectively; visual fatigue can be divided into five categories: "very tired," "tired," "neutral," "comfortable," and "not tired," corresponding to scores 1, 2, 3, 4, and 5 respectively, and so on. The user (i.e., the subject) of the BCI system is asked to select the corresponding category from the above categories based on their own feelings, and then a corresponding rating is obtained based on the user's selection.

[0041] In some embodiments of this disclosure, to enable the feature extraction network and fully connected layer to more accurately evaluate the usability of the BCI system, the aforementioned label samples are obtained from EEG data samples, which may include: the label samples are obtained from a first evaluation metric value, and the first evaluation metric value is obtained from the EEG data samples. For example, the first evaluation metric value may be determined based on the average signal-to-noise ratio (ASNR), the value of which is positively correlated with the ASNR, which is obtained from the EEG data samples.

[0042] Specifically, the first evaluation index value is positively correlated with the average signal-to-noise ratio obtained from the EEG data samples, and the sample labels are based on the first evaluation index value, which makes the reliability of the signals referenced by the evaluation network higher, thereby helping to improve the accuracy of the evaluation results of the evaluation network.

[0043] In some embodiments of this disclosure, to provide more implementation methods, the first evaluation index value is determined based on the average signal-to-noise ratio (ASNR), and may include: a preset minimum value Amin and a preset maximum value Amax of ASNR. When the ASNR obtained from the EEG data is less than the preset minimum value Amin, the first evaluation index value is determined to be the minimum value Bmin (e.g., 0). When the ASNR obtained from the EEG data is greater than or equal to the preset maximum value Amax, the first evaluation index value is determined to be the maximum value Bmax (e.g., 5, 10, or 100, etc.).

[0044] Furthermore, the ASNR can be partitioned between its minimum value Amin and maximum value Amax. For example, starting from the minimum value Amin, the ASNR can be divided into three segments: a first segment, a second segment, and a third segment. The maximum value in the third segment is less than the preset maximum value Amax of the ASNR. Therefore, when the ASNR obtained from the EEG data falls within the first segment, the first evaluation index value is determined to be B1; when the ASNR obtained from the EEG data falls within the second segment, the first evaluation index value is determined to be B2; and when the ASNR obtained from the EEG data falls within the third segment, the first evaluation index value is determined to be B3. Where Bmin... <B1<B2<B3<Bmax。

[0045] For example, the first evaluation index value, determined based on the average signal-to-noise ratio, may include: When ASNR is less than a1, the value of the first evaluation index is b1; When the ASNR ratio is greater than or equal to a1 and a2, the value of the first evaluation index is b2. When ASNR is greater than or equal to a2 and less than a3, the value of the first evaluation index is b3; When ASNR is greater than or equal to a3 and less than a4, the value of the first evaluation index is b4. When the ASNR is greater than or equal to a4 and less than a5, the value of the first evaluation index is b5; When the ASNR is greater than or equal to a5, the value of the first evaluation index is b6; wherein a1<a2<a3<a4<a5, b1<b2<b3<b4<b5<b6. For example, a1=6dB, a2=7dB, a3=8dB, a4=9dB, a5=10dB, or a1=5.8dB, a2=6.8dB, a3=7.8dB, a4=8.8dB, a5=9.8dB; etc. b1=0, b2=20, b3=40, b4=60, b5=80, b6=100; or b1=0, b2=2, b3=4, b4=6, b5=8, b6=10; etc.

[0046] In some embodiments of the present disclosure, in order to enable the feature extraction network and the fully connected layer to more effectively have the capability of collaborative user adaptability, the above label samples are obtained according to the subjects' scores for the second SSVEP paradigm, which may include: the label samples are obtained according to second evaluation index values, and the second evaluation index values are obtained according to adaptability scores. For example, the second evaluation index value s2=a×(Sc-Sf)+b, wherein a and b are integers, Sc is the aforementioned visual comfort score, and Sf is the aforementioned visual fatigue score.

[0047] It should be noted that the visual comfort score Sc and the visual fatigue score Sf can be scored and determined by the subjects themselves based on their intuitive feelings during use, can also be scored based on the subjects' performance during use (such as whether there is a behavior of manually adjusting the wearing device, whether there is a behavior of rubbing eyes, whether there is a behavior of adjusting sitting posture, etc.), and can also be scored in combination with the subjects' physiological signals (such as the number of blinks) during use.

[0048] Specifically, the second evaluation index value is obtained from the visual comfort score and the visual fatigue score, and the sample label is based on the second evaluation index value, so that the evaluation network trained through the sample label comprehensively refers to user adaptability, which is beneficial to improving the accuracy of the evaluation result of the evaluation network.

[0049] In some embodiments of the present disclosure, in order to obtain sample labels more effectively, the process of determining the above label samples may include: obtaining the adaptability scores of the subjects for the second SSVEP paradigm, for the acquisition of adaptability scores, reference may be made to the relevant description in the foregoing embodiments; calculating the recognition accuracy of EEG data samples; Based on the average signal-to-noise ratio, recognition accuracy, and adaptability score of the EEG data samples, a comprehensive evaluation index value is calculated by weighted summation; the formula for calculating the comprehensive evaluation index value is as follows: s = k1 × s1 + k2 × s2 + k3 × s3 In the formula, k1 is the first weight, k2 is the second weight, k3 is the third weight, s1 is the first evaluation index value, determined by the average signal-to-noise ratio of the EEG data samples based on a first preset piecewise function, s2 is the second evaluation index value, determined by the adaptability score, and s3 is the third evaluation index value, obtained from the recognition accuracy, which is determined based on the second preset piecewise function. The first and second evaluation index values ​​can be found in the relevant descriptions of the foregoing embodiments.

[0050] In some embodiments of this disclosure, k3 may be greater than k1 and k2, for example, k3=0.5, k1=0.3, k2=0.2.

[0051] In some embodiments of this disclosure, when the value of the third evaluation index is less than 80, k3 increases, and k1 and / or k2 decreases. Both s1 and s3 are indicators obtained from objective data. When s3 < 80, it is considered to have a certain impact on the subject's actual experience, and its influence on the assessment of signal reliability increases; therefore, its weight in the objective indicators is increased. For example, when s3 < 80, k3 is increased from 0.5 to 0.6, and k1 is decreased to 0.2.

[0052] For example, the labels in the label sample may include high availability, low availability, and general availability. High availability can be interpreted as a recommendation to use the BCI system directly; general availability can be interpreted as a need for targeted optimization of the BCI system configuration; and low availability can be interpreted as a recommendation to replace the BCI system configuration. The label sample is determined by a comprehensive evaluation index value. For example, when the comprehensive evaluation index value s ≥ 90, the label is high availability; when 70 ≤ s < 90, the label is general availability; and when the comprehensive evaluation index value s < 70, the label is low availability.

[0053] In some embodiments of this disclosure, the EEG data samples are influenced by the subject's own characteristics and parameters of the second SSVEP paradigm. The parameters of the second SSVEP paradigm may include one or more of the following: frequency range, stimulus unit parameters, single test duration, and total number of tests. Typically, a stimulus unit is displayed multiple times within an SSVEP paradigm; for example, it may be displayed for a period followed by a pause. Each display period is the single test duration, and the number of displays is the total number of tests. All user-set parameters determine the EEG data in the SSVEP signal, thereby affecting the ASNR and consequently the first evaluation index value s1. User-set parameters also directly affect the aforementioned visual comfort score Sc and the aforementioned visual fatigue score Sf, thereby affecting the second evaluation index value s2. Sc and Sf, in turn, affect the EEG data. The EEG data is also used to obtain the classification accuracy (Acc), thereby affecting the third evaluation index value s3. In this way, the adaptability of EEG data and users affects the first evaluation index value s1, the second evaluation index value s2, and the third evaluation index value s3, thereby affecting s, which in turn affects the label samples, and ultimately affects the evaluation accuracy when using EEG data samples and label samples for training.

[0054] For example, the parameters of the stimulation units may include any one or more of the following: number of stimulation units, frequency, phase, frequency interval, and morphology. The morphology of the stimulation units includes their size, shape, etc. For instance, larger stimulation units set by the user, and more dazzling the stimulation units, can easily lead to subject fatigue, thus affecting EEG data, Sc, and Sf.

[0055] In some embodiments of this disclosure, the calculation process of the comprehensive evaluation index value of the SSVEP paradigm, based on the above embodiments, may further include whether to use the function of pausing or stopping the real-time control system. Whether this function is used or not affects Sc and Sf, thereby affecting the second evaluation index value s2, and further affecting s. Ultimately, when training the evaluation network with EEG data samples and label samples, it affects the model parameters of the evaluation network and thus affects the accuracy of the evaluation network's evaluation.

[0056] For example, during the experiment, the subjects may experience visual fatigue due to wearing the experimental device for a long time. In this case, the real-time control system can be used to pause or stop the experiment, which will reduce Sc and Sf, thus affecting the value of the second evaluation index s2.

[0057] Determine whether a pause or abort command has been triggered. If so, reduce the value of the second evaluation index s2 to a preset percentage. The preset percentage is determined by the number of times the pause or abort command has been triggered. The more times the command is triggered, the smaller the preset percentage becomes. For example, if the pause or abort command is triggered twice, the value of the second evaluation index s2 will be reduced to 70%. If not, it will remain unchanged.

[0058] In some embodiments of this disclosure, the parameters of the first SSVEP paradigm may include: the number of stimulus units, the duration of a single test, and the frequency interval. The feature extraction network may include: a convolutional neural network (CNN) layer, a pooling layer, a dual-stream long short-term memory (LSTM) layer, and a feature fusion module.

[0059] The convolutional neural network layer can be used to receive EEG data corresponding to the first SSVEP paradigm and the number of stimulation units from the BCI system, and extract spatiotemporal features from the EEG data corresponding to the first SSVEP paradigm based on the number of stimulation units.

[0060] Pooling layers can be used to receive spatiotemporal features from convolutional neural network layers, receive single test durations from the BCI system, and extract salient features from the EEG data corresponding to the first SSVEP paradigm based on the single test durations.

[0061] The dual-stream long short-term memory network layer can be used to receive EEG data corresponding to the first SSVEP paradigm, wavelet transform coefficients of the EEG data corresponding to the first SSVEP paradigm, and frequency intervals of stimulation units from the BCI system, extract time features from the EEG data corresponding to the first SSVEP paradigm, and extract frequency features from the EEG data corresponding to the first SSVEP paradigm based on the frequency intervals of stimulation units and wavelet transform coefficients.

[0062] The feature fusion module can be used to fuse features extracted from convolutional neural network layers, pooling layers, and two-stream long short-term memory network layers. The feature fusion methods employed can include deconvolution, addition, multiplication, attention mechanisms, pyramid pooling, concatenation, etc., or attention-weighted concatenation. The fused features can contain multiple time-frequency domain features extracted from the convolutional neural network layers, pooling layers, and two-stream long short-term memory network layers.

[0063] In some embodiments of this disclosure, step 102 above, which involves inputting the parameters of the first SSVEP paradigm and EEG data into the evaluation network to generate a usability label corresponding to the first SSVEP paradigm, may include: The convolutional neural network layer receives the EEG data corresponding to the first SSVEP paradigm and the number of stimulation units from the BCI system, and extracts spatiotemporal features from the EEG data corresponding to the first SSVEP paradigm based on the number of stimulation units. The pooling layer receives spatiotemporal features from the convolutional neural network layer and the duration of a single test from the BCI system. Based on the duration of the single test, it extracts salient features from the EEG data corresponding to the first SSVEP paradigm from the spatiotemporal features. The dual-stream long short-term memory network layer receives EEG data corresponding to the first SSVEP paradigm, wavelet transform coefficients of the EEG data corresponding to the first SSVEP paradigm, and frequency intervals of the stimulation units from the BCI system. It also extracts time features from the EEG data corresponding to the second SSVEP paradigm and extracts frequency features from the EEG data corresponding to the second SSVEP paradigm based on the frequency intervals of the stimulation units and the wavelet transform coefficients. The feature fusion module integrates features extracted from convolutional neural network layers, pooling layers, and dual-stream long short-term memory network layers to generate dynamic evaluation features; The dynamic evaluation features are input into the fully connected layer, and the usability label is output.

[0064] In some embodiments of this disclosure, in order to accurately extract the spatiotemporal features of EEG data in different SSVEP paradigms of the convolutional neural network layer, the convolutional neural network layer may also be configured not to adjust the size of the convolutional kernel according to the number of stimulation units.

[0065] Specifically, when the number of stimulation units is small (e.g., less than or equal to 5), the spatial separation of the corresponding activation region in the visual cortex is high for each stimulation unit. Using a fixed small convolutional kernel, such as k=3, can accurately extract the independent response features of each stimulation unit and avoid signal confusion between neighboring units. When the number of stimulation units increases (e.g., less than or equal to 18), the increased stimulation unit density leads to partial overlap of the cortical activation regions. The convolutional kernel can be linearly increased to ensure that the independent response features of each stimulation unit can be extracted. When the spacing between stimulation units is too small (e.g., the number of stimulation units is greater than 18), the cortical responses are highly aliased. The convolutional kernel should be further increased to ensure that the independent response features of each stimulation unit can be extracted as much as possible. Thus, in different SSVEP paradigms with different numbers of stimulation units, the convolutional neural network layers of the evaluation network can accurately extract the spatiotemporal features of EEG data.

[0066] For example, a convolutional neural network layer can be one-dimensional. In some embodiments of this disclosure, the convolutional neural network layer can be used to: adjust the kernel size to 3 when the number of stimulating units is less than or equal to 8; and adjust the kernel size to k= when the number of stimulating units is greater than 8 and less than or equal to 20. Where k is the size of the convolution kernel and n is the number of stimulus units. For floor function operations; when the number of stimulus units is greater than 20, the kernel size is adjusted to k= , where the upper limit of k is 7.

[0067] In some embodiments of this disclosure, the convolutional neural network layer may include a kernel size controller and a convolutional neural network. The kernel size controller can be used to receive the number of stimulation units and determine the kernel size based on the number of stimulation units, as described above. The convolutional neural network can be used to receive EEG data corresponding to the first SSVEP paradigm and extract spatiotemporal features from the EEG data corresponding to the first SSVEP paradigm based on the kernel size determined by the kernel size controller.

[0068] In some embodiments of this disclosure, the convolutional neural network layer receives EEG data corresponding to the first SSVEP paradigm and the number of stimulation units from the BCI system, and extracts spatiotemporal features from the EEG data corresponding to the first SSVEP paradigm based on the number of stimulation units, which may include: The kernel size controller receives the number of stimulation units and determines the kernel size based on the number of stimulation units. For details on determining the kernel size based on the number of stimulation units, please refer to the relevant description in the foregoing embodiments. The convolutional neural network receives EEG data corresponding to the first SSVEP paradigm and extracts spatiotemporal features from the EEG data corresponding to the first SSVEP paradigm according to the convolutional kernel size determined by the convolutional kernel size controller.

[0069] In some embodiments of this disclosure, the pooling layer may include a compression engine and an adaptive time-frequency pooling module.

[0070] The compression engine can receive a single test duration t0 from the BCI system and determine the output dimension of the pooling layer based on the single test duration. For example, when 0.2 ≤ t0 < 1s, the early event-related potential (ERP) component of the primary visual processing is preserved, and average pooling is used to focus on background information; when t0 ≥ 1s, late noise is filtered out for medium- to long-term tasks, and max pooling is used to improve the compression level.

[0071] The adaptive time-frequency pooling module can be used to extract salient features from the spatiotemporal features extracted from the convolutional neural network layers and output the salient features in the output dimension determined by the compression engine.

[0072] In some embodiments of this disclosure, the pooling layer receives spatiotemporal features from the convolutional neural network layer, receives the duration of a single test from the BCI system, and extracts salient features from the EEG data corresponding to the first SSVEP paradigm based on the duration of the single test. This may include: The compression engine receives the duration of a single test from the BCI system and determines the output dimension of the pooling layer based on the duration of the single test. Adaptive time-frequency pooling extracts salient features from the spatiotemporal features extracted by the convolutional neural network layers and outputs the salient features in the output dimension determined by the compression engine.

[0073] In some embodiments of this disclosure, the aforementioned dual-stream long short-term memory network layer may include a step size assigner and a dual-stream LSTM. The step size assigner is used to determine a first step size and a frequency interval f0 for receiving stimulus units from the BCI system, and to determine a second step size based on the frequency interval f0. The dual-stream LSTM may include a time-stream LSTM and a frequency-stream LSTM. The time-stream LSTM is used to receive EEG data corresponding to a first SSVEP paradigm from the BCI system and extract time features from the EEG data according to the first step size. The frequency-stream LSTM is used to receive wavelet transform coefficients from the BCI system and extract frequency features based on the second step size and the wavelet transform coefficients. The dual-stream LSTM may, for example, be a bidirectional LSTM (BiLSTM).

[0074] In some embodiments of this disclosure, the dual-stream long short-term memory network layer receives EEG data corresponding to the first SSVEP paradigm, wavelet transform coefficients of the EEG data corresponding to the first SSVEP paradigm, and frequency intervals of the stimulation units from the BCI system, and extracts temporal features from the EEG data corresponding to the second SSVEP paradigm, and extracts frequency features from the EEG data corresponding to the second SSVEP paradigm based on the frequency intervals of the stimulation units and the wavelet transform coefficients, which may include: The step size distributor determines the first step size and the frequency interval of the stimulation units received from the BCI system, and determines the second step size based on the frequency interval of the stimulation units. The time-stream LSTM receives EEG data corresponding to the first SSVEP paradigm from the BCI system and extracts time features from the EEG data corresponding to the first SSVEP paradigm according to the first step length. The frequency-stream LSTM receives wavelet transform coefficients from the BCI system and extracts frequency features according to the second step length and the wavelet transform coefficients.

[0075] For example, the first step length can be fixed, such as 128 seconds, or it can be determined by the duration of a single test.

[0076] In some embodiments of this disclosure, the step size allocator may determine the first step size by receiving a single test duration from the BCI system and determining the first step size based on the single test duration.

[0077] The time-stream LSTM receives the raw time-stream signal carrying EEG data and can capture event-related potential dynamics using a first-step reference. The second step size is 100 / f0. The frequency-stream LSTM receives the frequency-stream signal carrying wavelet transform coefficients and can analyze the harmonic structure of a frequency-stream signal covering a 100Hz bandwidth. The wavelet transform coefficients can be calculated using wavelet transform. The wavelet transform calculation can use Haar wavelets as wavelet basis functions.

[0078] In some embodiments of this disclosure, the step size allocator can be used to receive the duration of a single test from the BCI system and determine the first step size based on the duration of the single test t0. For example, the first step size = max(16, min(128, t0 / 0.25)).

[0079] In some embodiments of this disclosure, the fully connected layer is used to determine and output a label for the first SSVEP paradigm based on the features extracted by the feature extraction network. This label represents an evaluation result of the usability of the first SSVEP paradigm. Exemplarily, the fully connected layer may consist of multiple neurons and corresponding activation functions, with full connectivity between layers. The multiple neurons can calculate and output evaluation index values ​​for different labels, such as evaluation index values ​​for low usability labels, general usability labels, and high usability labels. The activation function can calculate the probability corresponding to each evaluation index value and output the label corresponding to the highest probability as the usability evaluation result, i.e., the usability label of the first SSVEP paradigm generated by the evaluation network.

[0080] In some embodiments of this disclosure, the activation function may be a classification task (Softmax) layer, which is used to calculate the probability corresponding to each evaluation index value and output the label corresponding to the highest probability as the usability evaluation result, that is, the usability label of the first SSVEP paradigm generated by the evaluation network.

[0081] In some embodiments of this disclosure, the aforementioned EEG data samples and label samples may be stored in a storage medium for use when training and evaluating the network.

[0082] For example, the parameters of the first SSVEP paradigm may be provided by the BCI system, or a separate module (such as a parameter acquisition module) may be set up for the evaluation network to obtain the parameters of the first SSVEP paradigm from the BCI system.

[0083] One embodiment of this disclosure is, for example... Figure 3 As shown in the embodiment, in the application scenario of the usability testing and evaluation method for the brain-computer interface system, the evaluation network may include: a sample acquisition module 31, a sample library 32, and an evaluation network 33. The sample acquisition module 31 includes a data processing submodule, a dimension calculation submodule, and a label conversion submodule.

[0084] When the evaluation system is used for, for example Figure 3 When evaluating the BCI system using the SSVEP paradigm, the second SSVEP paradigm can be used to obtain samples first.

[0085] In the BCI system, the paradigm dynamic adjustment module can be used to dynamically configure SSVEP paradigm parameters and supports real-time interaction. Users can configure different SSVEP paradigms by setting parameters through the paradigm dynamic adjustment module. For example, users can set the frequency interval f0 in the range of 8–15.8Hz, adjust the step size S to 0.2–2Hz, and set the number of stimulus units n, single trial duration t0, total number of tests nt, real-time control function, stimulus unit phase, etc. The number of stimulus units n can be a minimum of 4 and a maximum of 40, and can be designed in conjunction with f0, for example, automatically reducing the number of stimulus units in the high-frequency band to reduce visual burden. The single trial duration t0 can range from 0.2–5 seconds, and the total number of tests nt can be a minimum of 10 and a maximum of 50. The stimulus unit phase can be uniformly distributed in the range of 0°–180°. The real-time control function allows users to actively pause or terminate the test during the trial interval (e.g., default 1–3 seconds), and the system automatically saves the breakpoint data. The settings for each parameter here are for illustrative purposes only and are not intended to be limiting; they can be adjusted according to actual needs.

[0086] After the subject wears the electrodes for collecting EEG data, they can set the parameters of the second SSVEP paradigm through the paradigm dynamic adjustment module and then look at the display screen of the visual stimulator. The visual stimulator displays the corresponding stimulation units according to the settings of the paradigm dynamic adjustment module, such as the number, frequency, phase, duration of each display, and number of displays, based on the user-set parameters. The EEG data acquisition module receives the SSVEP signals acquired by the electrodes when the subject looks at the stimulation units and extracts the raw EEG data corresponding to the second SSVEP paradigm.

[0087] In the sample acquisition module 31, the data processing submodule can perform Fast Fourier Transform (FFT) and Power Spectral Density (PSD) analysis on the raw EEG data of the second SSVEP paradigm to calculate the ASNR of the target frequency band (8–60Hz). The data processing submodule can also use the Filter Bank Canonical Correlation Analysis (FBCCA) classification algorithm (default 4th harmonic) to calculate the classification accuracy (Acc) on the raw EEG data of the second SSVEP paradigm. After the test, the data processing submodule can also display a subjective evaluation interface for the user to select from, thereby obtaining the subject's subjective score. The scoring can be found in the relevant descriptions in the aforementioned embodiments.

[0088] The dimensionality calculation submodule can obtain the first evaluation index value s1 based on the piecewise linear mapping of ASNR, as described in the relevant explanation in the aforementioned embodiments. The dimensionality calculation submodule can also obtain the second evaluation index value s2 based on the subject's subjective rating, for example, s2 = 10 × Sc + 50 - 10 × Sf. The dimensionality calculation submodule can also obtain the third evaluation index value s3 based on Acc. For example, when Acc ≥ 0.95, s3 = 100; when 0.90 ≤ Acc < 0.95, s3 = 90; and so on, decreasing s3 by 10 for every 0.05 decrease in Acc, such as 0.85 ≤ Acc < 0.90, s3 = 80; when Acc ≤ 1 / number of stimulus units n, s3 = 0.

[0089] The label conversion submodule can obtain the total value s used to determine the label based on the aforementioned dimensions, as described in the relevant descriptions in the previous embodiments.

[0090] In this embodiment, the original EEG data of the second SSVEP paradigm, user-defined parameters, and labels can be stored as samples in a sample library. Then, these samples are used to train the evaluation network 33. The evaluation network 33 evaluates subsequent new SSVEP paradigms (such as the first SSVEP paradigm).

[0091] Evaluation network 33 pairs Figure 3When evaluating the first SSVEP paradigm of the BCI system shown, the user first configures the parameters of the first SSVEP paradigm through the paradigm dynamic adjustment module. Then, the subject fixates on a visual stimulator, which, according to the configured stimulation unit, provides visual stimulation to the subject wearing electrodes. The electrodes worn by the subject transmit SSVEP signals to the EEG data acquisition module, which then sends the raw EEG data to the evaluation network 33 and the data analysis module. The data analysis module performs wavelet transform on the raw EEG data and sends the wavelet transform result to the evaluation network 33. Furthermore, the paradigm dynamic adjustment module also transmits the user-set parameters to the evaluation network 33. The evaluation network 33 evaluates the usability of the first SSVEP paradigm based on the raw EEG data, wavelet transform coefficients, and parameters.

[0092] In this embodiment, the internal structure of the sample acquisition module 31 can be divided in other ways, which are not limited here.

[0093] In this embodiment, the sample library can be omitted.

[0094] In this embodiment, the evaluation network 33 can be as follows: Figure 4 As shown, the system includes: a feature extraction network 41, a fully connected layer 42, and a softmax layer 43. The feature extraction network 41 may include a CNN layer 411, a pooling layer 412, a dual-stream LSTM layer 413, and a feature fusion module 414. The CNN layer 411 includes a kernel size controller and a multi-scale residual CNN. The kernel size controller adjusts the kernel size according to the number of stimulus units, as described in the preceding embodiments. The multi-scale residual CNN may be a multi-layer convolution, such as a 3-layer convolution, where the kernel size is determined by the kernel size controller. For example, when the kernel size controller adjusts the kernel size to 4, the kernel size of each layer in the multi-layer convolution is 4. The multi-scale residual CNN is used to extract the spatiotemporal features of the original EEG data, as described in the preceding embodiments.

[0095] Pooling layer 412 includes a compression engine and an adaptive time-frequency pooling module. The compression engine controls the output dimension of the pooling layer according to the single test duration t0 in the parameters. The adaptive time-frequency pooling module extracts salient features from the spatiotemporal features extracted by CNN layer 411 and outputs them in the dimension controlled by the compression engine, as described in the relevant descriptions in the foregoing embodiments.

[0096] The dual-stream LSTM layer 413 includes a step size assigner and a dual-stream LSTM. The step size assigner determines the second step size of the frequency-stream LSTM based on the frequency interval f0, or, further, the step size assigner determines the first step size of the time-stream LSTM based on the single test duration t0. The dual-stream LSTM extracts the temporal and frequency features of the EEG data based on the first and second step sizes, as described in the relevant descriptions in the foregoing embodiments.

[0097] The feature extraction network 41 fuses the features extracted from each layer and outputs them to the fully connected layer 42 for classification. The fully connected layer 42 sends the classification results to the softmax layer 43. The classification results can be evaluation metrics corresponding to different labels, such as the evaluation metrics for high availability, general availability, and low availability labels in the fully connected layer 42. The softmax layer 43 calculates the probability corresponding to the evaluation metric value of each label and outputs the label with the highest probability as the availability assessment result.

[0098] For example, the Softmax layer 43 can be used as the activation function of the fully connected layer 42 and incorporated into the fully connected layer 42.

[0099] In this embodiment, the Softmax layer 43 can be omitted. When the Softmax layer 43 is omitted, the fully connected layer 42 can output the label with the largest evaluation index value as the usability evaluation result.

[0100] In the embodiments of the usability testing and evaluation method for the brain-computer interface system disclosed herein, the evaluation network learns multimodal data such as EEG data and subject adaptability scores, enabling it to evaluate more data such as the reliability of network collaborative signals, user parameters, and user adaptability. This allows for the generation of high-precision usability labels in the usability evaluation of the first SSVEP paradigm set in the brain-computer interface system, improving the accuracy of the usability testing and evaluation method for the brain-computer interface system. It can also reduce the cost of deploying BCI systems in complex scenarios such as medical rehabilitation and game control, and contribute to providing a standardized evaluation paradigm for the practical application of brain-computer interface technology.

[0101] In embodiments of this disclosure, the stimulation unit may be a flashing block or the like.

[0102] The embodiments of this disclosure provide a usability testing and evaluation device for a brain-computer interface system. This device may include a data acquisition module and an evaluation module. The data acquisition module can be configured to acquire the subject's EEG data based on a first SSVEP paradigm set in the brain-computer interface system. The method for acquiring the EEG data can be found in the relevant descriptions in the foregoing embodiments. The evaluation module can be configured to input the parameters of the first SSVEP paradigm and the EEG data into an evaluation network to generate a usability label corresponding to the first SSVEP paradigm, as described in the relevant descriptions in the foregoing embodiments.

[0103] The evaluation network is trained using EEG data samples, parameter samples, and label samples. The EEG data samples are the EEG data corresponding to the second SSVEP paradigm in the brain-computer interface system. The parameter samples are the parameters of the second SSVEP paradigm. The label samples are the usability labels of the second SSVEP paradigm, determined by the EEG data and the subjects' fitness scores for the second SSVEP paradigm. The evaluation network can also be found in the relevant descriptions in the foregoing embodiments.

[0104] The edge computing device provided in the embodiments of this disclosure may include a processor, a memory, and a computer program / instructions. There may be one or more processors coupled to the memory, and the computer program is stored in the memory. When the processor invokes and executes the computer program in the memory, the edge computing device can perform the usability testing and evaluation method for the brain-computer interface system provided in any of the above embodiments.

[0105] like Figure 5 As shown, the edge computing device 50 includes a processor 51, a memory 52, and a computer program 53. The computer program 53 is stored in the memory 52, and the processor 51 is coupled to the memory 52. ​​When the processor 51 calls and executes the computer program 53 in the memory 52, the edge computing device 50 can execute the usability testing and evaluation method of the brain-computer interface system provided in any of the above embodiments.

[0106] The embodiments of this disclosure have now been described in detail. To avoid obscuring the concept of this disclosure, some details known in the art have not been described. Those skilled in the art can fully understand how to implement the technical solutions disclosed herein based on the above description.

[0107] While specific embodiments of this disclosure have been described in detail by way of examples, those skilled in the art should understand that the examples are for illustrative purposes only and not intended to limit the scope of this disclosure. Those skilled in the art should understand that modifications can be made to the above embodiments or equivalent substitutions can be made to some technical features without departing from the scope and spirit of this disclosure. In particular, as long as there is no structural conflict, the technical features mentioned in the various embodiments can be combined in any manner.

Claims

1. A method for usability testing and evaluation of a brain-computer interface system, characterized in that, include: Based on the first SSVEP paradigm set in the brain-computer interface system, the subject's EEG data was obtained; The parameters of the first SSVEP paradigm and the EEG data are input into the evaluation network to generate usability labels corresponding to the first SSVEP paradigm. The evaluation network is trained from EEG data samples, parameter samples, and label samples. The EEG data samples are the EEG data corresponding to the second SSVEP paradigm in the brain-computer interface system. The parameter samples are the parameters of the second SSVEP paradigm. The label samples are the usability labels of the second SSVEP paradigm, which are determined by the EEG data and the subject's fitness score for the second SSVEP paradigm. The process of determining the label sample includes: Obtain the subject's fitness score for the second SSVEP paradigm; Calculate the recognition accuracy of the EEG data samples; Based on the average signal-to-noise ratio of the EEG data samples, the recognition accuracy, and the adaptability score, a comprehensive evaluation index value is calculated by weighted summation. Based on the comprehensive evaluation index value, the label sample corresponding to the EEG data sample is determined; wherein, the formula for calculating the comprehensive evaluation index value is: s = k1 × s1 + k2 × s2 + k3 × s3 In the formula, k1 is the first weight, k2 is the second weight, k3 is the third weight, s1 is the first evaluation index value, which is determined by the average signal-to-noise ratio of the EEG data samples based on the first preset piecewise function, s2 is the second evaluation index value, which is determined by the adaptability score, and s3 is the third evaluation index value, which is determined by the recognition accuracy based on the second preset piecewise function.

2. The method according to claim 1, characterized in that, Obtaining the subject's adaptation score to the second SSVEP paradigm includes at least: obtaining a visual comfort score and a visual fatigue score. The second evaluation index value s2 is determined by the adaptation score, including: The value of the second evaluation index is determined using the following formula: s2 = a × (Sc - Sf) + b, where a and b are integers, Sc is the visual comfort score, and Sf is the visual fatigue score.

3. The method according to claim 1, characterized in that, The evaluation network includes at least a feature extraction network and a fully connected layer. The feature extraction network includes at least a convolutional neural network layer, a pooling layer, a dual-stream long short-term memory network layer, and a feature fusion module. The step of inputting the parameters of the first SSVEP paradigm and the EEG data into the evaluation network to generate a usability label corresponding to the first SSVEP paradigm includes: Obtain the EEG data and the number of stimulation units corresponding to the first SSVEP paradigm, and extract spatiotemporal features from the EEG data corresponding to the first SSVEP paradigm using the convolutional neural network layer based on the number of stimulation units. Based on the single test duration of the subject under the first SSVEP paradigm, the pooling layer is used to extract significant features from the spatiotemporal features in the EEG data corresponding to the first SSVEP paradigm. The EEG data corresponding to the first SSVEP paradigm, the wavelet transform coefficients of the EEG data, and the frequency interval of the stimulation units are determined. The temporal features in the EEG data corresponding to the first SSVEP paradigm are extracted using the temporal LSTM in the dual-stream long short-term memory network layer. The frequency features in the EEG data corresponding to the first SSVEP paradigm are extracted using the frequency LSTM in the dual-stream long short-term memory network layer based on the frequency interval of the stimulation units and the wavelet transform coefficients. Based on the spatiotemporal features, the salient features, the temporal features, and the frequency features, the feature fusion module is used to perform feature fusion and generate dynamic evaluation features. Based on the dynamic evaluation features, the availability label is output using the fully connected layer.

4. The method according to claim 3, characterized in that, The step of extracting spatiotemporal features from the EEG data corresponding to the first SSVEP paradigm using the convolutional neural network layer based on the number of stimulation units includes: The kernel size of the convolutional neural network layer is determined based on the number of stimulation units. Using a convolutional neural network layer with a determined kernel size, the spatiotemporal features are extracted from the EEG data corresponding to the first SSVEP paradigm.

5. The method according to claim 4, characterized in that, Determining the kernel size of the convolutional neural network layer based on the number of stimulation units includes: When the number of stimulation units is less than or equal to 8, the size of the convolution kernel is determined to be 3; When the number of stimulation units is greater than 8 and less than or equal to 20, the size of the convolution kernel is determined to be k= Where k is the size of the convolution kernel and n is the number of the stimulation units; When the number of stimulation units is greater than 20, the size of the convolution kernel is determined to be k= .

6. The method according to any one of claims 3-5, characterized in that, The dual-stream long short-term memory network layer further includes a step size allocator, and the method further includes: Based on the duration of a single test conducted by the subject in the first SSVEP paradigm, the step length is determined using the step length allocator. The second step size is determined using the step size distributor based on the frequency interval of the stimulation units. The first step length is used for the time-stream LSTM to extract the time features, and the second step length is used for the frequency-stream LSTM to extract the frequency features.

7. A usability testing and evaluation device for a brain-computer interface system, characterized in that, include: The data acquisition module is configured to acquire the subject's EEG data based on the first SSVEP paradigm set in the brain-computer interface system. An evaluation module is configured to input the parameters of the first SSVEP paradigm and the EEG data into an evaluation network to generate a usability label corresponding to the first SSVEP paradigm. The evaluation network is trained from EEG data samples, parameter samples, and label samples. The EEG data samples are the EEG data corresponding to the second SSVEP paradigm in the brain-computer interface system. The parameter samples are the parameters of the second SSVEP paradigm. The label samples are the usability labels of the second SSVEP paradigm, which are determined by the EEG data and the subject's fitness score for the second SSVEP paradigm. The process of determining the label sample includes: Obtain the subject's fitness score for the second SSVEP paradigm; Calculate the recognition accuracy of the EEG data samples; Based on the average signal-to-noise ratio of the EEG data samples, the recognition accuracy, and the adaptability score, a comprehensive evaluation index value is calculated by weighted summation. Based on the comprehensive evaluation index value, the label sample corresponding to the EEG data sample is determined; wherein, the formula for calculating the comprehensive evaluation index value is: s = k1 × s1 + k2 × s2 + k3 × s3 In the formula, k1 is the first weight, k2 is the second weight, k3 is the third weight, s1 is the first evaluation index value, which is determined by the average signal-to-noise ratio of the EEG data samples based on the first preset piecewise function, s2 is the second evaluation index value, which is determined by the adaptability score, and s3 is the third evaluation index value, which is determined by the recognition accuracy based on the second preset piecewise function.

8. An edge computing device, characterized in that, It includes a processor and a memory, the memory storing a program or instructions that can run on the processor, the program or instructions being executed by the processor to implement the method as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when run on a computer, enables the computer to perform the method described in any one of claims 1-6.

Images