Cross-subject passive automatic classification method and system for pitch perception based on eeg
Patent Information
- Authority / Receiving Office
- AU · AU
- Patent Type
- Applications
- Current Assignee / Owner
- SHANDONG UNIV
- Filing Date
- 2024-12-02
- Publication Date
- 2026-07-30
AI Technical Summary
Existing objective pitch perception methods based on EEG rely on single-subject modeling, which limits the generalization of built models and requires extensive data collection, making them impractical for assessing pitch perception in groups like newborns and prelingually deafened children with cochlear implants.
A cross-subject passive automatic classification method using EEGNet models, involving data filtering, single-subject optimization, and cross-subject modeling with a model library and voting strategy to enhance generalization and reduce data collection time.
The method achieves objective pitch perception classification with enhanced generalization and reduced data collection time, utilizing deep learning for feature extraction and model integration, improving classification accuracy and applicability across multiple subjects.
Smart Images
Figure 00000002_0000 
Figure 00000024_0000 
Figure 00000025_0000
Abstract
Description
TECHNICAL FIELD
[0001] The present invention belongs to the technical field of data processing, in particular to a cross-subject passive automatic classification method for pitch perception based on EEG. BACKGROUND OF THE PRESENT INVENTION
[0002] The statement in this part only provides the background information related to the present invention, and does not necessarily constitute the prior art.
[0003] Event-related potentials (ERP) induced by acoustic stimulation can be used as biomarkers for hearing assessment, such as mismatch negativity (MMN) which can be used for assessing the resolution of pitch perception. Although pure tone pitch (i.e. single frequency) stimulation can be used for auditory electroencephalogram (aEEG) assessment, it is not wide spectrum nature sound and not practical.
[0004] At present, subjective audiometry questionnaires are mainly used for hearing assessment clinically. However, not all people can perform subjective audiometry, e.g., newborns and prelingually deafened children with cochlear implants (CIs). The objective assessment of EEG based on acoustic stimulation does not require subjects to provide feedback to acoustic stimulation, but only needs to passively perceive acoustic stimulation so as to quantitatively assess the aEEG response modes of brain output. This will be more suitable for the hearing assessment of this group of people.
[0005] Although there are some researches on objective pitch perception based on EEG, they are mainly the qualitative detection method or the quantitative detection method. Both of the objective pitch classification methods rely on collecting large amounts of auditory EEG data for each subject, then manually extracting corresponding spatial and temporal features, and modeling aEEG classifiers based on machine learning. Modeling methods are mainly performed based on single subjects, making it difficult to improve the generality of 2024267092 02 Dec 2024 the built models in tasks of assessing pitch perception. SUMMARY OF PRESENT INVENTION
[0006] To overcome the shortcomings of the prior art, the present invention provides a cross-subject passive automatic classification method for pitch perception based on EEG, which can effectively perform objective pitch perception classification.
[0007] To achieve the above purpose, one or more embodiments of the present invention provide the following technical solutions:
[0008] In a first aspect, a cross-subject passive automatic classification method for pitch perception based on aEEG is disclosed, comprising:
[0009] collecting EEG data of each subject for pitch perception;
[0010] filtering all the collected EEG data by two filters, wherein one filter is a high-pass filter for removing DC components in EEG signals, and the other filter is a notch filter for removing power frequency interference;
[0011] dividing the EEG data collected from each subject into a training set and a test set, and determining optimal hyperparameters of EEGNet models based on the training set and the test set, wherein the data of N subjects are collected to obtain N modified EEGNet models, and the N EEGNet models constitute a model library for cross-subject training;
[0012] when cross-subject modeling is carried out on a new subject, using part of data of the new subject as a calibration data set, and using the remaining part as a test set, wherein the calibration data set is used for classification prediction of all built models for single subjects in the model library and sorting the accuracy of the N EEGNet models under the calibration data set to select the top M EEGNet models with superior classification performance to form a preferred model set, to obtain final classification prediction results based on the preferred model set.
[0013] In the technical solution of the present invention, the basis of deep learning is: a multilayer neural network structure, and an end-to-end model for automatic feature extraction. The task-related classification features can be automatically learned through the above mode. Compared with machine learning algorithms, the present invention has the advantage of strong generalization ability. 2024267092 02 Dec 2024
[0014] As a further technical solution, when the optimal hyperparameters of the EEGNet models are determined, a sequence of hyperparameter optimization of the EEGNet models is as follows:
[0015] 1) the number and the size of convolution kernels in a first layer of the EEGNet model;
[0016] 2) the number of convolution kernels in a second layer of the EEGNet model;
[0017] 3) an average pooling scale of the first layer of the EEGNet model;
[0018] 4) an average pooling scale of the second layer of the EEGNet model.
[0019] As a further technical solution, for single-subject optimization modeling, the number and the size of the convolution kernels in the first layer of the model are optimized as follows:
[0020] the number of the convolution kernels in the second layer, the average pooling scale of the first layer, and the average pooling scale of the second layer of the EEGNet models are fixed to train a plurality of EEGNet models respectively;
[0021] after 5-fold cross-validation of each model, the average accuracy of the models on the test set is recorded, and the number and the size of the convolution kernels in the first layer corresponding to the model with the highest average accuracy are recorded.
[0022] As a further technical solution, for single-subject optimization modeling, the number of the convolution kernels in the second layer of the model is optimized as follows:
[0023] the number and the size of the convolution kernels in the first layer, the average pooling scale of the first layer, and the average pooling scale of the second layer of the EEGNet models are fixed to train a plurality of EEGNet models respectively; and after 5-fold cross-validation of each model, the average accuracy of each model on the test set is recorded, and the number of the convolution kernels in the second layer corresponding to the model with the highest average accuracy is recorded.
[0024] As a further technical solution, for single-subject optimization modeling, the average pooling scale of the first layer of the model is optimized as follows:
[0025] the number and the size of the convolution kernels in the first layer, the number of the convolution kernels in the second layer and the average pooling scale of the second layer of the EEGNet models are fixed to train a plurality of EEGNet models respectively; and after 2024267092 02 Dec 2024 5-fold cross-validation of each model, the average accuracy of each model on the test set is recorded, and the average pooling scale of the first layer corresponding to the model with the highest average accuracy is recorded.
[0026] As a further technical solution, for single-subject optimization modeling, the number of the convolution kernels in the second layer of the model is optimized as follows:
[0027] the number of the convolution kernels in the first layer, the number of the convolution kernels in the second layer and the average pooling scale of the first layer of the EEGNet models are fixed to train a plurality of EEGNet models respectively; and after 5-fold cross-validation of each model, the average accuracy of each model on the test set is recorded, and the number of the convolution kernels in the second layer corresponding to the model with the highest average accuracy is recorded.
[0028] As a further technical solution, the modified EEGNet models are used for pitch perception decoding and predictive classification:
[0029] the collected EEG data of each subject for pitch perception is specifically a CxT signal matrix, wherein C is the number of channels and T is the number of EEG signal sampling points;
[0030] inputted CxT signals pass through a subchannel convolutional layer containing N1 convolution kernels which can be regarded as time domain filters in each channel, then pass through an average pooling layer with a pooling scale of N3 to reduce the feature dimension, pass through a convolutional layer with N2 convolution kernels of size (C, 1) which can be regarded as space domain filters, pass through an average pooling layer with a pooling scale of N4 to further reduce the feature dimension, then pass through a subchannel convolutional layer with 1 convolution kernel in each channel, and then pass through a convolutional layer with N5 convolution kernels; the final outputted features are mapped through Softmax to obtain a posteriori probability of each class; and finally, a kind of labels with the highest probability is selected as final model output.
[0031] In a second aspect, a cross-subject passive automatic classification system for pitch perception based on EEG on the basis of deep learning is disclosed, comprising:
[0032] an EEG data collecting module configured to collect EEG data of each subject for pitch perception; 2024267092 02 Dec 2024
[0033] an EEG data filtering module configured to filter all the collected EEG data by two filters, wherein one filter is a high-pass filter for removing DC components in EEG signals, and the other filter is a notch filter for removing power frequency interference;
[0034] a model library building block configured to divide the EEG data collected from each subject into a training set and a test set, and determine optimal hyperparameters of the modified EEGNet models based on the training set and the test set, wherein the data of N subjects are collected to obtain N modified EEGNet models, and the N EEGNet models constitute a model library for cross-subject training;
[0035] a cross-subject modeling module configured to use part of data of the new subject as a calibration data set, and use the remaining part as a test set when cross-subject modeling is carried out on a new subject, wherein the calibration data set is used for classification prediction of all built models for single subjects in the model library and sorting the accuracy of the N modified EEGNet models under the calibration data set to select the top M EEGNet models with superior classification performance to form a preferred model set, to obtain final classification prediction results based on the preferred model set.
[0036] The above one or more technical solutions have the following beneficial effects:
[0037] The technical solution of the present invention is a general automatic classification method for pitch perception based on objective EEG data under natural pitch stimulation. The EEG signals of 11 normal hearing subjects stimulated by G3 and B6 violin sound are collected based on the Go / No-Go paradigm of passive perception, to constitute the data set for research. On this basis, the modified EEGNet models are used to perform classification experiments under pitch perception tasks, and the hyperparameters of the models are optimized, so as to obtain the modified EEGNet model library of the single subjects.
[0038] In the technical solution of the present invention, a cross-subject general pitch perception classification model is constructed based on the model library of 11 single subjects through a classifier integration method. In addition, in order to determine the effective data segments for decoding pitch perception, a plurality of EEG data segments are selected for classification and comparison experiments. The modified EEGNet models can effectively classify pitch perception objectively for the pitch perception tasks. A novel cross-subject pitch perception assessment method based on model migration and classifier 2024267092 02 Dec 2024 integration can significantly enhance the generalization performance of the models. The results of the technical solution of the present invention can provide guidance for the design of an objective pitch perception assessment system.
[0039] Advantages of appended aspects of the present invention will be partially supplied in the following description. Parts will become apparent from the following description, or will be known through the practice of the present invention. DESCRIPTION OF THE DRAWINGS
[0040] Drawings of the Description forming a part of the present invention are used for providing further understanding of the present invention. Exemplary embodiments of the present invention and the description are used for explaining the present invention, but do not constitute an improper limitation to the present invention.
[0041] Fig. 1 is a flow chart of a cross-subject modeling process based on modified EEGNet models in an embodiment of the present invention;
[0042] Fig. 2 is a structural schematic diagram of an the modified EEGNet model involved in the present invention;
[0043] Fig. 3 is a schematic diagram of classification performance under cross-subject modeling conditions in an embodiment of the present invention. DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS
[0044] It should be pointed out that the following detailed descriptions are all illustrative and are intended to provide further description of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those generally understood by those ordinary skilled in the technical field to which the present invention belongs.
[0045] It should be noted that the terms used herein are intended to merely describe specific embodiments, not to limit the exemplary embodiments according to the present invention.
[0046] If there is no conflict, the embodiments in the present invention and features in the embodiments can be mutually combined.
[0047] Overall conception:
[0048] The EEG signals under stimulation of G3 and B6 violin sound (or note) pitch are 2024267092 02 Dec 2024 recorded by the Go / No-Go paradigm of passive perception, to constitute the used data set.
[0049] In this paradigm, the probability of stimulation is equal, and the subjects perceive the sound passively without selection by buttons. Compared with the Oddball paradigm, the Go / No-Go paradigm has the advantage of short experiment time (high efficiency) under the same acoustic stimulation and interval conditions.
[0050] To illustrate the effect of EEG time periods on pitch classification, the performance of pitch classification of modified EEGNet models based on EEG data under multiple time windows is analyzed.
[0051] At the same time, in order to explore the effectiveness of objective pitch prediction under the deep learning model, a single-subject pitch perception classification experiment is conducted based on the modified EEGNet models, and the hyperparameters of the models are optimized, so as to construct a single-subject EEGNet model library.
[0052] Further, in order to construct a general, high-precision and cross-subject pitch perception classification model, a classifier integration method is introduced on the basis of the modified EEGNet model library to propose a cross-subject pitch perception classification model.
[0053] In addition, in order to determine the effective data segments for decoding pitch perception, a plurality of EEG data segments are selected in the research for classification comparison experiments.
[0054] Explanation of terms
[0055] Single-subject optimization modeling:
[0056] Definition: An independent model is built for each subject respectively. The core of this modeling mode is individualized processing, that is, to customize a model most suitable for the data features for each subject.
[0057] Process: In single-subject optimization modeling, firstly, the model is established independently for each subject. Then, multiple hyperparameters of model, such as different EEG window lengths, different number of convolution kernels, etc., are traversed to find and determine the model configuration that can achieve the best average accuracy on the test data set for a specific subject.
[0058] Purpose: The goal of the method is to find an optimal model for each subject. The 2024267092 02 Dec 2024 “best” here refers to the highest average accuracy that the model exhibits on the test data set for the subject.
[0059] Cross-subject optimization modeling:
[0060] Definition: Unlike single-subject optimization modeling, the cross-subject optimization modeling uses the data from multiple subjects to build a general model. This method aims to discover a unified model that can be applied to multiple subjects.
[0061] Process: Firstly, the data of multiple subjects is collected, and used as the basis for building a common model. Then, different model hyperparameters are adjusted and traversed to find a model configuration that can perform best on the test data set for all subjects.
[0062] Purpose: The ultimate purpose of the method is to create an optimal model that is universally applicable. The “best” here still means that the model has the highest average accuracy on the test data set for all subjects.
[0063] Subchannel convolution:
[0064] Channel convolution refers to convolution operation in which each channel of the input is convolved with a group of own filters respectively. This method is used for applying different filters to different channels to allow for more customized and efficient processing of input data.
[0065] Back propagation algorithm:
[0066] The back propagation algorithm is an efficient gradient calculation method for optimizing the weights of a neural network. By calculating the gradient of a loss function relative to network parameters, the back propagation algorithm can effectively adjust the weights in the network to minimize the loss function. In the modified EEGNet models, the back propagation algorithm enables the model to gradually improve the classification accuracy of the EEG data through iterative training, so as to realize the effective analysis and application of EEG signals.
[0067] Embodiment 1
[0068] The present embodiment discloses a cross-subject passive automatic classification method for pitch perception based on EEG, comprising:
[0069] Fig. 1 shows the process of cross-subject modeling based on EEGNet. Firstly, the cross-subject modeling needs to build a model library based on the single-subject modeling 2024267092 02 Dec 2024 method. Assuming that there are EEG data of N subjects, there will be N modified EEGNet single models, and the structure of each modified EEGNet single model is shown in Fig. 2. When cross-subject modeling is performed on a new subject, part (assumed as m%) of the data of the new subject is used as a calibration data set, and the remaining part is used as a test set (1-m%). The functions of the calibration data set are classification prediction for all the models built for single subjects in the model library, and sorting of the accuracy of N models under the calibration data set. Then, the top M=3 modified EEGNet models with superior classification performance are selected to form an integrated optimization model. Finally, based on the integrated optimization model, the final classification prediction result is obtained through a voting strategy under the test set. The prediction result is the accuracy of the model to identify the samples of the test set. The specific steps of the solution are as follows:
[0070] Step 1: The training data is collected with a data collector (EEG signal amplifier), and the Go / No-Go paradigm of passive perception is adopted for collection from each subject to obtain the EEG data of Z trials. In the present embodiment, Z is 200. Specifically, the Go / No-Go paradigm of passive perception is used for conducting experiments of 200 trials for each subject, and the experiment length of each trial is 5 seconds, wherein the first 2 seconds (0-2 seconds) are the preparation time, and a cross appears on the screen (the subjects are required to stare at the cross to concentrate the attention). The next 2 seconds (2-4 seconds) are the experiment time, and the subjects hear two different note sounds (such as violin G3 tone or B6 tone) at random. At the last second (4-5 seconds), the screen is black, and it is the rest time of the subjects. Throughout the experiment, the subjects are allowed to rest for 1-2 minutes after 40 trials. In this paradigm, the subjects only need to listen to the sound passively without subjective feedback. Therefore, compared with the traditional Go / No-Go paradigm with subjective feedback, the Go / No-Go paradigm of passive perception takes less time and can capture the objective EEG features of the subjects for pitch perception.
[0071] Step 2: For each subject, if the sampling frequency of the EEG collector is 500 Hz, 200 Cx T signal matrices can be obtained under the Go / No-Go paradigm of passive perception, wherein C=59 is the number of channels, and T=1000 is the number of EEG 2024267092 02 Dec 2024 signal sampling points.
[0072] Step 3: All the collected EEG data, i.e., the signal matrices, are filtered by two Butterworth filters in sequence. One Butterworth filter is a high-pass filter of order Nf=5, and the cutoff frequency is 0.1 Hz, which can effectively remove the DC component in the EEG signal; and the other Butterworth filter is a notch filter with a notch frequency of 50 Hz to remove power frequency interference. By removing noise signals unrelated to EEG, the model is prevented from learning noise-related features to avoid interfering with the classification results.
[0073] Step 4: Single-subject optimization modeling based on typical EEGNet is conducted. The data of 200 trials of each subject after filtering are randomly divided into a training set and a test set based on 5-fold cross-validation, and 10% of the training set data is randomly selected from the training set as a validation set. Then, the hyperparameters of the modified EEGNet models are optimized. In order to determine the optimal hyperparameters to improve the classification performance of pitch perception, the optimal hyperparameter of each subject under the EEGNet model is determined in sequence by a fixed variable method based on the classification prediction accuracy in the research. A sequence of hyperparameter optimization of the models is as follows:
[0074] 1) the number N1 of convolution kernels in a first layer of the EEGNet model;
[0075] 2) the number N2of convolution kernels in a second layer of the EEGNet model;
[0076] 3) an average pooling scale N3 of the first layer of the EEGNet model;
[0077] 4) an average pooling scale N4of the second layer of the EEGNet model.
[0078] The number of the convolution kernels in the first layer of the EEGNet model is optimized from 8, 16, 32, 48 and 56; the number of the convolution kernels in the second layer of the EEGNet model is optimized from 1, 2, 4, 6 and 8; the average pooling scale of the first layer of the EEGNet model is optimized from 2, 4, 6, 8 and 10; and the average pooling scale of the second layer of the EEGNet model is optimized from 4, 6, 8, 10 and 12.
[0079] In addition, the scale (order) of each convolution kernel of the first-layer convolution is L1=64, and the scale (order) of each convolution kernel of the second-layer convolution is equal to the number C of channels. The number of the convolution kernels of the third-layer convolution is N^ N'ixN;, the size of the convolution kernels is (1,8), and the 2024267092 02 Dec 2024 size of the average pooling layer at the third layer is (1,8).
[0080] For single-subject optimization modeling, the model is optimized as follows:
[0081] Step 4-1): The number of the convolution kernels in the second layer, the average pooling scale of the first layer, and the average pooling scale of the second layer of the EEGNet models are fixed as 1, 2 and 6 to train 5 EEGNet models respectively; and the number of the convolution kernels in the first layers of the five EEGNet models is 8, 16, 32, 48 and 56 respectively. After 5-fold cross-validation of each model, the average accuracy of each model on the test set is recorded. The number of the convolution kernels in the first layer corresponding to the model with the highest average accuracy is recorded as as N1=16.
[0082] Step 4-2): The number of the convolution kernels in the first layer, the average pooling scale of the first layer, and the average pooling scale of the second layer of the EEGNet models are fixed as N1=16, 2 and 6 to train 5 EEGNet models respectively; and the number of the convolution kernels in the second layers of the five EEGNet models is 1, 2, 4, 6 and 8 respectively. After 5-fold cross-validation of each model, the average accuracy of each model on the test set is recorded. The number of the convolution kernels in the second layer corresponding to the model with the highest average accuracy is recorded as N2=6.
[0083] Step 4-3): The number of the convolution kernels in the first layer, the number of the convolution kernels in the second layer and the average pooling scale of the second layer of the EEGNet models are fixed as N1=16, N2=6 and 6 to train 5 EEGNet models respectively; and the average pooling scale of the first layers of the five EEGNet models is 2, 4, 6, 8 and 10 respectively. After 5-fold cross-validation of each model, the average accuracy of each model on the test set is recorded. The average pooling scale of the first layer corresponding to the model with the highest average accuracy is recorded as N3=10.
[0084] Step 4-4): The number of the convolution kernels in the first layer, the number of the convolution kernels in the second layer and the average pooling scale of the first layer of the EEGNet models are fixed as N1=16, N2=6 and N3=10 to train 5 EEGNet models respectively; and the average pooling scale of the second layers of the five EEGNet models is 4, 6, 8, 10 and 12 respectively. After 5-fold cross-validation of each model, the average accuracy of each model on the test set is recorded. The average pooling scale of the second layer corresponding to the model with the highest average accuracy is recorded as N4=4. 2024267092 02 Dec 2024
[0085] Step 4-5): After the above steps, the optimal structure parameters of the modified EEGNet models corresponding to the subjects are obtained.
[0086] Step 5: The data of N=11 subjects are collected, so the N modified EEGNet models will be obtained in the research. The N models will constitute a model library used in cross-subject training for cross-subject modeling.
[0087] Step 6: Cross-subject modeling is conducted, as shown in Fig. 1. After single-subject optimization modeling, the modified models of all subjects are built into a model library. Assuming that there are EEG data of N subjects, N single-subject models can be modeled by the single-subject modeling method. When cross-subject modeling is performed on a new subject, part (assumed as m%) of the data of the new subject is used as a calibration data set, and the remaining part is used as a test set (1-m%). In the present invention, the proportion of the calibration data set includes 20%, 40%, 60% and 80% (that is, m=20, 40, 60 and 80), and the proportion of the test set includes 80%, 60%, 40% and 20%. The functions of the calibration data set are classification prediction for all the models built for single subjects in the model library, and sorting of the accuracy of N models under the calibration data set. Next, the top M EEGNet models with superior classification performance are selected to form a preferred model set. Finally, final classification prediction results are obtained under the test set based on the preferred model set through a voting strategy.
[0088] It should be indicated about the voting strategy that M models predict the input samples, and M prediction results will appear. The M results are statistically sorted, and the result with the largest number of predictions is outputted as the final result, that is, the voting strategy.
[0089] Specifically, cross-subject modeling is performed on a new subject. Firstly, the Go / No-Go paradigm of passive perception is used to collect EEG data of 200 trials. Then, m=20, that is, 20% of the data (corresponding to the EEG data of 40 trials) is taken as the calibration data set, and the remaining 80% of the data (160 tries) is used as the test set. N=11 single subject models in the model library are tested on the calibration data set to obtain N=11 accuracy, and the top M=3 models with the best classification performance are selected to form a preferred model set. Finally, all models in the preferred model set are 2024267092 02 Dec 2024 tested by voting strategy on the test set to obtain the average accuracy as the final assessment result. Higher average accuracy indicates that the EEG signals of the new subject include more features related to pitch perception, and thus the pitch perception ability is better.
[0090] It should be noted that the embodiment of the present invention adopts the EEGNet models for automatic feature extraction and classification. The EEGNet models are lightweight deep networks proved to learn various features in multiple brain-computer interface tasks, and have certain robustness.
[0091] Therefore, the present invention selects the EEGNet models for pitch perception decoding and prediction classification, specifically including:
[0092] Firstly, the inputted C* T signals pass through a subchannel convolutional layer containing N1 convolution kernels of size (1, L1) in each channel, and the convolution kernels can be regarded as time domain filters.
[0093] Then, the signals pass through an average pooling layer with a pooling scale of N3 to reduce the feature dimension.
[0094] Subsequently, the signals pass through a convolutional layer having N2 convolution kernels of size (C, 1), and the convolution kernels can be regarded as space domain filters.
[0095] The signals pass through an average pooling layer with a pooling scale of N4 to further reduce the feature dimension.
[0096] Then, the signals pass through a convolutional layer containing N5=N1*N2 convolution kernels of size (1, 16) and pass through an average pooling layer with of size (1, 8) to further reduce the feature dimension. N5 is a model parameter, which is specifically a feature matrix generated after spatial filtering.
[0097] The final outputted features are mapped through Softmax to obtain a posteriori probability of each class; and finally, a kind of labels with the highest probability is selected as final model output.
[0098] The convolutional layer is the core of a convolutional network, which can realize feature extraction of input data by learning one or more groups of convolution kernels. Given ■K g IR • • • ± ~ " with length ki and width k2, with T being the number of EEG signal sampling points, then the feature S outputted by the 2024267092 02 Dec 2024 convolutional layer can be expressed as:
[0099] s = x * k + b (1)
[00100] where b is the offset of the convolution kernels.
[00101] In the embodiment of the present invention, the EEGNet models are trained by a back propagation algorithm, with the cross-entropy loss function as an optimization objective. The cross-entropy loss function is a loss function commonly used in classification problems, especially suitable for multi-class classification tasks. The loss function in the model training of the present invention is a minimum classification cross-entropy loss function, and the formula is as follows: 1N E=-—EC y„ log y n+(1 - y-) log (1 - y n)]
[00102] N n=1 (2)
[00103] where N represents the number of samples (trials), yn represents a label value of the corresponding sample, and yn represents a predicted value of the sample label. The cross-entropy loss function measures the classification prediction accuracy of the model for electroencephalogram (EEG) data. A lower value of the loss function indicates that the prediction of the model is closer to the actual data label, that is, the performance of the model is better.
[00104] The present embodiment is based on the classification performance of 11 subjects in different cross-subject situations. In Fig. 3, the dotted line at the top indicates that the average classification accuracy of the model is 77.3% in the case of a single subject, the dotted line in the bottom indicates that the average classification accuracy is 51.8% in the case that the last three classifiers are selected in the integrated classification method, and the dotted line at the middle indicates that the average classification accuracy is 60% in the case that three classifiers are randomly selected in the integrated classification method. For the cross-subject case, a classifier integration optimization method has a classification accuracy of 71.2% when 3 out of 10 and 80% of the data are in the calibration data set. On the whole, the classification accuracy of a classifier integration model is increased with the increase of the number of optional subjects. This indicates that the classifier integration model based on the modified EEGNet can learn the general features of the pitch perception task when there 2024267092 02 Dec 2024 are a large number of subjects, so as to effectively classify two kinds of pitch stimulation of new subjects. Meanwhile, the classification accuracy of the classifier integration optimization method is increased with the increase of the calibration data set, which may be because larger calibration data enables the integrated classifier to learn more pitch features of new subjects, so as to improve the effectiveness of cross-subject models.
[00105] Meanwhile, the classification accuracy of the classifier integration optimization method is significantly higher than that of the selected last three classifiers or the randomly selected three classifiers, which indicates that there is good consistency between the features learned by the optimized first three classifiers and the features of the new subjects in the pitch perception task, thereby improving the accuracy of the prediction. However, the highest classification accuracy of the classifier integration optimization method is lower than that of the single-subject case by 6.1%, which may be because there are certain differences in the brain network of each subject in the pitch perception task. Thus, it is difficult for the classifier integration optimization method to learn the personality characteristics of each subject. Although the accuracy of the proposed cross-subject modeling method is lower than that of the single-subject modeling method, the cross-subject modeling method can reduce the EEG collecting time of new subjects, and has practical application value.
[00106] The Go / No-Go paradigm of passive perception adopted in the present embodiment records EEG signals under the stimulation of G3 and B6 violin sound, and uses the EEGNet models to conduct a single-subject modeling classification experiment, a cross-subject modeling classification experiment and an EEG data segment optimization experiment. The research shows that the proposed cross-subject modeling method can effectively reduce the EEG data collection time of new subjects, and has application value and potential for objective pitch perception classification.
[00107] Embodiment 2
[00108] The purpose of the present embodiment is to provide a computer device comprising a memory, a processor, and a computer program stored on the memory and run on the processor, and the steps of the method are implemented when the program is executed by the processor.
[00109] Embodiment 3 2024267092 02 Dec 2024
[00110] The purpose of the present embodiment is to provide a computer readable storage medium.
[00111] A computer readable storage medium stores computer programs which can be executed the steps of the method by a processor.
[00112] Embodiment 4
[00113] The purpose of the present embodiment is to provide a cross-subject passive automatic classification system for pitch perception based on EEG on the basis of deep learning, comprising:
[00114] an EEG data collecting module configured to collect EEG data of each subject for pitch perception;
[00115] an EEG data filtering module configured to filter all the collected EEG data by two filters, wherein one filter is a high-pass filter for removing DC components in EEG signals, and the other filter is a notch filter for removing power frequency interference;
[00116] a model library building block configured to divide the EEG data collected from each subject into a training set and a test set, and determine optimal hyperparameters of EEGNet models based on the training set and the test set, wherein the data of N subjects are collected to obtain N modified EEGNet models, and the N EEGNet models constitute a model library for cross-subject training;
[00117] a cross-subject modeling module configured to use part of data of the new subject as a calibration data set, and use the remaining part as a test set when cross-subject modeling is carried out on a new subject, wherein the calibration data set is used for classification prediction of all built models for single subjects in the model library and sorting the accuracy of the N EEGNet models under the calibration data set to select the top M modified EEGNet models with superior classification performance to form a preferred model set, to obtain final classification prediction results based on the preferred model set and voting strategy.
[00118] The steps involved in the devices of embodiments 2, 3, and 4 above correspond to embodiment 1 of the method, and specific embodiments can be found in the relevant description of embodiment 1. The term “computer readable storage medium” shall be understood as a single medium or more media comprising one or more instruction sets, and shall also be understood to include any medium capable of storing, encoding, or carrying 2024267092 02 Dec 2024 instruction sets for execution by a processor and capable of enabling the processor to execute any of the methods in the present invention.
[00119] Those skilled in the art shall understand that the modules or steps of the present invention can be implemented by a general-purpose computer device and alternatively, can be implemented by program codes executable by a computing device, so that they can be stored in a storage device and executed by the computing device, or they can be made into various integrated circuit modules respectively, or multiple modules or steps among them are made into a single integrated circuit module for implementation. The present invention is not limited to any specific combination of hardware and software.
[00120] Although the above describes the specific embodiments of the present invention in combination with the drawings, it is not a limitation to the protection scope of the present invention. Those skilled in the art shall understand that on the basis of the technical solution of the present invention, various changes or modifications that can be made by those skilled in the art without contributing creative labor are still within the protection scope of the present invention.
Claims
1. A cross-subject passive automatic classification method for pitch perception based on EEG, comprising:collecting EEG data of each subject for pitch perception;filtering all the collected EEG data by two filters, wherein one filter is a high-pass filter for removing DC components in EEG signals, and the other filter is a notch filter for removing power frequency interference;dividing the EEG data collected from each subject into a training set and a test set, and determining optimal hyperparameters of modified EEGNet models based on the training set and the test set, wherein the data of N subjects are collected to obtain N modified EEGNet models, and the N EEGNet models constitute a model library for cross-subject training;wherein the modified EEGNet models are used for pitch perception decoding and predictive classification:the collected EEG data of each subject for pitch perception is specifically a C*T signal matrix, wherein C is the number of channels and T is the number of EEG signal sampling points;inputted C*T signals pass through a subchannel convolutional layer containing N1 convolution kernels capable of being regarded as time domain filters and having a scale of L1 in each channel, then pass through an average pooling layer with a pooling scale of N3 to reduce the feature dimension, subsequently pass through a convolutional layer with N2 convolution kernels of size (C, 1) which can be regarded as space domain filters, pass through an average pooling layer with a pooling scale of N4 to further reduce the feature dimension, then pass through a subchannel convolutional layer with 1 convolution kernel in each channel, and then pass through a convolutional layer with N5 convolution kernels; the final outputted features are mapped through Softmax to obtain a posteriori probability of each class; and finally, a kind of labels with the highest probability is selected as final model output;when cross-subject modeling is carried out on a new subject, using part of data of the new subject as a calibration data set, and using the remaining part as a test set, wherein the calibration data set is used for classification prediction of all built models for single subjects in the model library and sorting the accuracy of the N modified EEGNet models under the calibration data set to select top M EEGNet models with superior classification performance to form a preferred model set, to obtain final classification prediction results by voting strategy based on the preferred model set.2024267092 02 Dec 20242. The cross-subject passive automatic classification method for pitch perception based on EEG according to claim 1, wherein when the optimal hyperparameters of the modified EEGNet models are determined, a sequence of hyperparameter optimization of the modified EEGNet models is as follows:1) the number and the scale of convolution kernels in a first layer of the EEGNet model;2) the number of convolution kernels in a second layer of the EEGNet model;3) an average pooling scale of the first layer of the EEGNet model;4) an average pooling scale of the second layer of the EEGNet model.
3. The cross-subject passive automatic classification method for pitch perception based on EEG according to claim 2, wherein for single-subject optimization modeling, the number and the scale of the convolution kernels in the first layer of the model are optimized as follows:the number of the convolution kernels in the second layer, the average pooling scale of the first layer, and the average pooling scale of the second layer of the EEGNet models are fixed to train a plurality of EEGNet models respectively;after 5-fold cross-validation of each model, the average accuracy of the models on the test set is recorded, and the number and the scale of the convolution kernels in the first layer corresponding to the model with the highest average accuracy are recorded.
4. The cross-subject passive automatic classification method for pitch perception based on EEG according to claim 2, wherein for single-subject optimization modeling, the number of the convolution kernels in the second layer of the model is optimized as follows:the number and the scale of the convolution kernels in the first layer, the average pooling scale of the first layer, and the average pooling scale of the second layer of the EEGNet models are fixed to train a plurality of EEGNet models respectively; and after 5-fold cross-validation of each model, the average accuracy of each model on the test set is recorded, and the number of the convolution kernels in the second layer corresponding to the model with the highest average accuracy is recorded.
5. The cross-subject passive automatic classification method for pitch perception based on EEG according to claim 2, wherein for single-subject optimization modeling, the average pooling scale of the first layer of the model is optimized as follows:2024267092 02 Dec 2024the number and the scale of the convolution kernels in the first layer, the number of the convolution kernels in the second layer and the average pooling scale of the second layer of the EEGNet models are fixed to train a plurality of EEGNet models respectively; and after 5-fold cross-validation of each model, the average accuracy of each model on the test set is recorded, and the average pooling scale of the first layer corresponding to the model with the highest average accuracy is recorded.
6. The cross-subject passive automatic classification method for pitch perception based on EEG according to claim 2, wherein for single-subject optimization modeling, the average pooling scale of the second layer of the model is optimized as follows:the number and the scale of the convolution kernels in the first layer, the number of the convolution kernels in the second layer and the average pooling scale of the first layer of the EEGNet models are fixed to train a plurality of EEGNet models respectively; and after 5-fold cross-validation of each model, the average accuracy of each model on the test set is recorded, and the average pooling scale of the second layer corresponding to the model with the highest average accuracy is recorded.
7. A cross-subject passive automatic classification system for pitch perception based on EEG on the basis of deep learning, comprising:an EEG data collecting module configured to collect EEG data of each subject for pitch perception;an EEG data filtering module configured to filter all the collected EEG data by two filters, wherein one filter is a high-pass filter for removing DC components in EEG signals, and the other filter is a notch filter for removing power frequency interference;a model library building block configured to divide the EEG data collected from each subject into a training set and a test set, and determine optimal hyperparameters of modified EEGNet models based on the training set and the test set, wherein the data of N subjects are collected to obtain N modified EEGNet models, and the N EEGNet models constitute a model library for cross-subject training;wherein the modified EEGNet models are used for pitch perception decoding and predictive classification:the collected EEG data of each subject for pitch perception is specifically a CxT signal matrix, wherein C is the number of channels and T is the number of EEG signal sampling points;2024267092 02 Dec 2024inputted Cx T signals pass through a subchannel convolutional layer containing Ni convolution kernels capable of being regarded as time domain filters and having a scale of L1 in each channel, then pass through an average pooling layer with a pooling scale of N3 to reduce the feature dimension, subsequently pass through a convolutional layer with N2 convolution kernels of size (C, i) which can be regarded as space domain filters, pass through an average pooling layer with a pooling scale of N4 to further reduce the feature dimension, then pass through a subchannel convolutional layer with i convolution kernel in each channel, and then pass through a convolutional layer with N5 convolution kernels; the final outputted features are mapped through Softmax to obtain a posteriori probability of each class; and finally, a kind of labels with the highest probability is selected as final model output;a cross-subject modeling module configured to use part of data of the new subject as a calibration data set, and use the remaining part as a test set when cross-subject modeling is carried out on a new subject, wherein the calibration data set is used for classification prediction of all built models for single subjects in the model library and sorting the accuracy of the N modified EEGNet models under the calibration data set to select top M modified EEGNet models with superior classification performance to form a preferred model set, to obtain final classification prediction results by voting strategy based on the preferred model set.
8. A computer device, comprising a memory, a processor, and computer programs stored on the memory and run on the processor, wherein the steps of the method according to any one of claims 1-6 are implemented when the programs are executed by the processor.
9. A computer readable storage medium, storing computer programs, wherein the programs, when executed by a processor, execute the steps of the method according to any one of claims 1-6.