Music perception brain response analysis method and system based on residual network model
Through the music perception brain response analysis method based on the residual network model, the problems of insufficient feature extraction and low classification accuracy in EEG signal classification are solved, and efficient classification and feature extraction of brain response activity patterns under music stimulation are achieved, thereby improving the classification performance.
Patent Information
- Application Number
- CN202510606539.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-12
- Publication Date
- 2025-09-12
AI Technical Summary
Existing deep learning models have problems in EEG signal classification, such as high model complexity, overfitting, and poor generalization ability, which leads to insufficient feature extraction and low classification accuracy.
A music perception brain response analysis method based on the residual network model is adopted. The EEG signals are acquired through a multi-channel EEG acquisition device, and artifact interference removal, power spectral density analysis and brain network regional difference analysis are performed. The residual network model is combined for feature fusion and classification evaluation, taking advantage of its deep structure and residual connection.
It significantly improved the classification performance of EEG signals, achieved efficient classification and feature extraction of brain response activity patterns under music stimulation, and improved classification accuracy.
Smart Images
Figure CN120632612A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of brain-computer interface technology, and in particular to a method and system for analyzing brain responses to music perception based on a residual network model. Background Art
[0002] A brain-computer interface (BCI) is a technology that directly establishes a communication path between the brain and external devices, enabling device control or information transfer by decoding brain activity signals. The core of BCI technology lies in the acquisition, processing, and analysis of brain signals, and converting them into executable instructions or classification results. Depending on the signal acquisition method, BCI can be divided into two categories: invasive and non-invasive. Non-invasive BCI is based on electroencephalogram (EEG) signals collected from the scalp surface and is widely used due to its high safety and simple operation.
[0003] Music, as a non-invasive external stimulus, can significantly influence human brain activity and has shown broad application potential in areas such as emotion regulation, cognitive enhancement, and neurorehabilitation. Studies have shown that different types of musical stimulation (such as classical music, pop music, and environmental music) can activate different areas of the brain and induce specific neural oscillation patterns (such as alpha waves, beta waves, and gamma waves), thereby positively affecting cognitive functions such as emotion, memory, and attention. In recent years, with the development of BCI technology, the combination of music stimulation and EEG signal analysis has provided new technical means for music therapy and music intervention, making real-time monitoring and personalized intervention possible.
[0004] In recent years, deep learning techniques have been widely used in the field of biosignal processing, particularly in EEG (electroencephalogram) (EEG) analysis. Common deep learning models, such as convolutional neural networks (CNN), recurrent neural networks (RNN), and graph neural networks (GNN), have been widely used in EEG classification. While these methods have achieved some success in EEG signal classification and analysis, they still face challenges such as high model complexity, overfitting of deep networks, and poor generalization. Summary of the Invention
[0005] (1) Technical issues to be solved
[0006] In view of the above-mentioned shortcomings and deficiencies of the prior art, the present invention provides a method and system for analyzing brain responses to music perception based on a residual network model, aiming to achieve efficient classification and feature extraction of brain response activity patterns, so as to provide new technical means for music cognition research and brain-computer interface applications.
[0007] (2) Technical solution
[0008] In order to achieve the above objectives, the main technical solutions adopted by the present invention include:
[0009] In a first aspect, an embodiment of the present invention provides a method for analyzing brain responses to music perception based on a residual network model, comprising:
[0010] The EEG signals of the subjects under different music stimulations are acquired through a multi-channel EEG acquisition device, and the electrode signals belonging to the EEG signals are mapped to the preset brain network areas;
[0011] Based on the subject's brain state information, the EEG signal is processed to remove artifact interference fragments to obtain optimized EEG signals under different music stimuli;
[0012] The music stimulation signals were pre-classified based on their physical characteristics. The optimized EEG signals corresponding to each type of music stimulation were subjected to power spectrum density analysis and brain network regional difference analysis to obtain the maximum frequency band difference characteristics of the brain network regions under each type of music stimulation.
[0013] The maximum frequency band difference features are fused with the preset functional connectivity matrix and then input into the residual network model to perform brain response classification evaluation based on brain network areas to obtain the brain response activity patterns corresponding to different music stimuli.
[0014] Optionally, obtaining the EEG signals of the subject under different music stimulations by a multi-channel EEG acquisition device, and mapping the electrode signals belonging to the EEG signals to a preset brain network area includes:
[0015] The original EEG signals of the subjects under different music stimulations were collected through multi-channel EEG acquisition equipment;
[0016] Performing preprocessing including filtering, downsampling and detrending on the original EEG signal to obtain the preprocessed EEG signal;
[0017] According to the brain functional partitioning of the subjects, the electrode signals belonging to the EEG signals are mapped to the corresponding brain network areas, which include the default network, salience network, auditory network, frontoparietal network and sensorimotor network.
[0018] Optionally, collecting original EEG signals of the subject under different music stimulations by a multi-channel EEG acquisition device includes:
[0019] Collect electrode signals from the electrodes of a multi-channel EEG acquisition device under different music stimulations of the subjects;
[0020] The electrode signals were input into a notch filter and a bandpass filter for filtering to obtain the original EEG signals of the subjects under different music stimuli;
[0021] The filtering frequency range of the notch filter is 48 Hz to 52 Hz, and the filtering frequency range of the bandpass filter is 0.5 Hz to 200 Hz.
[0022] Optionally, the original EEG signal is preprocessed by filtering, downsampling, and detrending to obtain a preprocessed EEG signal, including:
[0023] The original EEG signal is filtered by a Butterworth bandpass filter to obtain a filtered EEG signal;
[0024] Downsampling the filtered EEG signal to reduce the sampling rate of the EEG signal to a target value;
[0025] The downsampled EEG signals are detrended to remove the linear trend components in the EEG signals.
[0026] Optionally, based on the subject's brain state information, the EEG signal is subjected to artifact interference segment removal processing to obtain optimized EEG signals under different music stimuli, including:
[0027] Each EEG signal was segmented using a non-overlapping rectangular time window with a time length of 0.5 s;
[0028] Based on visual inspection and automatic artifact detection algorithm, double artifact interference detection is performed on each signal segment in the EEG signal, and the EEG signal segments containing electrooculogram signals and electromyography signals are eliminated based on the detection results.
[0029] Optionally, the music stimulation signals are pre-classified based on their physical characteristics, and the optimized EEG signals corresponding to each type of music stimulation are subjected to power spectrum density analysis and brain network area difference analysis to obtain the maximum frequency band difference characteristics of the brain network areas under each type of music stimulation, including:
[0030] Based on the physical characteristics of the music stimulus signals, different music stimulus signals acting on the subjects are pre-classified;
[0031] Perform power spectrum density analysis on the optimized EEG signals corresponding to each type of music stimulation to obtain the energy characteristics of different frequency bands in the optimized EEG signals;
[0032] The energy characteristics of different frequency bands were analyzed between groups, and the maximum frequency band difference characteristics of brain network areas under various types of music stimulation were extracted based on the analysis results.
[0033] Among them, the frequency bands include: theta waves of 4Hz to 8Hz, alpha waves of 8Hz to 13Hz, beta waves of 13Hz to 30Hz, gamma waves of 30Hz to 48Hz, and high gamma waves of 52Hz to 100Hz.
[0034] Optionally, pre-classifying different music stimulation signals acting on the subject based on physical characteristics of the music stimulation signals includes:
[0035] Perform spectrum bandwidth analysis on music stimulus signals to obtain frequency distribution characteristics of different music stimulus signals;
[0036] Perform spectrum flatness analysis on music stimulus signals to obtain waveform characteristics of different music stimulus signals;
[0037] All music stimulus signals are pre-classified based on frequency distribution characteristics and waveform properties.
[0038] Optionally, before obtaining brain response activity patterns corresponding to different music stimuli, the method further includes: fusing the maximum frequency band difference feature with a preset functional connectivity matrix and inputting the resultant feature into a residual network model for brain response classification evaluation based on brain network regions.
[0039] Obtain data samples for training the residual network model and split the data samples into training data sets and test data sets;
[0040] Taking minimizing the cross entropy loss function as the optimization goal, the model parameters of the residual network model are iterated through the back propagation algorithm based on the training data set;
[0041] The test data set is input into the trained residual network model for classification evaluation, and the classification accuracy of the trained residual network model is obtained based on the brain response classification evaluation of the brain network area.
[0042] Optionally, the maximum frequency band difference feature is fused with a preset functional connectivity matrix and then input into a residual network model for brain response classification evaluation based on brain network regions. The brain response activity patterns corresponding to different music stimuli include:
[0043] The Pearson correlation coefficient was calculated for the electrode signals corresponding to the brain network areas to construct the functional connectivity matrix;
[0044] The maximum frequency band difference feature is fused with the preset functional connection matrix and then input into the residual block of the residual network model for spatial feature extraction to obtain a feature map;
[0045] Use the global average pooling layer of the residual network model to perform global average pooling on the feature map to obtain a feature vector of a set dimension;
[0046] The fully connected layer of the residual network model is used to map the feature vector to the classification label space to obtain brain response classification results based on brain network regions;
[0047] The output layer of the residual network model is used to score and evaluate the brain response classification results, and the brain response activity patterns corresponding to different music stimuli are determined based on the evaluation scores of each brain response classification.
[0048] In a second aspect, an embodiment of the present invention provides a music perception brain response analysis system based on a residual network model, comprising:
[0049] Multi-channel EEG acquisition equipment, including several electrodes;
[0050] The processor is connected to the multi-channel EEG acquisition device and is used to execute the above-mentioned steps of the music perception brain response analysis method based on the residual network model.
[0051] (3) Beneficial effects
[0052] The beneficial effects of the present invention are as follows: the present invention uses multiple signal processing methods to optimize the processing of EEG signals, and combines the residual network model of EEG signals as a classification model, making full use of its deep structure and residual connection advantages, and solving the problems of insufficient feature extraction and low classification accuracy in EEG signal classification by traditional methods. Secondly, the present invention fuses the maximum frequency band difference features with the functional connection matrix and uses them as the residual network model input, which enables the model to automatically learn the spatial features in EEG signals, significantly improving the classification performance. BRIEF DESCRIPTION OF THE DRAWINGS
[0053] Figure 1 A schematic diagram of a flow chart of a method for analyzing brain responses to music perception based on a residual network model according to an embodiment of the present invention;
[0054] Figure 2 A schematic diagram of the positions of 16-channel electrodes in a multi-channel EEG acquisition device provided in one embodiment of the present invention;
[0055] Figure 3 A schematic diagram of a residual network model provided in one embodiment of the present invention for extracting high-order features to adapt to EEG signal data classification in various scenarios. DETAILED DESCRIPTION
[0056] In order to better explain the present invention and facilitate understanding, the present invention is described in detail below through specific implementation methods in conjunction with the accompanying drawings.
[0057] refer to Figures 1 to 3As shown, an embodiment of the present invention proposes a music perception brain response analysis method based on a residual network model, which includes: obtaining the EEG signals of the subject under different music stimuli through a multi-channel EEG acquisition device, and mapping the electrode signals to which the EEG signals belong to a preset brain network area; based on the brain state information of the subject, performing artifact interference fragment removal processing on the EEG signals to obtain optimized EEG signals under different music stimuli; pre-classifying the music stimulation signals according to the physical characteristics of the music stimulation signals, performing power spectrum density analysis and brain network area difference analysis on the optimized EEG signals corresponding to each type of music stimulation, and obtaining the maximum frequency band difference characteristics of the brain network areas under various types of music stimulation; performing feature fusion of the maximum frequency band difference characteristics and the preset functional connection matrix, and then inputting the input into the residual network model to perform brain response classification evaluation based on the brain network area, and obtain the brain response activity patterns corresponding to different music stimuli.
[0058] This embodiment uses multiple signal processing methods to optimize the processing of EEG signals and combines the residual network model of EEG signals as a classification model, fully leveraging the advantages of its deep structure and residual connections to solve the problems of insufficient feature extraction and low classification accuracy in traditional EEG signal classification methods. Secondly, this embodiment fuses the maximum frequency band difference features with the functional connectivity matrix and uses them as the input of the residual network model, which enables the model to automatically learn the spatial features in the EEG signals, significantly improving classification performance.
[0059] To better understand the above technical solutions, exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments described herein. Rather, these embodiments are provided to enable a clearer and more thorough understanding of the present invention and to fully convey the scope of the present invention to those skilled in the art.
[0060] Specifically, refer to Figure 2 As shown, this embodiment provides a music perception brain response analysis method based on a residual network model, comprising the following steps S100 to S400:
[0061] S100. Obtain the EEG signals of the subject under different music stimulations through a multi-channel EEG acquisition device, and map the electrode signals belonging to the EEG signals to a preset brain network area.
[0062] In this example, subjects were selected based on the principles of good health, normal vision and motor skills, no brain damage or neurological disease, similar age, and similar work experience. For example, 33 subjects were selected for the experiment. They were students aged 18 to 24 years old, healthy, with normal vision and motor skills and no brain damage.
[0063] In this embodiment, step S100 may include the following sub-steps S110 to S130:
[0064] S110 collects the original EEG signals of the subjects under different music stimulations through multi-channel EEG acquisition equipment.
[0065] In this embodiment, step S110 may include the following sub-steps S111 to S112:
[0066] S111. Collect electrode signals from the electrode patch of a multi-channel EEG acquisition device under different music stimulations of the subject.
[0067] S112. Input the electrode signal into a notch filter and a bandpass filter for filtering processing to obtain the original EEG signal of the subject under different music stimulations.
[0068] The filtering frequency range of the notch filter is 48 Hz to 52 Hz, and the filtering frequency range of the bandpass filter is 0.5 Hz to 200 Hz.
[0069] In a specific embodiment, the multi-channel EEG acquisition device is a 16-channel EEG signal acquisition device, and the electrode placement standard is based on the 10 / 10 system of the International Federation of Clinical Neurophysiology. The 16-channel active electrodes are Fp1, Fp2, F5, Fz, F6, C3, C4, Cz, P5, P6, Pz, PO3, PO4, O1, O2, and Oz. The earlobe on one side is used as the standard electrode, and the forehead is used as the ground electrode. The specific arrangement positions of the electrodes are as follows: Figure 2 shown.
[0070] In preparation for EEG signal acquisition, the subject's hair is washed and blow-dried. The mid-axis meridian is measured to locate and mark the Cz point. The electrode cap is then aligned with the Cz lead and placed on the cap. Conductive paste is then injected using a syringe. The impedance is continuously monitored during injection to ensure good contact between the active electrode and the scalp through the conductive paste. The contact resistance should be kept below 30 kΩ to ensure a high signal-to-noise ratio (SNR) of the scalp EEG signal.
[0071] After the preparation phase, the subjects sat in a comfortable chair facing a computer screen, remained silent, and minimized blinking to minimize artifacts. At the start of the experiment, the subjects remained silent and eyes open for 1 minute. Active electrodes were used to collect EEG signals from the awake state at a sampling rate of 1200 Hz. A 20-second segment of quiet EEG signals was collected as rest data. The active electrodes then transmitted the EEG signals to a g.USBamp signal amplifier via an electrode box connected to the electrode wires for amplification. The amplified EEG signals were then transferred to a computer for storage, display, and analysis. A 48-52 Hz notch filter and a 0.5-200 Hz bandpass filter were used during EEG signal acquisition to eliminate power frequency interference and suppress noise. Each paradigm lasted 20 seconds. After each trial, the subjects were allowed a 1-minute rest period to adjust. The duration and intensity of the audio stimulation were kept constant across all music stimulation conditions. A total of 11 data segments were collected for each subject, consisting of data from 10 different music stimulations and a silent state.
[0072] S120 , performing preprocessing including filtering, downsampling, and detrending on the original EEG signal to obtain a preprocessed EEG signal.
[0073] Raw EEG signals often contain low-frequency drift (such as baseline drift caused by breathing or movement) and high-frequency noise (such as myoelectric interference or environmental electromagnetic interference). To remove this interference and retain neural activity information in the target frequency band, a Butterworth bandpass filter can be used to filter the raw EEG signals. Preprocessing of the acquired raw EEG signals, including filtering, downsampling, and detrending, improves signal quality and reduces noise interference.
[0074] In this embodiment, step S120 may include the following sub-steps S121 to S123:
[0075] S121. Filter the original EEG signal using a Butterworth bandpass filter to obtain a filtered EEG signal.
[0076] The raw EEG signals were filtered using a Butterworth bandpass filter with a filtering frequency range of 1 Hz to 100 Hz to remove low-frequency drift and high-frequency noise while retaining the neural activity information in the target frequency band.
[0077] S122 . Downsampling the filtered EEG signal to reduce the sampling rate of the EEG signal to a target value.
[0078] The filtered EEG signal is downsampled, reducing the sampling rate from 1200 Hz to 256 Hz to reduce the data volume and computational complexity.
[0079] S123. Perform detrending processing on the downsampled EEG signal to remove linear trend components in the EEG signal.
[0080] The downsampled EEG signal may still contain linear trend components caused by device drift or physiological activity, which can affect subsequent signal analysis and feature extraction. Therefore, the downsampled EEG signal is detrended using the detrend function in MATLAB to remove the linear trend components in the EEG signal and eliminate the interference of baseline drift on subsequent analysis.
[0081] S130. Mapping the electrode signals of the EEG signals to corresponding brain network areas according to the brain functional partitioning of the subject. The brain network areas include the default network, the salience network, the auditory network, the frontoparietal network, and the sensorimotor network.
[0082] In this embodiment, the 16-channel electrodes are divided into six networks according to the brain functional network, namely the default mode network (DMN), the salience network (SN), the visual network (VN), the auditory network (AN), the frontoparietal network (FPN) and the sensorimotor network (SMN).
[0083] S200: Based on the subject's brain state information, the EEG signal is processed to remove artifact interference fragments to obtain optimized EEG signals under different music stimuli.
[0084] In this embodiment, step S200 may include the following sub-steps S210 to S220:
[0085] S210 , segment each EEG signal using a non-overlapping rectangular time window with a time length of 0.5 s.
[0086] S220, performing double artifact interference detection on each signal segment in the EEG signal based on visual inspection and automatic artifact detection algorithm, and eliminating EEG signal segments containing electrooculogram signals and electromyography signals according to the detection results.
[0087] S300. Pre-classify the music stimulation signals according to their physical characteristics, perform power spectrum density analysis and brain network area difference analysis on the optimized EEG signals corresponding to each type of music stimulation, and obtain the maximum frequency band difference characteristics of the brain network areas under each type of music stimulation.
[0088] In this embodiment, power spectral density (PSD) analysis was performed on the EEG signals under each type of music stimulation to extract the energy characteristics of different frequency bands. The frequency bands and brain regions with statistically significant differences were screened out through inter-group significance difference analysis. The differences in brain network areas under different music stimulation groups were further analyzed to obtain the maximum frequency band difference characteristics of the brain network areas under various types of music stimulation.
[0089] In this embodiment, step S300 may include the following sub-steps S310 to S330:
[0090] S310 : Pre-classify different music stimulation signals acting on the subject based on the physical characteristics of the music stimulation signals.
[0091] The physical characteristics of music stimulation signals include spectral bandwidth and spectral flatness, which characterize the frequency distribution and waveform characteristics of music stimulation signals. Based on the physical characteristics of music, music stimulation signals can be divided into several groups, each corresponding to a different type of music stimulation.
[0092] In this embodiment, step S310 may include the following sub-steps S311 to S313:
[0093] S311. Perform spectrum bandwidth analysis on the music stimulation signal to obtain frequency distribution characteristics of different music stimulation signals.
[0094] For example, the bandwidth function in MATLAB is used to calculate the spectrum bandwidth. The spectrum bandwidth is used to describe the width of the frequency distribution of the music stimulus signal and reflects the frequency range of the signal.
[0095] S312. Perform spectrum flatness analysis on the music stimulation signal to obtain waveform characteristics of different music stimulation signals.
[0096] For example, the spectralFlatness function in MATLAB is used to calculate spectral flatness. Spectral flatness is used to describe the waveform characteristics of the music stimulus signal and reflects the flatness of the signal in the frequency domain.
[0097] S313. Pre-classify all music stimulation signals based on frequency distribution characteristics and waveform characteristics.
[0098] For example, the spectral bandwidth and spectral flatness characteristics of the above 10 different music stimulation signals are extracted, and the physical characteristics of the music signals are characterized from the two dimensions of frequency distribution and waveform characteristics. The 10 different music used in the experiment are divided into 8 groups.
[0099] S320. Perform power spectrum density analysis on the optimized EEG signal corresponding to each type of music stimulation to obtain energy characteristics of different frequency bands in the optimized EEG signal.
[0100] The Welch (signal processing and statistical analysis) method was used to calculate the power spectral density of the EEG signal to characterize the energy distribution of the signal in different frequency bands. For example, the pwelch function in MATLAB was used to extract the energy characteristics of different frequency bands (θ waves: 4Hz-8Hz, α waves: 8Hz-13Hz, β waves: 13Hz-30Hz, γ waves: 30Hz-48Hz, and high γ waves: 52Hz-100Hz).
[0101] S330. Conduct inter-group difference analysis on the energy characteristics of different frequency bands, and extract the maximum frequency band difference characteristics of brain network areas under various types of music stimulation based on the analysis results.
[0102] The extracted PSD energy features were analyzed for significant differences between groups. Using nonparametric tests, a normality test revealed that the data did not meet the normal distribution assumption. For example, the Kruskal-Wallis test in SPSS software was used for inter-group comparisons. Based on the results of the significant difference analysis, brain network regions that showed significant differences under various musical stimuli were extracted, and the maximum frequency band differences between brain network regions under various musical stimuli were then obtained.
[0103] S400 fuses the maximum frequency band difference features with the preset functional connection matrix and inputs them into the residual network model to perform brain response classification evaluation based on brain network areas to obtain brain response activity patterns corresponding to different music stimuli.
[0104] In this embodiment, the deep learning model is a residual connection (ResNet), refer to Figure 3 As shown in the figure, its structure includes input layer, convolution layer, residual block, global average pooling layer, fully connected layer and output layer; the input layer takes the functional connection matrix as input, and the matrix size is N×N×1 (N is the number of electrodes); the convolution layer is used to extract the spatial features of the functional connection matrix; the residual block consists of two convolution layers and a skip connection, which is used to solve the gradient disappearance problem in deep networks; the global average pooling layer reduces the dimension of the feature map output by the convolution layer to generate a fixed-length feature vector; the fully connected layer maps the feature vector to the classification label space; the output layer uses the Softmax function to calculate the probability of each category.
[0105] In this embodiment, before executing step S400, the following steps F100 to F300 are also included:
[0106] F100. Obtain data samples for training the residual network model and split the data samples into a training data set and a test data set.
[0107] F200, with the minimization of the cross entropy loss function as the optimization goal, uses the back propagation algorithm to iterate the model parameters of the residual network model based on the training data set.
[0108] F300: Input the test data set into the trained residual network model for classification evaluation. Based on the brain response classification evaluation of the brain network area, the classification accuracy of the trained residual network model is obtained. The mathematical expression of the residual network model evaluation is:
[0109]
[0110] In formula (1), y i is the true label, is the probability predicted by the model, is the cross entropy loss, and C is the total number of music stimulus categories.
[0111] In this embodiment, step S400 may include the following sub-steps S410 to S450:
[0112] S410. Calculate the Pearson correlation coefficient of the electrode signals corresponding to the brain network areas to construct a functional connectivity matrix.
[0113] The functional connectivity matrix of electrode data corresponding to different brain network regions was used as input to a residual network model, which was then used to classify brain activity patterns in response to different musical stimuli. For each brain network region, the Pearson Correlation Coefficient (PCC) matrix of the corresponding electrode signals was calculated to characterize the strength of functional connectivity between electrodes. By calculating the PCC values for all electrode pairs, a symmetric functional connectivity matrix of size N × N × 1 (where N is the number of electrodes in each brain network region) was constructed as input to the residual network model.
[0114] S420, performing feature fusion on the maximum frequency band difference feature and the preset functional connection matrix, and then inputting the result into the residual block of the residual network model to extract spatial features and obtain a feature map.
[0115] The residual network model's residual part consists of multiple residual blocks, each of which consists of two convolutional layers and a skip connection. Multiple convolutional layers are used to extract the spatial features of the functional connectivity matrix that incorporates the maximum frequency band difference features. Each convolutional layer is followed by batch normalization and a ReLU activation function. The mathematical expression of the residual part is:
[0116]
[0117] In formula (2), y is the output, is the residual function and x is the input.
[0118] S430. Perform global average pooling on the feature map using the global average pooling layer of the residual network model to obtain a feature vector of a set dimension.
[0119] S440. Use the fully connected layer of the residual network model to map the feature vector to the classification label space to obtain a brain response classification result based on the brain network area.
[0120] S450. Use the output layer of the residual network model to score and evaluate the brain response classification results, and determine the brain response activity patterns corresponding to different music stimuli based on the evaluation scores of each brain response classification.
[0121] The output layer uses the Softmax function to calculate the score of each music stimulus category. The mathematical expression of the Softmax function is:
[0122]
[0123] In formula (3), z i is the score of the i-th music stimulus category, C is the total number of music stimulus categories, z j is the score for all categories {j∈{1,2,...,C}.
[0124] In one specific embodiment, based on the comprehensive judgment of the results in Table 1, it was concluded that the best classification results were achieved when the five network features other than the auditory network (AN) were used for the nine-category classification. As shown in Table 1, the classification accuracy of the control group was 72.9% when the whole-brain input was used. After excluding the AN network, the five-network input increased the classification accuracy of the control group to 75.0%, demonstrating that this embodiment can effectively extract music cognition-related features through the functional network optimization strategy. Table 2 shows the classification results of further removing the default mode network (DMN) or the salience network (SN) based on the five-network. When the DMN was removed from the five-network, the accuracy of the experimental group decreased significantly by 6.0%, compared to only a 2.2% decrease when the SN was removed (p = 0.038), indicating that the default mode network plays an irreplaceable role as a hub for higher-order musical cognitive processing.
[0125] Table 1. Classification results of whole-brain and five-network inputs
[0126]
[0127] Table 2. Classification results after removing key networks (DMN / SN)
[0128]
[0129] In addition, this embodiment also provides a music perception brain response analysis system based on a residual network model, including:
[0130] Multi-channel EEG acquisition equipment, including several electrode sheets.
[0131] The processor is connected to the multi-channel EEG acquisition device and is used to execute the above-mentioned steps of the music perception brain response analysis method based on the residual network model.
[0132] In summary, the present embodiment discloses a method and system for analyzing brain responses to music perception based on a residual network model. The residual network model is used as a classification model, and the advantages of its deep structure and residual connection are fully utilized to solve the problems of insufficient feature extraction and low classification accuracy in EEG signal classification of traditional methods. By taking the brain network functional connection matrix (such as the PCC matrix) as the model input, the residual network model can automatically learn the spatial features in the EEG signal, significantly improving the classification performance. The present invention also combines brain network areas with music stimulation EEG signal analysis, conducts in-depth research on the specificity of brain network activity under music stimulation, and extracts power spectral density features and functional connection features of different frequency bands, combined with the classification ability of the residual network model, to achieve high-precision recognition and classification of brain network activity patterns under music stimulation. Therefore, the present invention achieves efficient classification and feature extraction of brain network activity patterns by applying the residual network model to music stimulation EEG signal analysis, providing a new technical means for the development of music cognition research and brain-computer interface technology.
[0133] Since the systems / devices described in the above embodiments of the present invention are systems / devices used to implement the methods of the above embodiments of the present invention, those skilled in the art will be able to understand the specific structures and variations of these systems / devices based on the methods described in the above embodiments of the present invention, and thus will not be described in detail here. All systems / devices used in the methods of the above embodiments of the present invention are within the scope of protection of the present invention.
[0134] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0135] The present invention is described with reference to flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions.
[0136] It should be noted that, in the description of the present invention, the word "a" or "an" preceding a component does not exclude the presence of a plurality of such components. The present invention can be implemented by means of hardware comprising several distinct components and by means of a suitably programmed computer. The use of the words first, second, third, etc., is merely for convenience and does not imply any order. These words should be understood as part of the component name.
[0137] In addition, it should be noted that, in the description of this specification, the description of the terms "one embodiment", "some embodiments", "embodiment", "example", "specific example" or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and the features of different embodiments or examples, unless they are contradictory.
[0138] Although preferred embodiments of the present invention have been described, those skilled in the art will be able to make additional changes and modifications to these embodiments after obtaining the basic inventive concepts.
[0139] Obviously, those skilled in the art can make various modifications and variations to the present invention without departing from the spirit and scope of the invention.
Claims
1. A method for analyzing brain responses to music perception based on a residual network model, characterized in that: include: The EEG signals of the subjects under different music stimulations are acquired through a multi-channel EEG acquisition device, and the electrode signals belonging to the EEG signals are mapped to the preset brain network areas; Based on the subject's brain state information, the EEG signal is processed to remove artifact interference fragments to obtain optimized EEG signals under different music stimuli; The music stimulation signals were pre-classified based on their physical characteristics. The optimized EEG signals corresponding to each type of music stimulation were subjected to power spectrum density analysis and brain network regional difference analysis to obtain the maximum frequency band difference characteristics of the brain network regions under each type of music stimulation. The maximum frequency band difference features are fused with the preset functional connectivity matrix and then input into the residual network model to perform brain response classification evaluation based on brain network areas to obtain the brain response activity patterns corresponding to different music stimuli.
2. The method according to claim 1, wherein The EEG signals of the subjects under different music stimulations were acquired through multi-channel EEG acquisition equipment, and the electrode signals belonging to the EEG signals were mapped to the preset brain network areas including: The original EEG signals of the subjects under different music stimulations were collected through multi-channel EEG acquisition equipment; Performing preprocessing including filtering, downsampling and detrending on the original EEG signal to obtain the preprocessed EEG signal; According to the brain functional partitioning of the subjects, the electrode signals belonging to the EEG signals are mapped to the corresponding brain network areas, which include the default network, salience network, auditory network, frontoparietal network and sensorimotor network.
3. The method according to claim 2, wherein The original EEG signals of the subjects under different music stimulations are collected through multi-channel EEG acquisition equipment, including: Collect electrode signals from the electrodes of a multi-channel EEG acquisition device under different music stimulations of the subjects; The electrode signals were input into the notch filter and the bandpass filter for filtering to obtain the original EEG signals of the subjects under different music stimuli; The filtering frequency range of the notch filter is 48 Hz to 52 Hz, and the filtering frequency range of the bandpass filter is 0.5 Hz to 200 Hz.
4. The method according to claim 2, wherein The original EEG signal is preprocessed by filtering, downsampling and detrending to obtain the following preprocessed EEG signal: The original EEG signal is filtered by a Butterworth bandpass filter to obtain a filtered EEG signal; Downsampling the filtered EEG signal to reduce the sampling rate of the EEG signal to a target value; The downsampled EEG signals are detrended to remove the linear trend components in the EEG signals.
5. The method according to claim 1, wherein Based on the subject's brain state information, the EEG signal is processed to remove artifact interference fragments, and the optimized EEG signals under different music stimuli are obtained, including: Each EEG signal was segmented using a non-overlapping rectangular time window with a time length of 0.5 s; Based on visual inspection and automatic artifact detection algorithm, double artifact interference detection is performed on each signal segment in the EEG signal, and the EEG signal segments containing electrooculogram signals and electromyography signals are eliminated based on the detection results.
6. The method according to claim 1, wherein The music stimulation signals were pre-classified based on their physical characteristics. The optimized EEG signals corresponding to each type of music stimulation were subjected to power spectrum density analysis and brain network regional difference analysis. The maximum frequency band difference characteristics of the brain network regions under various types of music stimulation were obtained, including: Based on the physical characteristics of the music stimulus signals, different music stimulus signals acting on the subjects are pre-classified; Perform power spectrum density analysis on the optimized EEG signals corresponding to each type of music stimulation to obtain the energy characteristics of different frequency bands in the optimized EEG signals; The energy characteristics of different frequency bands were analyzed between groups, and the maximum frequency band difference characteristics of brain network areas under various types of music stimulation were extracted based on the analysis results; Among them, the frequency bands include: theta waves of 4Hz to 8Hz, alpha waves of 8Hz to 13Hz, beta waves of 13Hz to 30Hz, gamma waves of 30Hz to 48Hz, and high gamma waves of 52Hz to 100Hz.
7. The method according to claim 6, wherein Based on the physical characteristics of the music stimulus signals, the pre-classification of different music stimulus signals acting on the subjects includes: Perform spectrum bandwidth analysis on music stimulus signals to obtain frequency distribution characteristics of different music stimulus signals; Perform spectrum flatness analysis on music stimulus signals to obtain waveform characteristics of different music stimulus signals; All music stimulus signals are pre-classified based on frequency distribution characteristics and waveform properties.
8. The method according to claim 1, wherein After the maximum frequency band difference feature is fused with the preset functional connectivity matrix, it is input into the residual network model to perform brain response classification evaluation based on brain network areas. Before obtaining the brain response activity patterns corresponding to different music stimuli, it also includes: Obtain data samples for training the residual network model and split the data samples into training data sets and test data sets; Taking minimizing the cross entropy loss function as the optimization goal, the model parameters of the residual network model are iterated through the back propagation algorithm based on the training data set; The test data set is input into the trained residual network model for classification evaluation, and the classification accuracy of the trained residual network model is obtained based on the brain response classification evaluation of the brain network area.
9. The method according to claim 1, wherein The maximum frequency band difference feature is fused with the preset functional connectivity matrix and then input into the residual network model to perform brain response classification evaluation based on brain network areas. The brain response activity patterns corresponding to different music stimuli include: The Pearson correlation coefficient was calculated for the electrode signals corresponding to the brain network areas to construct the functional connectivity matrix; The maximum frequency band difference feature is fused with the preset functional connection matrix and then input into the residual block of the residual network model for spatial feature extraction to obtain a feature map; Use the global average pooling layer of the residual network model to perform global average pooling on the feature map to obtain a feature vector of a set dimension; The fully connected layer of the residual network model is used to map the feature vector to the classification label space to obtain brain response classification results based on brain network regions; The output layer of the residual network model is used to score and evaluate the brain response classification results, and the brain response activity patterns corresponding to different music stimuli are determined based on the evaluation scores of each brain response classification.
10. A music perception brain response analysis system based on a residual network model, characterized in that: include: Multi-channel EEG acquisition equipment, including several electrodes; A processor is connected to a multi-channel EEG acquisition device and is used to execute the steps of the music perception brain response analysis method based on the residual network model as described in any one of claims 1 to 9.