Method and device for constructing model of interval consonance perception and electroencephalogram response, and storage medium

Abstract

The application discloses a method for constructing a model for associating interval harmony perception and electroencephalogram response, which comprises the following steps: preparing audio clips including various intervals as audio samples; selecting two categories of experimental subjects; the subjects wear earphones to score the interval harmony degree of the audio clips in the audio sample set, and the interval harmony degree scores of the subjects corresponding to different audio clips are counted; the subjects wear earphones, and based on the Oddball paradigm, the audio clips in the audio sample set are used for multi-round interval stimulation electroencephalogram experiment to collect the electroencephalogram data of the subjects corresponding to different audio clips, the interval classification and arrangement of the electroencephalogram data are carried out based on the EEGLAB module of MATLAB, and difference test is carried out; based on the same audio clip, the interval harmony degree sorting and classification are associated with the corresponding electroencephalogram data; and the model for associating interval harmony perception and electroencephalogram response is constructed. The application provides a quantitative means for the neural response sensitivity under interval stimulation.

Description

Technical Field

[0001] This invention relates to a method for auditory perception of pitch intervals, and more particularly to a method, device, and storage medium for constructing a correlation model between pitch interval consonance perception and electroencephalogram (EEG) response. Background Technology

[0002] Currently, in the development of tonal music, the relationship between notes is the most core tonal characteristic, and the interval, as the smallest unit of the relational feature between notes, is the foundation for the formation of tonal characteristics. From an acoustic physics perspective, an interval is formed by the continuous or simultaneous vertical and horizontal superposition of two single notes of the same or different frequencies. It not only has stable physical properties but can also form complex tonal characteristic structures and induce auditory responses, possessing multidimensional perceptual attributes.

[0003] In musicology, common tuning systems include Pythagorean tuning, just intonation, and twelve-tone equal temperament. However, due to the current popularity of electronic and pop music, twelve-tone equal temperament has a growing influence on contemporary audiences. Twelve-tone equal temperament refers to the twelve intervals within an octave, with each interval divided into 12 equal parts, and a minimum frequency ratio of 2. 112 And it increases in integer ratios, with the smallest perfect second frequency ratio among the twelve intervals being 2. 112 :1, the frequency ratio of the maximum pure octave is 2:1. Based on the twelve equal temperament, this invention produces 12 audio groups within the octave, which are defined as minor second (m2), major second (M2), minor third (m3), major third (M3), perfect fourth (P4), augmented fourth (TT), perfect fifth (P5), minor sixth (m6), major sixth (M6), minor seventh (m7), major seventh (M7), and perfect octave (Oct) [1].

[0004] With the development of tonal music, the characteristics of intervals in auditory perception as "harmonious" and "dissonant" have been gradually discovered. The physical explanations related to the harmonicity of intervals mainly include two aspects: critical band and periodicity. The critical band can be regarded as a bandpass filter. The frequency difference between two notes gradually increases from 0, and the auditory perception will gradually change from beat, instability to stability, thus giving rise to the auditory perception of harmony and dissonance [2]. Periodicity is reflected in the repetitive waveform in the time domain and the harmonic components in the frequency domain. Harmonious intervals have simple integer frequency ratios. For example, in a pure octave, the ratio of the fundamental frequency to the crown frequency is 1:2. Based on just intonation, it gives people a pleasant feeling. In contrast, dissonant intervals, such as a major second, have a ratio of the fundamental frequency to the crown frequency of 8:9. Based on just intonation, they give people a rough feeling.

[0005] According to the harmonic and inharmonic characteristics of interval, some scholars proposed the theories of "three classification", "four classification", "six classification" and the like [3], and the evaluation of the degree of interval harmony is not consistent at present, among which, the theory of "three classification" is as follows: (1) extremely complete harmonic interval: Oct; (2) complete and incomplete harmonic interval: P5, P4, m3, M3, m6, M6; (3) inharmonic interval: m2, M2, m7, M7, TT, and the present application simultaneously investigates the difference in the degree of harmony perception of twelve intervals and interval three classification.

[0006] At present, most of the researches on the perception of the degree of interval harmony are focused on subjective feeling, and the research objects mainly include infants of 2-6 months, ordinary people, musicians, single-sided deaf patients and animals and the like. For example, in the early stage of human life, the infants have observed the perception deviation of the degree of interval harmony, and the newborns are more inclined to harmonic music [4], in the animal kingdom, sparrows can distinguish between harmonic interval and inharmonic interval, but pigeons and mice cannot distinguish [5][6], young chimpanzees prefer harmonic pitch relationship [7], but it is not embodied on monkeys [8]. In addition, the harmonic judgment of western listeners and Japanese listeners is similar [9], but there is difference in the judgment of Indians

[10] . They have very similar average harmonic scores on the 12 pitch heights within an octave, and think that the harmonic interval is more attractive, but the existing evaluation method is based on subjective evaluation, and the subjective scoring has great uncertainty, and it is also necessary to construct an evaluation method capable of realizing the objective evaluation of the perception of the degree of interval harmony. SUMMARY

[0007] The present application provides a method for constructing a model for correlating the perception of the degree of interval harmony with the brain electrical response, a device and a storage medium to solve the technical problems in the prior art.

[0008] The technical solution adopted by the present application to solve the technical problems in the prior art is as follows:

[0009] A method for constructing a model for correlating the perception of the degree of interval harmony with the brain electrical response, comprising the following steps:

[0010] Step 1, preparing audio clips including various intervals as audio samples, wherein the occurrence probability of pure octave interval is 70%, and the occurrence probability of the remaining intervals is 30%; the time length of each audio clip is 500-600 ms; the audio samples are divided into two groups, namely a high-frequency sample group and a low-frequency sample group; the audio types in each group of audio samples are 12; the root audio frequency of the low-frequency sample group is unified as 440.01 Hz, and the root audio frequency of the high-frequency sample group is unified as 523.264 Hz;

[0011] Step 2, according to the music learning experience, two categories of experimental subjects are selected: subjects who have learned music for a long time and subjects who have learned music for a short time; the ratio of men to women in each category is 1:1; the selected subjects have no hearing impairment, and all subjects are right-handed;

[0012] Step 3, interval harmony perception experiment: the subject wears a headset and scores the interval harmony degree of the audio clips in the audio sample set; the interval harmony degree scores of each subject corresponding to different audio clips are counted; the audio clips are sorted and classified according to the interval harmony degree scores, and difference test is performed;

[0013] Step 4, interval stimulation EEG response experiment: the subject wears a headset, based on the Oddball paradigm, uses audio clips in the audio sample set to conduct multiple rounds of interval stimulation EEG experiment; each round includes multiple audio stimuli; each audio clip is used once; the interval between two rounds of audio stimuli is a period of time; EEG data of each subject corresponding to different audio clips are collected, EEG data are sorted and classified based on the EEGLAB module of MATLAB, and difference test is performed;

[0014] Step 5, based on the same audio clip, the interval harmony degree sorting and classification is associated with the corresponding EEG data; an interval harmony perception and EEG response correlation model is constructed.

[0015] Further, in step 3, the method for scoring the interval harmony degree of the audio clip includes the following steps:

[0016] Step A1, design a questionnaire to collect the music learning experience, gender, music preference of the subject, and the harmony degree score of the audio clip in the interval harmony perception experiment; wherein the harmony degree score of the audio clip can be changed;

[0017] Step A2, screen the filled valid questionnaires to remove the questionnaires not filled according to the requirements; count the interval harmony perception experiment data;

[0018] Step A3, according to the statistical data, measure and analyze the psychological perception dimension of interval harmony, classify the data according to the interval harmony theory classification mode, including three classification and four classification, and perform difference test on the data.

[0019] Further, in step 4, the method for conducting interval stimulation EEG experiment includes the following steps:

[0020] Step B1, use Curry7 software to record the brain response before and after interval stimulation, and pre-process the EEG ERP data under interval stimulation, including positioning, filtering, removing bad segments, re-referencing, running ICA, and removing artifacts for multiple electrodes used;

[0021] Step B2, according to the three classification method in the theory of interval harmony, convert the data label in the EEG data under the original twelve interval stimuli, and save the data again;

[0022] Step B3, according to the observation of time domain waveform under interval stimulus, find the significant component of brain electrical response induced by interval stimulus;

[0023] Step B4, select the ERP feature of the central brain area induced by interval harmony degree, average the ERP response component data of the central electrode under multiple interval stimuli, and perform difference test on the data.

[0024] Further, in steps 3 and 4, the difference test method includes: normal distribution test, Mann-Whitney U test and independent sample T test on the collected data of interval harmony degree perception experiment and interval stimulus brain electrical response experiment.

[0025] Further, the Mann-Whitney U test method includes: grouping the collected data according to the following categories: each two intervals of interval three classification, no music learning experience subjects and music learning experience subjects, low frequency stimulus and high frequency stimulus; Mann-Whitney U test is used for each group of comparison data, and whether there is significant difference between each group of comparison data is judged according to the test result.

[0026] Further, the independent sample T test method includes: grouping the collected data according to the following categories: no music learning experience subjects and music learning experience subjects, low frequency stimulus and high frequency stimulus; independent sample T test is used for each group of comparison data.

[0027] Further, X is the data collected in the interval harmony degree perception experiment; Y is the data collected in the interval stimulus brain electrical response experiment;

[0028] The grouping categories of X and Y include: no music learning experience subjects and music learning experience subjects, low frequency sample group and high frequency sample group; the data in X and Y are grouped into J group data individually or mixedly according to the grouping categories;

[0029] Each group data of Y is further divided into multiple unit data, and the unit classification categories include: N400 and LPC, amplitude and latency; each group data in Y is divided into L unit data individually or mixedly according to the unit classification categories;

[0030] Each group of X includes 12 data; each group of data is obtained under the stimulation of twelve interval audio samples, and one data corresponds to one interval;

[0031] Each unit of Y includes 12 data; each unit data is obtained under the audio sample stimulation of twelve intervals, and one data corresponds to one interval;

[0032] Pearson correlation analysis method is used to calculate the correlation between the data collected in the interval harmony perception experiment and the data collected in the interval stimulation electroencephalogram response experiment, and the calculation formula is as follows:

[0033]

[0034] In the formula:

[0035] i is the interval type number; i = 1, 2, … 12;

[0036] J is the grouping group number of X and Y, j = 1, 2, … J;

[0037] L is the unit number of the jth group in Y; l = 1, 2, … L;

[0038] x ji is the ith data of the jth group in X;

[0039] is the average value of the data of the jth group in X;

[0040] y jli is the ith data of the lth unit of the jth group in Y;

[0041] is the average value of the lth unit of the jth group in Y;

[0042] r jl is the correlation between the jth group data of the interval harmony perception experiment and the lth unit data of the jth group of the electroencephalogram response experiment.

[0043] Further, based on the correlation data between the data collected in the interval harmony perception experiment and the data collected in the interval stimulation electroencephalogram response experiment, a correlation model of interval harmony perception and electroencephalogram response is constructed by using a correlation heat map or a two-dimensional coordinate graph.

[0044] The application also provides a device for constructing a correlation model of interval harmony perception and electroencephalogram response, which comprises a memory and a processor, wherein the memory is used to store a computer program; and the processor is used to execute the computer program and realize the steps of the method for constructing a correlation model of interval harmony perception and electroencephalogram response as described above.

[0045] The application also provides a storage medium, wherein the storage medium stores a computer program, and the computer program is executed by a processor to realize the steps of the method for constructing a correlation model of interval harmony perception and electroencephalogram response as described above.

[0046] The present application has the advantages and positive effects that:

[0047] 1. The present application provides a quantitative means for the sensitivity of neural response under interval stimulation by constructing an interval auditory evaluation system.

[0048] 2. The present application realizes the extraction of the brain neural response under interval stimulation by analyzing the electroencephalogram data based on the interval consonance perception degree.

[0049] 3. The present application establishes an objective classification method of interval consonance degree with biomarker characteristics by constructing a psychological perception and electroencephalogram response correlation model of interval consonance degree.

[0050] The present application is based on the acoustic physical properties of interval, and obtains the subjective perception characteristics under different interval stimulation through interval consonance perception experiments, and combines the objective representation data of brain neural response under interval stimulation to jointly construct a correlation model based on interval consonance perception and electroencephalogram response. The present application selects subjects with music learning experience and subjects without music learning experience, and selects two groups of audio (low frequency sample group and high frequency sample group, each group has 12 audios) to carry out experiments, and finally constructs a correlation model between the subjective ranking of interval consonance degree by different people and the electroencephalogram component index. The present application constructs an optimal classification method of interval consonance degree based on the model. BRIEF DESCRIPTION OF DRAWINGS

[0051] Figure 1 FIG. 1 is a flowchart of a method for constructing a correlation model of interval consonance perception and electroencephalogram response according to the present application.

[0052] Figure 2 FIG. 2 is a schematic diagram of interval consonance rating in an interval consonance perception experiment according to the present application.

[0053] Figure 3 FIG. 3 is a schematic diagram of interval three-class consonance rating in an interval consonance perception experiment according to the present application.

[0054] Figure 4 FIG. 4 is a data analysis diagram of an electroencephalogram response experiment under interval stimulation according to the present application.

[0055] Figure 5 FIG. 5 is a data analysis diagram of the N400 component corresponding to the interval three-class in an electroencephalogram response experiment under interval stimulation according to the present application.

[0056] Figure 6 FIG. 6 is a data analysis diagram of the LPC component corresponding to the interval three-class in an electroencephalogram response experiment under interval stimulation according to the present application.

[0057] Figure 7 FIG. 7 is a correlation model in the form of a correlation heat map according to the present application.

[0058] Figure 8is a low-frequency sample group, a harmonic score of a music learning experienced subject and a N400 amplitude correlation graph of the present application.

[0059] Figure 9 is a low-frequency sample group, a harmonic score of a music learning experienced subject and a LPC amplitude correlation graph of the present application.

[0060] In the figure:

[0061] The harmonic degree perception experiment refers to an interval harmonic degree perception experiment score.

[0062] The EEG experiment refers to an interval stimulation EEG response experiment.

[0063] m2 represents a small second; M2 represents a large second; m3 represents a small third; M3 represents a large third; m6 represents a small sixth; M6 represents a large sixth; m7 represents a small seventh; M7 represents a large seventh; P4 represents a pure fourth; P5 represents a pure fifth; TT represents an augmented fourth; Oct represents a pure octave; Ext represents an extremely complete harmonic interval; dis represents an inharmonic interval; P&I represents a complete and incomplete harmonic interval.

[0064] r represents a correlation coefficient of an interval harmonic degree perception score and EEG experiment response data; p represents a significant correlation degree of an interval harmonic degree perception score and EEG experiment response data.

[0065] * represents a difference between two groups of data; wherein: one * represents that there is a difference between the data, two * represents that there is a significant difference between the data, three * represents that there is a particularly significant difference between the data, and four * represents that the significant difference between the data is the highest. DETAILED DESCRIPTION

[0066] The application will be described in detail below with reference to the accompanying drawings and embodiments, and it should be understood that the preferred embodiments described herein are only used to illustrate and explain the application, and are not used to limit the application.

[0067] The following English words, phrases and abbreviations are explained in Chinese as follows:

[0068] EEG: electroencephalogram.

[0069] ERP: event-related potential.

[0070] Oddball: a commonly used paradigm in psychology and neuroelectrophysiology.

[0071] MATLAB: data processing software.

[0072] EEGLAB: electroencephalogram signal processing.

[0073] ICA: independent component analysis.

[0074] Curry7: Electroencephalogram signal collection software.

[0075] SPSS: Data analysis software.

[0076] LPC: A positive wave about 300-800 ms after the start of stimulation.

[0077] Mann-Whitney U test: Mann-Whitney U test.

[0078] K-S test: Kolmogorov-Smirnov test.

[0079] S-W test: Shapiro-Wilk test.

[0080] See Figures 1 to 9 A method for constructing a model for associating interval harmony perception and electroencephalogram response, comprising the following steps:

[0081] Step 1: Prepare audio clips including various intervals as audio samples, wherein the probability of occurrence of pure octave interval is 70%, and the probability of occurrence of the remaining intervals is 30%; the duration of each audio clip is 500-600 ms; the audio samples are divided into two groups, namely a high-frequency sample group and a low-frequency sample group; the audio types in each group of audio samples are 12; the root audio frequency of the low-frequency sample group is unified as 440.01 Hz, and the root audio frequency of the high-frequency sample group is unified as 523.264 Hz.

[0082] Step 2: According to music learning experience, two categories of experimental subjects are selected: subjects who have learned music for a long time and subjects who have learned music for a short time; the male-to-female ratio of each category of subjects is 1:1; the selected subjects have no hearing impairment, and all the subjects are right-handed.

[0083] Step 3: Interval harmony perception experiment: the subject wears earphones, scores the interval harmony degree of the audio clips in the audio sample set, and statistically analyzes the interval harmony degree scores of the corresponding different audio clips of each subject; the audio clips are sorted and classified according to the interval harmony degree scores, and difference test is performed.

[0084] Step 4: Interval stimulation electroencephalogram response experiment: the subject wears earphones, and based on the Oddball paradigm, uses the audio clips in the audio sample set to conduct multiple rounds of interval stimulation electroencephalogram experiment; each round includes multiple audio stimuli; each audio clip is used once; the interval between two rounds of audio stimuli is a period of time; the electroencephalogram data of each subject corresponding to different audio clips are collected, the interval classification and arrangement of the electroencephalogram data are performed based on the EEGLAB module of MATLAB, and difference test is performed.

[0085] Step 5, based on the same audio segment, the degree of harmony of the audio segment is sorted and classified, and the corresponding electroencephalogram data is associated; and a model of the degree of harmony perception and the electroencephalogram response is constructed.

[0086] Preferably, in step 3, the method for scoring the degree of interval harmony of the audio segment can include the following steps:

[0087] Step A1, a questionnaire is designed to collect the music learning experience, gender, music preference degree and the score of the degree of harmony of the audio segment in the interval harmony perception experiment of the subject; wherein the score of the degree of harmony of the audio segment can be changed.

[0088] Step A2, the filled valid questionnaires are screened to remove the questionnaires not filled according to the requirements; and the interval harmony perception experiment data is counted.

[0089] Step A3, according to the statistical data, the psychological perception dimension of the interval harmony is measured and analyzed, the data is classified according to the classification mode of the interval harmony theory, including three classifications and four classifications, and the difference of the data is tested.

[0090] Preferably, in step 4, the method for performing the interval stimulation electroencephalogram experiment can include the following steps:

[0091] Step B1, the brain response before and after the interval stimulation is recorded by using Curry7 software, the electroencephalogram ERP data under the interval stimulation is preprocessed, including positioning, filtering, removing bad segments, re-referencing, running ICA and removing artifacts for the multiple electrodes used.

[0092] Step B2, according to the three classification method in the interval harmony theory, the data label in the original EEG data under the twelve interval stimulation is converted, and the data is saved again.

[0093] Step B3, according to the observed time domain waveform under the interval stimulation, the brain electrical response components induced by the interval stimulation are found.

[0094] Step B4, the central brain area ERP feature with the most prominent electroencephalogram response induced by the degree of interval harmony is selected, the ERP response component data induced by the central electrode under multiple interval stimulations is averaged, and the difference of the data is tested.

[0095] Preferably, in steps 3 and 4, the method for performing the difference test can include: performing normal distribution test, Mann-Whitney U test and independent sample T test on the collected data of the interval harmony perception experiment and the interval stimulation electroencephalogram response experiment.

[0096] Preferably, the Mann-Whitney U test method may include: grouping the collected data into the following categories: between every two intervals in the three-category interval classification, between subjects with no music learning experience and subjects with music learning experience, and between low-frequency stimuli and high-frequency stimuli; using the Mann-Whitney U test on each group of comparative data, and determining whether there is a significant difference between each group of comparative data based on the test results.

[0097] Preferably, the method for independent samples t-test may include: grouping the collected data into the following categories: subjects with no music learning experience and subjects with music learning experience, low-frequency stimuli and high-frequency stimuli; and using independent samples t-test for each group of comparative data.

[0098] Preferably, X can be set as data collected from the pitch harmonicity perception experiment; Y can be set as data collected from the pitch stimulus EEG response experiment.

[0099] The grouping categories for X and Y can include: participants with no music learning experience and participants with music learning experience, low-frequency sample group and high-frequency sample group; the data in X and Y can be individually or jointly grouped into group J data according to the grouping categories;

[0100] Each group of data in Y can be further divided into multiple unit data. The unit classification categories can include: N400 and LPC, amplitude and latency. The data in each group of Y can be divided into L unit data individually or in combination according to the unit classification category.

[0101] Each group of X may include 12 data points; each group of data points is obtained under the stimulation of audio samples of twelve different intervals, with one data point corresponding to one interval.

[0102] Each unit of Y may include 12 data points; each unit data point is obtained under the stimulation of audio samples of twelve different intervals, with one data point corresponding to one interval.

[0103] The correlation between data collected from the interval consonance perception experiment and data collected from the interval stimulation EEG response experiment was calculated using the Pearson correlation analysis method. The calculation formula is as follows:

[0104]

[0105] In the formula:

[0106] i represents the interval type number; i = 1, 2, ... 12;

[0107] j is the group number of X and Y, j = 1, 2, ... J;

[0108] l is the cell number of the j-th group in Y; l = 1, 2, ..., L;

[0109] x ji Let i be the i-th data in the j-th group of X;

[0110] is the average value of the jth group of data in X;

[0111] y jli is the ith data of the jth group of the first unit in Y;

[0112] is the average value of the jth group of the first unit in Y;

[0113] r jl is the correlation between the jth group of data in the interval harmony perception experiment and the jth group of the first unit data in the brain electrical response experiment.

[0114] Preferably, based on the correlation data of the data collected in the interval harmony perception experiment and the data collected in the interval stimulation brain electrical response experiment, a correlation heat map or a two-dimensional coordinate graph can be used to construct an interval harmony perception and brain electrical response correlation model.

[0115] The present application also provides a device for constructing an interval harmony perception and brain electrical response correlation model, comprising a memory and a processor, wherein the memory is used to store a computer program; and the processor is used to execute the computer program and realize the steps of the interval harmony perception and brain electrical response correlation model construction method as described above.

[0116] The present application also provides a storage medium storing a computer program, wherein the computer program is executed by a processor to realize the steps of the interval harmony perception and brain electrical response correlation model construction method as described above.

[0117] The working process and working principle of the present application will be further described below with reference to a preferred embodiment of the present application:

[0118] The present application is based on the acoustic physical properties of the interval, and the subjective perception characteristics under different interval stimulation are obtained through the interval harmony perception experiment, and the objective representation data of the brain neural response under interval stimulation are combined to jointly construct an interval harmony perception and brain electrical response correlation model. Music learning experienced subjects and non-music learning experienced subjects are selected, and two groups of audio (low frequency sample group and high frequency sample group, each group has 12 audios) are selected to carry out the experiment, and finally the subjective ranking of interval harmony degree of different people is associated with the brain electrical component index to construct a correlation model, and the optimal classification method of interval harmony degree is constructed based on the model.

[0119] Input: The interval audio stimulus material of the interval harmony perception experiment and the interval stimulus EEG response experiment. In order to study whether the interval harmony perception difference exists in the interval range of different frequencies, two sets of experimental stimulus materials are set. Considering that the frequency range of sound that can be perceived by the human ear is mainly 20-20000Hz in general, the root frequency of the first set of audio is set to 440.01Hz, which is defined as a low-frequency sample group, and the root frequency of the second set of audio is set to 523.264Hz, which is defined as a high-frequency sample group. Both groups contain 12 audios, each corresponding to two frequency values. Based on the twelve equal temperament, the audio stimulus material is made.

[0120] Step one, according to the music learning experience, two types of different experimental subjects are selected, including music learning experience subjects with more than 7 years of music learning experience and non-music learning experience subjects with less than 1 year of music learning experience. The male to female ratio of each type of subject is 1:1, and the selected subjects have no hearing impairment, and all subjects are right-handed.

[0121] Step two, interval harmony perception analysis, the subject is required to wear earphones to reduce external noise interference and concentrate on the audio heard. Two sets of audio materials are provided for the subjects, each set of audio material including 12 audio segments, and the subjects are required to perceive which of the two audios is more harmonious, so as to achieve the ordering of the 12 audios in each set in the harmony dimension, with the most harmonious score being 12 and the least harmonious score being 1.

[0122] Step three, based on the interval harmony perception analysis, the interval stimulus EEG response experiment is carried out. The Oddball paradigm is used in the interval stimulus EEG response experiment paradigm. Since auditory stimulation is a short-time stimulation, the present invention sets a single 500ms stimulation as the stimulation of an audio, and pure octave Oct is the high-probability stimulation with a probability of 70%, and the remaining 11 intervals are low-probability stimulations with a probability of 30%. After 80 stimulations, the listener selects the rest time, and after the rest is completed, the next 80 stimulations are started, and when 14 groups of 80 stimulations are cycled, the low-frequency sample group is completed. Since the audio has two groups, the high-frequency sample group is carried out after a short rest, and the experimental process is the same as that of the low-frequency sample group.

[0123] Step four, collect the EEG signals of different subjects, classify and arrange the EEG data based on the EEGLAB module of MATLAB (EEGLAB: an open-source MATLAB toolbox for analyzing and processing EEG data), and complete the extraction of EEG-related components.

[0124] Step five, construct an interval harmony perception and interval stimulus EEG response correlation model.

[0125] In the above steps, the interval harmony perception experiment is completed by step three, and the interval stimulus EEG response experiment is completed by step four.

[0126] 2. Technical principle of the present application.

[0127] 2.1 Interval harmony perception analysis.

[0128] The specific technical principle of the interval harmony perception experiment data processing method used in the present application is as follows.

[0129] Step A1: Design a questionnaire containing whether the subject has professional music learning experience and whether he / she likes to listen to music. In the questionnaire, the scores of 12 groups of audio in the harmony dimension are changeable to improve the operability of the subject's sorting.

[0130] Step A2: Screen the filled valid questionnaires to remove the questionnaires not filled according to the requirements, and ensure the accuracy of the data.

[0131] Step A3: Measure and analyze the interval harmony perception data, classify the data according to the interval harmony theory classification mode, including three classification and four classification, and perform difference test on the data in different classifications.

[0132] 2.2 EEG experiment data processing.

[0133] After the EEG experiment under interval stimulation is carried out, the data is processed, and the principle and steps are as follows:

[0134] Step B1: Use Curry7 software to record the brain response before and after interval stimulation, and preprocess the EEG ERP data under interval stimulation. The main steps include positioning of the 60 electrodes used, filtering (50Hz band-pass filtering), removing bad segments (removing signal segments disturbed during the experiment), re-referencing, running ICA, and removing artifacts, etc., to reduce the interference of subsequent ERP response component extraction data accuracy and difference analysis.

[0135] Step B2: According to the three classification method in the interval harmony theory, convert the data label in the original EEG data under twelve interval stimulation, and save the data again.

[0136] Step B3: According to the observation of time domain waveform under interval stimulation, find the significant component of EEG response induced by interval stimulation.

[0137] Step B4: The present application selects the ERP feature of the central brain area induced by interval harmony degree, averages the ERP response component data induced by the central electrode under twelve interval stimulation, performs difference test, and is also used for constructing the correlation model between interval harmony perception and EEG response.

[0138] 2.3 Difference test.

[0139] Difference test is a statistical test method for testing whether the difference between two groups or multiple groups of data is significant. The present application simultaneously performs difference test on the data of both the interval harmony perception experiment and the interval stimulation EEG response experiment, including the difference between the subjects with music learning experience and the subjects without music learning experience, the difference of twelve intervals, the difference between the three classifications of intervals, and the perception difference brought by the low-frequency and high-frequency sample groups. In this part, the interval harmony perception quantifies the difference in harmony perception through the score of the harmony dimension, the EEG response starts from the latency and amplitude of N400 (N400: a negative wave near 400 ms after the start of stimulation) and LPC component (LPC: a positive wave near 300-800 ms after the start of stimulation), and the difference test of the above data is carried out by SPSS software.

[0140] The present application mainly adopts the methods of normal distribution, Mann-Whitney U test, and independent sample T test.

[0141] (1) Before performing the data difference analysis, in order to ensure the rationality and correctness of the subsequent selection of difference test method, the present application firstly performs normal distribution test on the data of the interval harmony perception experiment and the component data extracted from the interval stimulation EEG response experiment. The commonly used methods generally include calculation of skewness coefficient, kurtosis coefficient, S-W test, K-S test, etc. The present application selects S-W test considering that the sample size of the data is small. This is a correlation algorithm, which calculates the z bc arranged in ascending order according to the numerical value, wherein According to the formula:

[0142]

[0143] In the formula:

[0144] The grouping categories of Z include: twelve intervals and three classifications of intervals, interval harmony perception experiment and interval stimulation EEG response experiment; the data in Z are grouped separately or mixedly according to the grouping categories; there are K groups of data;

[0145] z bc : the cth data in the bth group in Z; b=1, 2, …K; c=1, 2, …M b ; M b is the total sample size of the data in the bth group in Z;

[0146] is the average value of the data in the bth group in Z; b=1, 2, …K;

[0147] W b : the test statistic of the data in the bth group in Z;

[0148] a bc :z bc weight coefficient of the bth group of data samples in Z, according to the amount M b , the normality W test a bc table to determine the coefficient;

[0149] If W b ≥a, a fixed value of 0.05; then accept the normality assumption, indicating that the bth group of data in Z conforms to the normal distribution, if W b <a, it is indicated that the bth group of data in Z does not conform to the normal distribution.

[0150] (2) The present application is found that the interval three classification data does not satisfy the normal distribution when the interval harmony perception experimental data is subjected to normal test, therefore, the three groups of comparison data of interval harmony perception experiment (interval three classification between each two intervals, no music learning experience subjects and music learning experience subjects, low frequency sample group and high frequency sample group), each group of comparison data adopts Mann-Whitney U test, and whether there is significant difference between each group of comparison data is judged according to the test result.

[0151] (3) The present application is subjected to normal test when the brain central area ERP response component data under interval stimulation, the data conforms to the normal distribution, two groups of response component comparison data (no music learning experience subjects and music learning experience subjects, low frequency sample group and high frequency sample group), each group of comparison data adopts independent sample T test.

[0152] 2.4 Association model construction.

[0153] The present application is subjected to normal test when the brain central area ERP response component data under interval stimulation, the data conforms to the normal distribution, two groups of response component comparison data (no music learning experience subjects and music learning experience subjects, low frequency sample group and high frequency sample group), each group of comparison data adopts independent sample T test.

[0154] Let X be the data collected in the interval harmony perception experiment; Y be the data collected in the interval stimulation EEG response experiment;

[0155] The grouping categories of X and Y include: no music learning experience subjects and music learning experience subjects, low frequency sample group and high frequency sample group; the data in X and Y are individually or mixedly grouped into J groups of data according to the grouping categories;

[0156] Each group of data of Y is further divided into a plurality of unit data, and the unit classification categories include: N400 and LPC, amplitude and latency; each group of data in Y is individually or mixedly divided into L unit data according to the unit classification categories;

[0157] Each group of X includes 12 data; each group of data is obtained under the audio sample stimulus of twelve intervals, and one data corresponds to one interval;

[0158] Each unit of Y includes 12 data; each unit of data is obtained under the audio sample stimulus of twelve intervals, and one data corresponds to one interval;

[0159] The Pearson correlation analysis method is used to calculate the correlation between the data collected in the interval harmony perception experiment and the data collected in the interval stimulus electroencephalogram response experiment, and the calculation formula is as follows:

[0160]

[0161] In the formula:

[0162] i is the interval type number; i = 1, 2, … 12;

[0163] j is the group number of X and Y, j = 1, 2, … J;

[0164] l is the unit number of the jth group in Y; l = 1, 2, … L;

[0165] x ji is the ith data of the jth group in X;

[0166] is the average value of the data of the jth group in X;

[0167] y jli is the ith data of the lth unit of the jth group in Y;

[0168] is the average value of the lth unit of the jth group in Y;

[0169] r jl is the correlation between the jth group of data in the interval harmony perception experiment and the lth unit data of the jth group in the electroencephalogram response experiment.

[0170] 3. Verification experiment.

[0171] 3.1. Analysis of interval harmony perception experiment data - harmony score.

[0172] Two types of subjects (subjects without music learning experience and subjects with music learning experience) respectively in low frequency sample group and high frequency sample group, subjective feeling harmony score of twelve intervals, the results are shown in Figure 2 .

[0173] 3.2. Analysis of interval harmony perception experiment data - interval harmony classification research.

[0174] The present application divides the twelve intervals into three categories and four categories, and finds that the three categories have more significant differences in the difference test. The harmony score of the interval twelve categories is displayed according to the interval three categories, and the significant stars of the difference test are marked in the figure, * indicates that there is a difference between the data, ** indicates that there is a significant difference between the data, *** indicates that there is a particularly significant difference between the data, and **** indicates that the degree of significant difference between the data is the highest. Each data is represented as median ± interquartile range, and the results are shown in Figure 3 .

[0175] 3.3. Electroencephalogram data input for building a correlation model.

[0176] Under the stimulation of twelve intervals, the ERP responses of two types of subjects (subjects with or without music learning experience) and two groups of experiments (low frequency sample group and high frequency sample group) in the central region of the brain are displayed in terms of the latency and amplitude of response components N400 and LPC, as shown in Figure 4 , and the significant stars are marked in the figure, * indicates that there is a difference between the data, ** indicates that there is a significant difference between the data, and *** indicates that there is a particularly significant difference between the data.

[0177] 3.4. Electroencephalogram correlation data analysis based on the method of interval harmony degree three categories.

[0178] The response component data of the twelve intervals is divided into interval three categories, and each column of bar data is mean ± standard error, and the significant stars are marked in the figure, * indicates that there is a difference between the data, N400 component data (interval three categories) is shown in Figure 5 , and LPC component data (interval three categories) is shown in Figure 6 .

[0179] 3.5. Building an interval harmony degree perception and electroencephalogram response correlation model.

[0180] Based on the above interval harmony degree perception experiment data and electroencephalogram ERP response components, an interval harmony degree perception and electroencephalogram response correlation model is built. Among them, no indicates no music learning experience subjects, and yes indicates subjects with long-term music learning experience. Under the stimulation of two groups of audio (twelve intervals in each group), the interval harmony degree perception experiment (harmony score, data reference Figure 2 ) and the central region of the brain ERP response component (N400 and LPC component data reference Figure 4 ) are pearson correlated, and the significant stars are marked in the figure, * indicates that the data are significantly correlated, ** indicates that the data are particularly significantly correlated, "-" represents negative correlation, and "+" represents positive correlation. The correlation model (correlation heat map) is shown in Figure 7 , and the correlation of the harmony score of the subject with music learning experience and the N400 amplitude in the low frequency sample group isFigure 8 As shown, the correlation of the harmonic score of the music learning experience subject and the LPC amplitude is as shown in the following table. Figure 9 As shown.

[0181] The above detailedly gives the design principle and design result of the proposed construction method of the construction of the interval harmony degree perception and EEG response correlation model, and the verification experiment of the model, and some rules can be found through the correlation model:

[0182] (1) For the interval harmony degree perception experimental result, from Figure 2 and Figure 3 , it can be seen that:

[0183] a. Oct is considered the most harmonic interval, whether the subject has music learning experience or not, although individual interval scores have slight differences, but the overall discrimination trend of interval harmony degree is similar, and the score of the inharmonic interval is lower.

[0184] b. The twelve intervals have the most significant harmonic perception difference according to the interval three classification, the music learning experience subjects have significant differences in the interval harmony degree perception scores of the three types of intervals, the extremely complete harmonic interval has the highest harmony score, and the comparison of the two groups of intervals in the range of different frequencies finds that the high and low frequencies do not produce significant differences in the harmony dimension.

[0185] c. The perception of the octave interval of the subject without music learning experience has great difference, and it is difficult to distinguish the harmony degree of this interval.

[0186] (2) For the ERP response difference in the central area under the interval stimulation, combining Figure 4 , Figure 5 and Figure 6 , the brain response difference of the interval three classification can be seen, the extremely complete harmonic interval has earlier and smaller N400 response (absolute value) and higher LPC amplitude. The responses of the subjects with and without music learning experience under the interval three classification have similarity and difference.

[0187] (3) For the interval harmony degree perception and EEG response correlation model, combining Figure 7 , Figure 8 and Figure 9 , it can be seen that:

[0188] a. From the correlation model display of the correlation heat map, the interval harmony degree perception and EEG response correlation of the subject without music learning experience is not obvious, the music learning experience subject shows stronger interval harmony degree perception and brain response correlation, and more concentrates on the first group of tone frequency stimuli.

[0189] b.In the low frequency sample group, the harmony rating of the music learning experience subjects was significantly correlated with the N400 amplitude, r=0.624, p=0.03, the harmonic interval was embodied as higher harmony rating and shorter N400 amplitude (absolute value).

[0190] c.Also in the low frequency sample group, the harmony rating of the music learning experience subjects was significantly correlated with the LPC amplitude, r=0.806, p=0.002, the harmonic interval was embodied as longer LPC amplitude, and the interval three classification (dashed circle) also corresponded to different regions.

[0191] d.The constructed model has the advantage of accurately correlating the interval harmony perception and the electroencephalogram response.

[0192] The present application is based on the acoustic physical properties of the interval, acquires the subjective perception characteristics under different interval stimulation through the interval harmony perception experiment, and combines the objective representation data of the brain neural response under the interval stimulation to jointly construct the interval harmony perception and electroencephalogram response correlation model. The music learning experience subjects and the non-music learning experience subjects are selected, and two groups of audio (low frequency sample group and high frequency sample group, each group has 12 audios) are selected to carry out the experiment, finally the subjective perception of the interval harmony degree of different people and the electroencephalogram component index are constructed to construct the optimal classification method of the interval harmony degree based on the model.

[0193] The above-described embodiments are only used to illustrate the technical ideas and characteristics of the present application, and the purpose is to enable those skilled in the art to understand the content of the present application and implement it, and cannot be limited to the patent scope of the present application only by the present embodiment, that is, any equivalent changes or modifications made in the spirit disclosed by the present application still fall within the patent scope of the present application.

[0194] The references in the background art are as follows:

[0195] [1] Aldwell, E., Schachter, C., 2003. Harmony & Voice Leading. Thomson / Schirmer, Boston.

[0196] [2] Plomp R, Levelt W J M. Tonal consonance and critical bandwidth [J]. The journal of the Acoustical Society of America, 1965, 38(4): 548-560.

[0197] [3] Kameoka A, Kuriyagawa M. Consonance theory part I: Consonance of dyads [J]. The Journal of the Acoustical Society of America, 1969, 45(6): 1451-1459.

[0198] [4] Di Stefano, N., Focaroli, V., Giuliani, A., Formica, D., Taffoni, F., & Keller, F. (2017). A new research method to test auditory preferences in young listeners: Results from a consonance versus dissonance perception study. Psychology of Music, 45, 699-712.

[0199] [5] Brooks, D. I., & Cook, R. G. (2010). Chord discrimination by pigeons. Music Perception, 27, 183-196.

[0200] http: / / dx.doi.org / 10.1525 / mp.2010.27.3.183

[0201] [6] Crespo-Bojorque, P., & Toro, J. M. (2015). The use of interval ratios in consonance perception by rats (Rattus norvegicus) and humans (Homo sapiens). Journal of Comparative Psychology, 129, 42-51.

[0202] http: / / dx.doi.org / 10.1037 / a0037991

[0203] [7] Sugimoto, T., Kobayashi, H., Nobuyoshi, N., Kiriyama, Y., Takeshita, H., Nakamura, T., Hashiya, K., 2010. Preference for consonant music over dissonant music by an infant chimpanzee. Primates 51, 7-12.

[0204] [8] Koda, H., Nagumo, S., Basile, M., Olivier, M., Remeuf, K., Blois-Heulin, C., & Lemasson, A. (2013). Validation of an auditory sensory reinforcement paradigm: Campbell’s monkeys (Cercopithecus campbelli) do not prefer consonant over dissonant sounds. Journal of Comparative Psychology, 127, 265-271.

[0205] [9] Butler, J. W., & Daston, P. G. (1968). Musical consonance as musical preference: A cross-cultural study. The Journal of General Psychology, 79, 129-142.

[0206]

[10] McDermott, J. H., Schultz, A. F., Undurraga, E. A., & Godoy, R. A. (2016). Indifference to dissonance in native Amazonians reveals cultural variation in music perception. Nature, 535, 547-550.

[0207]

[11] Bidelman, G. M., Krishnan, A., 2009. Neural correlates of consonance, dissonance, and the hierarchy of musical pitch in the human brainstem. J. Neurosci. 29, 13165-13171.

[0208]

[12] Bidelman G M, Grall J. Functional organization for musical consonance and tonal pitch hierarchy in human auditory cortex [J]. NeuroImage, 2014, 101: 204-214.

Claims

1. A method for constructing a model associating interval consonance perception and electroencephalographic response, characterized in that, The method comprises the following steps: Step 1, preparing audio clips including multiple intervals as audio samples, wherein the probability of occurrence of pure octave interval is 70%, and the probability of occurrence of the remaining intervals is 30%; the time length of each audio clip is 500-600 ms; the audio samples are divided into two groups, namely a high-frequency sample group and a low-frequency sample group; the audio types in each group of audio samples are 12; the root audio frequency of the low-frequency sample group is unified as 440.01 Hz, and the root audio frequency of the high-frequency sample group is unified as 523.264 Hz; Step 2, selecting two types of experimental subjects according to music learning experience: subjects who have learned music for a long time and subjects who have learned music for a short time; the male-to-female ratio of each type of subject is 1:1; the selected subjects have no hearing impairment, and all the subjects are right-handed; Step 3, interval harmony perception experiment: the subject wears a headset, and scores the interval harmony degree of the audio clips in the audio sample set; the interval harmony degree scores of the subjects corresponding to different audio clips are counted; the interval harmony degree of the audio clips is sorted and classified according to the interval harmony degree scores; and difference test is performed; Step 4, interval stimulation EEG response experiment: the subject wears a headset, and performs multiple rounds of interval stimulation EEG experiments based on the Oddball paradigm using the audio clips in the audio sample set; each round includes multiple audio stimuli; each time an audio clip is used; the interval between the two rounds of audio stimuli is a period of time; EEG data corresponding to different audio clips of each subject is collected; the EEG data is sorted and classified based on the EEGLAB module of MATLAB; and difference test is performed; Step 5, based on the same audio clip, the interval harmony degree sorting and classification is associated with the corresponding EEG data; and an interval harmony perception and EEG response correlation model is constructed.

2. The method of claim 1, wherein the method further comprises: In step 3, the method for scoring the interval harmony degree of the audio clips comprises the following steps: Step A1, designing a questionnaire to collect the music learning experience, gender, music preference degree and the score of the interval harmony degree of the audio clips in the interval harmony perception experiment of the subject; wherein the score of the interval harmony degree of the audio clips can be changed; Step A2, screening the filled valid questionnaires to remove the questionnaires not filled according to the requirements; and counting the interval harmony perception experiment data; Step A3, according to the statistical data, measuring and analyzing the psychological perception dimension of the interval harmony; classifying the data according to the three-classification and four-classification classification mode of the interval harmony theory; and performing difference test on the data.

3. The method of claim 1, wherein the method further comprises: In step 4, the method for performing the interval stimulation EEG experiment comprises the following steps: Step B1, using Curry7 software to record the brain response before and after the interval stimulation, and pre-processing the EEG ERP data under the interval stimulation, including positioning, filtering, removing bad segments, re-referencing, running ICA and removing artifacts for multiple electrodes used; Step B2, converting the data labels in the original EEG data under the twelve-interval stimulation according to the three-classification method in the interval harmony theory, and saving the data again; Step B3, finding the brain response components significantly induced by the interval stimulation according to the observed time-domain waveforms under the interval stimulation; Step B4, the ERP feature of the most prominent central brain area in the brain electric response induced by the interval consonance degree is selected, and the ERP response component data of the central electrode under multiple interval stimuli are averaged, and difference test is performed on the data.

4. The method of claim 1, wherein the method further comprises: In steps 3 and 4, the method of difference test comprises: normal distribution test, Mann-Whitney U test and independent sample T test are performed on the collected data of the interval consonance perception experiment and the interval stimulation brain electric response experiment.

5. The method of claim 4, wherein the method further comprises: The method of Mann-Whitney U test comprises: the collected data are grouped according to the following classifications: each two intervals of the interval three classifications, the subjects without music learning experience and the subjects with music learning experience, the low-frequency stimulus and the high-frequency stimulus; Mann-Whitney U test is used for each group of compared data, and whether there is a significant difference between each group of compared data is judged according to the test result.

6. The method of claim 4, wherein the method further comprises: The method of independent sample T test comprises: the collected data are grouped according to the following classifications: the subjects without music learning experience and the subjects with music learning experience, the low-frequency stimulus and the high-frequency stimulus; independent sample T test is used for each group of compared data.

7. The method of claim 1, wherein the method further comprises: X is the data collected in the interval consonance perception experiment; Y is the data collected in the interval stimulation brain electric response experiment; The grouping categories of X and Y comprise: the subjects without music learning experience and the subjects with music learning experience, the low-frequency sample group and the high-frequency sample group; the data in X and Y are grouped into J groups of data individually or mixedly according to the grouping categories; Each group of data of Y is further divided into a plurality of unit data, and the unit classification categories comprise: N400 and LPC, amplitude and latency; each group of data in Y is divided into L unit data individually or mixedly according to the unit classification categories; Each group of X comprises 12 data; each group of data is obtained under the stimulation of twelve kinds of interval audio samples, and one data corresponds to one interval; Each unit of Y comprises 12 data; each unit of data is obtained under the stimulation of twelve kinds of interval audio samples, and one data corresponds to one interval; Pearson correlation analysis method is used to calculate the correlation between the data collected in the interval consonance perception experiment and the data collected in the interval stimulation brain electric response experiment, and the calculation formula is as follows: In the formula: i is the interval type number, i = 1, 2, …, 12; J is the grouping group number of X and Y, j = 1, 2, …, J; L is the unit number of the jth group in Y, l = 1, 2, …, L; x ji Xi,j is the i-th data of the j-th group in X; is the average value for the jth group of data in X; y jli is the i-th data of the l-th unit of the j-th group in Y; is the average value for the jth group of the lth unit in Y; r jl Correlation of the jth group of data of the interval harmony perception experiment and the jth group of the lth unit data of the brain electrical response experiment.

8. The method of claim 7, wherein the method further comprises: Based on the correlation data between the data collected in the interval consonance perception experiment and the data collected in the interval stimulation brain electric response experiment, a correlation heat map or a two-dimensional coordinate graph is used to construct an interval consonance perception and brain electric response correlation model.

9. A device for constructing a model associating a sense of consonance and dissonance with a brain electrical response, comprising a memory and a processor, characterized in that, The memory is used to store a computer program; the processor is used to execute the computer program and realize the interval consonance perception and brain electric response correlation model construction method steps in any one of claims 1 to 8 when executing the computer program.

10. A storage medium storing a computer program, characterized by The computer program is executed by the processor to realize the interval consonance perception and brain electric response correlation model construction method steps in any one of claims 1 to 8.

Images