Emotional chord music generation method and system based on EEG signal

Abstract

The invention relates to the field of neuroscience and music therapy, and discloses an emotional chord music generation method and system based on EEG signals, and the method comprises the following steps: selecting an area which is located in the brain and is related to basic emotional processing, emotional regulation and cognitive processing as an EEG electrode position; acquiring a signal of the position of the EEG electrode, and preprocessing the signal; classifying the EEG signals by using a machine learning algorithm, and identifying different emotional states; constructing an emotional state-music parameter mapping multi-dimensional model; and mapping the identified emotional state to a music parameter conforming to the harmony of the trichord so as to generate music matched with the emotional state. According to the method, the EEG signal and music generation are creatively combined, the trichord music which is matched with the emotional state and has a harmonious pitch relation is generated according to the emotional state, and the individuation and accuracy of music generation are improved.

Description

Technical Field

[0001] The present invention relates to the fields of neuroscience and music therapy, and in particular to a method and system for generating emotional chord music based on EEG signals. Background Art

[0002] With the development of neuroscience and music therapy, people have gradually realized the close connection between emotions and music. In the existing technology, there are studies on using biological signals (such as heart rate and breathing rate) to influence music generation, but using EEG signals to capture emotional states and generate music that matches them is still an emerging and challenging field.

[0003] Currently, EEG signals are mainly used for emotion recognition, and EEG signals are classified through machine learning or deep learning algorithms to identify different emotional states. However, most of these studies remain at the level of emotion recognition and fail to effectively convert the identified emotional states into music generation parameters. In addition, although there are existing music generation algorithms that can generate harmonious music, these algorithms often lack a direct connection with the emotional state, resulting in the generated music not matching the user's emotional state.

[0004] Therefore, the existing technology has the following problems and shortcomings: first, the correlation between EEG signals and music generation is not close enough, and it is impossible to generate music that matches the emotional state; second, the music generation algorithm lacks flexibility and is difficult to expand to more complex chord forms; third, there is a lack of real-time feedback and adjustment mechanism, which cannot meet the user's personalized needs for emotional regulation. Summary of the invention

[0005] The main purpose of the present invention is to solve the technical problem that the correlation between EEG signals and music generation in the prior art is not close enough. A method for generating emotional chord music based on EEG signals comprises the following steps: Selecting an area in the brain related to basic emotion processing, emotion regulation and cognitive processing as an EEG electrode location; collecting signals from the EEG electrode location and performing preprocessing; Apply machine learning algorithms to classify EEG signals and identify different emotional states; Construct a multidimensional model of emotional state-music parameter mapping; The identified emotional state is mapped to musical parameters that conform to triad harmony to generate music that matches the emotional state.

[0006] The EEG electrode locations are: Frontal lobe Fp1, frontal lobe Fp2, frontal lobe F3, frontal lobe F4, temporal lobe T7, temporal lobe T8, occipital lobe O1, occipital lobe O2, parietal lobe P3, parietal lobe P4.

[0007] The pre-processing steps are specifically as follows: Collect signals from EEG electrodes, assuming that the collected original signal is X(t)∈R C×T , where C represents the number of electrodes and T represents the sampling time length; Bandpass filter X(t) to extract the frequency bands related to emotions, including: θ4−8Hz, α8−13Hz, β13−30Hz; Extract the time-frequency domain feature power spectral density: Where F is Fourier transform; Non-negative matrix factorization (NMF) is used to reduce the data dimension and obtain the time series signal Xr.

[0008] Apply machine learning algorithms to classify EEG signals and identify different emotional states, including: Using Russell's emotion circle model, emotions are discretized into four emotion categories: high pleasure-high arousal (happiness), high pleasure-low arousal (calmness), low pleasure-high arousal (anger), and low pleasure-low arousal (sadness); Build an emotion classifier based on deep learning and emotion label data. Use convolutional neural network combined with long short-term memory network to process time series signal Xr: Where y is the emotional state vector, representing different emotional categories; The goal of classifier training is to minimize the cross entropy loss: Where K is the number of emotion categories, is the predicted emotion probability.

[0009] Construct a multidimensional model of emotional state-music parameter mapping, including: Construct a mapping function P = Wy + b, where P = [p1, p2, ..., pd], including chord type, rhythm, pitch, speed parameters, W∈R d×K is the weight matrix, b is the bias; Ensure that the generated music conforms to the harmony of three chords: Chord(p1)∈{Major,Minor,Diminished} And constrain the speed p4 to satisfy the range: 40≤p4≤180 BP.

[0010] The step of mapping the identified emotional state to music parameters that conform to the harmony of a triad to generate music that matches the emotional state includes: A variational autoencoder is used to generate music that matches the emotional state. The VAE model encodes the music parameter P into a latent vector z and generates a note sequence through a decoder: The loss function of the variational autoencoder is: Where D KL is the Kullback-Leibler divergence; Music sequence generation, the music parameters output by the decoder are mapped to MIDI format notes.

[0011] Variational Autoencoder (VAE) module: Encoder: Encodes the music parameters P (including chord type, rhythm, pitch, speed, etc.) into a latent vector z. The latent vector z is a low-dimensional representation of the music parameters and contains the key information needed to generate music.

[0012] Decoder: Decodes the latent vector z into musical parameters to generate a sequence of notes. The generated sequence of notes should conform to triad harmony and match the input emotional state vector y.

[0013] The method also includes real-time feedback adjustment: Adjust music parameters through real-time emotional feedback, using adaptive control strategy: Where α is the learning rate, Δy is the real-time emotion change, and the specific formula is as follows: in, yt, yt-1 represent the emotional state vectors at time t and t-1; β is the smoothing coefficient, and its value range is [0,1]; represents the emotion change after smoothing at time t−1; μ represents the mean of the variation; σ represents the standard deviation of the variation.

[0014] A second aspect of the present invention provides an emotional chord music generation system based on EEG signals, comprising: A signal acquisition module selects an area in the brain related to basic emotion processing, emotion regulation and cognitive processing as an EEG electrode location; collects signals from the EEG electrode location and performs preprocessing; The emotion recognition module applies machine learning algorithms to classify EEG signals and identify different emotional states; Model building module, building a multidimensional model of emotional state-music parameter mapping; The music generation module maps the identified emotional state to music parameters that conform to the harmony of triads to generate music that matches the emotional state.

[0015] A third aspect of the present invention provides an electronic device, comprising: a memory and at least one processor, wherein instructions are stored in the memory, and the memory and the at least one processor are interconnected via lines; the at least one processor calls the instructions in the memory so that the electronic device executes the above-mentioned method for generating emotional chord music based on EEG signals.

[0016] A fourth aspect of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores instructions, which, when executed on a computer, enable the computer to execute the above-mentioned method for generating emotional chord music based on EEG signals.

[0017] The present invention has the following beneficial effects: (1) The present invention innovatively combines EEG signals with music generation, achieving the generation of triad music that matches the emotional state and has harmonious interval relationships, thereby improving the personalization and accuracy of music generation.

[0018] (2) The present invention designs a flexible music generation algorithm to support future expansion to more complex chord forms, such as seventh chords, ninth chords, etc., thereby enhancing the scalability and practicality of the system.

[0019] (3) The present invention implements a real-time feedback and adjustment mechanism, allowing users to adjust the generated music in real time according to personal needs, thereby improving the interactivity of the system and user experience.

[0020] (4) The present invention focuses on safety and ethics, ensures the safety and comfort of the EEG acquisition process, respects the privacy and autonomy of the participants, and complies with relevant ethical norms and legal and regulatory requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 This is a first flow chart of a method for generating emotional chord music based on EEG signals provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0022] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0023] For ease of understanding, the specific process of the embodiment of the present invention is described below. Figure 1 The first embodiment of the method for generating emotional chord music based on EEG signals in the embodiment of the present invention includes: The regions in the brain related to basic emotion processing, emotion regulation and cognitive processing were selected as EEG electrode positions; the EEG electrode positions were: frontal lobe Fp1, frontal lobe Fp2, frontal lobe F3, frontal lobe F4, temporal lobe T7, temporal lobe T8, occipital lobe O1, occipital lobe O2, parietal lobe P3, parietal lobe P4. The signals at the EEG electrode positions were collected and preprocessed.

[0024] Electrode locations: The frontal lobe (Fp1, Fp2, F3, F4), temporal lobe (T7, T8), occipital lobe (O1, O2), and parietal lobe (P3, P4) regions were selected as EEG electrode locations. These regions are closely related to basic emotion processing, emotion regulation, and cognitive processing.

[0025] Signal acquisition: Use EEG acquisition equipment to collect EEG signals from the above electrode positions. The collected original signal is represented by X(t)∈R C×T , where C represents the number of electrodes and T represents the sampling time length.

[0026] The pre-processing steps are specifically as follows: Collect signals from EEG electrodes, assuming that the collected original signal is X(t)∈R C×T , where C represents the number of electrodes and T represents the sampling time length; Bandpass filter X(t) to extract the frequency bands related to emotions, including: θ4−8Hz, α8−13Hz, β13−30Hz; Extract the time-frequency domain feature power spectral density: Where F is Fourier transform; Non-negative matrix factorization (NMF) is used to reduce the data dimension and obtain the time series signal Xr.

[0027] Apply machine learning algorithms to classify EEG signals and identify different emotional states; Using Russell's emotion circle model, emotions are discretized into four emotion categories: High Pleasure-High Arousal (Happiness): This emotional state indicates that the individual is in a very positive and highly activated state, usually associated with emotions such as happiness and excitement.

[0028] High Pleasure-Low Arousal (Calm): This emotional state indicates that the individual is in a positive but relatively calm state, usually associated with emotions such as contentment and tranquility.

[0029] Low Pleasure-High Arousal (Anger): This emotional state indicates that the individual is in a negative and highly activated state, usually associated with emotions such as anger and tension.

[0030] Low Pleasure-Low Arousal (Sadness): This emotional state indicates that the individual is in a negative and relatively calm state, usually associated with emotions such as sadness and depression.

[0031] Build an emotion classifier based on deep learning and emotion label data. Use convolutional neural network combined with long short-term memory network to process time series signal Xr: Where y is the emotional state vector, representing different emotional categories; Data preparation: Collect EEG signal data with emotion labels. This data should contain EEG recordings of different individuals in different emotional states.

[0032] The EEG signal is preprocessed, including filtering, feature extraction and other steps, to obtain the time series signal Xr.

[0033] Convolutional Neural Network (CNN): used to extract local features from EEG signals. CNN can automatically learn and extract key information from signals through structures such as convolutional layers and pooling layers.

[0034] Long Short-Term Memory (LSTM): used to process long-term dependencies in time series signals. LSTM can capture the temporal dynamic characteristics of signals through its unique gating mechanism.

[0035] The output of CNN is used as the input of LSTM, which further processes the time series signal and outputs the emotional state vector y.

[0036] Define the loss function: The cross entropy loss function is used to measure the difference between the emotional state vector y predicted by the model and the true emotional label. The cross entropy loss function can reflect the accuracy of the model prediction and is used to guide the model training process.

[0037] Optimization algorithm: Select an appropriate optimization algorithm (such as Adam, SGD, etc.) to update the weight parameters of the model to minimize the cross entropy loss.

[0038] Training process: The preprocessed EEG signal data is input into the model, the loss is calculated through forward propagation, and then the weight parameters are updated through back propagation. This process is repeated until the model converges or the preset number of training rounds is reached.

[0039] After the model training is completed, the new EEG signal is input into the model for emotion classification. The model will output an emotion state vector y, which represents the probability distribution of the emotion category corresponding to the signal.

[0040] According to the maximum probability value in the emotional state vector y, the emotional category to which the signal belongs is determined.

[0041] The goal of classifier training is to minimize the cross entropy loss: Where K is the number of emotion categories, is the predicted emotion probability.

[0042] Construct a multidimensional model of emotional state-music parameter mapping; Construct a multidimensional model of emotional state-music parameter mapping, including: Construct a mapping function P = Wy + b, where P = [p1, p2, ..., pd], including chord type, rhythm, pitch, speed parameters, W∈R d×K is the weight matrix, b is the bias; Define the emotional state vector: The emotional state vector y represents the probability distribution of different emotion categories and is output by the deep learning model.

[0043] The dimension K of y is equal to the number of emotion categories, corresponding to the four emotions of high pleasure-high arousal (happiness), high pleasure-low arousal (calmness), low pleasure-high arousal (anger), and low pleasure-low arousal (sadness).

[0044] Define the music parameter vector: The music parameter vector P=[p1, p2, ..., pd], where d represents the number of music parameters.

[0045] In this solution, P at least includes parameters such as chord type p1, rhythm p2, pitch p3, speed p4, etc.

[0046] The value range of the chord type p1 is {Major (major chord), Minor (minor chord), Diminished (diminished chord)}, indicating the chord type of the music.

[0047] The tempo p4 needs to satisfy certain range constraints, such as 40≤p4≤180 BPM (beats per minute).

[0048] Ensure that the generated music conforms to the harmony of three chords: Chord(p1)∈{Major,Minor,Diminished} The step of mapping the identified emotional state to music parameters that conform to the harmony of a triad to generate music that matches the emotional state includes: A variational autoencoder is used to generate music that matches the emotional state. The VAE model encodes the music parameter P into a latent vector z and generates a note sequence through a decoder: The loss function of the variational autoencoder is: Where D KL is the Kullback-Leibler divergence; Music sequence generation, the music parameters output by the decoder are mapped to MIDI format notes.

[0049] The method also includes real-time feedback adjustment: Adjust music parameters through real-time emotional feedback, using adaptive control strategy: Where α is the learning rate and Δy is the real-time emotion change.

[0050] The Euclidean distance is used to calculate the change. In order to avoid the instability of Δy caused by noise, the sliding average method can be used to smooth the change. The calculated Δy is a smooth and standardized real-time change, which can stably adjust the music parameters.

[0051] In order to make the real-time emotion change more robust and adapt to individual differences, an individual adaptability weight γ can be introduced to solve the problem that different users have different sensitivities to emotion changes. The following is the improved formula γ tis a dynamic weight that represents the current user's sensitivity adjustment coefficient to emotional changes. Different individuals have different representations of emotions. For example, some users have large emotional fluctuations in their EEG signals, and need to reduce the response to the change; while other users have small fluctuations, and need to amplify the response.

[0052] γ t Dynamic adjustments are made based on historical EEG signal fluctuations and individual baselines.

[0053] λ: sensitivity coefficient, used to adjust γ t The degree to which fluctuations are amplified or reduced.

[0054] ybaseline: The user's individual emotional baseline, obtained through statistics of EEG data sampled in the early stage.

[0055] The user's individual emotional baseline is a statistic obtained through the previously sampled EEG data and the corresponding emotional state annotations, which is used to describe the emotional distribution characteristics of the user in a natural state.

[0056] Combined with user subjective reports or through existing emotion classification models, EEG signals are mapped into emotional states, emotional state vectors, including happiness, sadness, calmness, etc.

[0057] The labeled emotional state vector is statistically analyzed to obtain the baseline value: EEG data collection: Through professional EEG equipment, long-term EEG data collection is performed when the user is in a natural state. These data can reflect the electrophysiological activity of the user's brain and reveal their emotional state.

[0058] Emotional state annotation: While collecting EEG data, users’ emotional states are annotated through questionnaires, interviews, or observations. These annotation information will serve as an important basis for subsequent analysis.

[0059] Statistical analysis: Match the collected EEG data with the corresponding emotional state annotations, and use statistical methods to analyze them to obtain the statistics of the user's individual emotional baseline. These statistics may include mean, standard deviation, distribution range, etc., which are used to describe the electrophysiological characteristics of users in different emotional states.

[0060] The above describes the method for generating emotional chord music based on EEG signals in an embodiment of the present invention. The following describes the device for generating emotional chord music based on EEG signals in an embodiment of the present invention: A signal acquisition module selects an area in the brain related to basic emotion processing, emotion regulation and cognitive processing as an EEG electrode location; collects signals from the EEG electrode location and performs preprocessing; The emotion recognition module applies machine learning algorithms to classify EEG signals and identify different emotional states; Model building module, building a multidimensional model of emotional state-music parameter mapping; The music generation module maps the identified emotional state to music parameters that conform to the harmony of triads to generate music that matches the emotional state.

[0061] An embodiment of the present invention also provides an electronic device, which may have relatively large differences due to different configurations or performances, and may include one or more processors (central processing units, CPU) (for example, one or more processors) and memories, and one or more storage media for storing applications or data (for example, one or more mass storage devices). Among them, the memory and the storage medium may be short-term storage or permanent storage. The program stored in the storage medium may include one or more modules, each module may include a series of instruction operations in the electronic device. Furthermore, the processor may be configured to communicate with the storage medium and execute a series of instruction operations in the storage medium on the electronic device.

[0062] The electronic device may also include one or more power supplies, one or more wired or wireless network interfaces, one or more input and output interfaces, and / or one or more operating systems, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc. Those skilled in the art will appreciate that the electronic device structure in this embodiment does not constitute a limitation on the electronic device, and may include more or fewer components, or combine certain components, or arrange components differently.

[0063] An embodiment of the present invention provides a structure of an electronic device, which may have relatively large differences due to different configurations or performances, and may include one or more processors (central processing units, CPU) (for example, one or more processors) and memories, and one or more storage media for storing applications or data (for example, one or more mass storage devices). Among them, the memory and the storage medium can be short-term storage or permanent storage. The program stored in the storage medium may include one or more modules, each module may include a series of instruction operations in the electronic device. Furthermore, the processor can be configured to communicate with the storage medium and execute a series of instruction operations in the storage medium on the electronic device.

[0064] The electronic device may also include one or more power supplies, one or more wired or wireless network interfaces, one or more input and output interfaces, and / or one or more operating systems, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc. Those skilled in the art will appreciate that the electronic device structure does not constitute a limitation on the electronic device, and may include more or less components than the aforementioned, or combine certain components, or arrange the components differently.

[0065] The present invention also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. Instructions are stored in the computer-readable storage medium, and when the instructions are executed on a computer, the computer executes the steps of the aforementioned method.

[0066] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described system, device, or unit can refer to the corresponding process in the aforementioned method embodiment and will not be repeated here.

[0067] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the whole or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0068] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for generating emotional chord music based on EEG signals, characterized in that: The following steps are involved: Selecting an area in the brain related to basic emotion processing, emotion regulation and cognitive processing as an EEG electrode location; collecting signals from the EEG electrode location and performing preprocessing; Apply machine learning algorithms to classify EEG signals and identify different emotional states; Construct a multidimensional model of emotional state-music parameter mapping; The identified emotional state is mapped to musical parameters that conform to triad harmony to generate music that matches the emotional state.

2. The method for generating emotional chord music based on EEG signals according to claim 1, characterized in that: The EEG electrode positions are: Frontal lobe Fp1, frontal lobe Fp2, frontal lobe F3, frontal lobe F4, temporal lobe T7, temporal lobe T8, occipital lobe O1, occipital lobe O2, parietal lobe P3, parietal lobe P4.

3. The method for generating emotional chord music based on EEG signals according to claim 1, characterized in that: The pre-processing steps are specifically: Collect signals from EEG electrodes, assuming that the collected original signal is X(t)∈R C×T , where C represents the number of electrodes and T represents the sampling time length; Bandpass filter X(t) to extract the frequency bands related to emotions, including: θ4−8Hz, α8−13Hz, β13−30Hz; Extract the time-frequency domain feature power spectral density: Where F is Fourier transform; Non-negative matrix factorization (NMF) is used to reduce the data dimension and obtain the time series signal Xr.

4. The method for generating emotional chord music based on EEG signals according to claim 1, characterized in that: The machine learning algorithm is applied to classify EEG signals and identify different emotional states, including: Using Russell's emotion circle model, emotions are discretized into four emotion categories: high pleasure-high arousal (happiness), high pleasure-low arousal (calmness), low pleasure-high arousal (anger), and low pleasure-low arousal (sadness); Build an emotion classifier based on deep learning and emotion label data. Use convolutional neural network combined with long short-term memory network to process time series signal Xr: Where y is the emotional state vector, representing different emotional categories; The goal of classifier training is to minimize the cross entropy loss: Where K is the number of emotion categories, is the predicted emotion probability.

5. The method for generating emotional chord music based on EEG signals according to claim 1, characterized in that: The construction of the emotional state-music parameter mapping multidimensional model includes: Construct a mapping function P = Wy + b, where P = [p1, p2, ..., pd], including chord type, rhythm, pitch, speed parameters, W∈R d×K is the weight matrix, b is the bias; Ensure that the generated music conforms to the harmony of three chords: Chord(p1)∈{Major,Minor,Diminished} And constrain the speed p3 to satisfy the range: 40≤p3≤180 BP.

6. The method for generating emotional chord music based on EEG signals according to claim 1, characterized in that: The step of mapping the identified emotional state to music parameters that conform to the harmony of a triad to generate music that matches the emotional state includes: A variational autoencoder is used to generate music that matches the emotional state. The VAE model encodes the music parameter P into a latent vector z and generates a note sequence through a decoder: The loss function of the variational autoencoder is: Where D KL is the Kullback-Leibler divergence; Music sequence generation, the music parameters output by the decoder are mapped to MIDI format notes.

7. The method for generating emotional chord music based on EEG signals according to claim 1, characterized in that: The method also includes real-time feedback adjustment: Adjust music parameters through real-time emotional feedback, using adaptive control strategy: Where α is the learning rate, Δy is the real-time emotion change, and the specific formula is as follows: in, yt, yt-1 represent the emotional state vectors at time t and t-1; β is the smoothing coefficient, and its value range is [0,1]; represents the emotion change after smoothing at time t−1; μ represents the mean of the variation; σ represents the standard deviation of the variation.

8. An emotional chord music generation system based on EEG signals, characterized in that: The system comprises: A signal acquisition module selects an area in the brain related to basic emotion processing, emotion regulation and cognitive processing as an EEG electrode location; collects signals from the EEG electrode location and performs preprocessing; The emotion recognition module applies machine learning algorithms to classify EEG signals and identify different emotional states; Model building module, building a multidimensional model of emotional state-music parameter mapping; The music generation module maps the identified emotional state to music parameters that conform to the harmony of triads to generate music that matches the emotional state.

9. An electronic device, comprising a memory and at least one processor, wherein instructions are stored in the memory; The at least one processor calls the instructions in the memory to enable the electronic device to execute the various steps of the method for generating emotional chord music based on EEG signals as described in any one of claims 1-7.

10. A computer-readable storage medium having instructions stored thereon, characterized in that: When the instructions are executed by the processor, the various steps of the method for generating emotional chord music based on EEG signals as described in any one of claims 1 to 7 are implemented.

Images