Self-adaptive emotion adjustment method and device based on personalized reconfigurable music, computer equipment and storage medium

Abstract

The self-adaptive emotion adjusting method based on the personalized reconfigurable music comprises the following steps: collecting a brain wave signal of a user, and extracting a characteristic quantity from the brain wave signal; performing emotion recognition on the feature quantity to obtain a plurality of emotion states; playing multiple segments of different sample music to the user, and testing the emotion fluctuation condition of the user; establishing a music emotion association relationship corresponding to the user based on the sample music and the emotion fluctuation condition; updating the sample music according to the music emotion association management, and generating a personalized music library corresponding to the user; the current emotion of the user is monitored in real time, and the personalized music library is called according to the current emotion to adjust the emotion of the user. According to the method, the personalized music library corresponding to the user is constructed according to the individual difference of the user, so that the pertinence and the adaptability of performing emotion regulation on the user through music are improved, and the rapid and effective regulation on the emotion state of the user is further realized.

Description

Technical Field

[0001] The present invention relates to the field of brain-computer interface and emotion recognition, and in particular to an adaptive emotion regulation method, device, computer equipment and storage medium based on personalized reconfigurable music. Background Art

[0002] Emotion regulation is a key factor in maintaining people's mental health. Traditional emotion regulation methods mainly rely on manual counseling, psychotherapy, etc., which take a long time to produce results. With the development of brain-computer interface technology, emotion recognition and regulation based on EEG signals has become a new research hotspot.

[0003] Existing EEG emotion recognition technology determines the user's current emotional state by analyzing the EEG characteristics generated by the user when facing various emotional stimuli, such as alpha waves and beta waves. Emotion regulation technology attempts to guide users to regulate their emotions through feedback methods such as sound, light, and touch. However, most of these technologies are based on fixed databases for emotion regulation. For different individuals, the effects are uneven and it is difficult to produce a stable emotion regulation effect. Summary of the invention

[0004] In order to address the deficiencies in the prior art, the present invention provides an adaptive emotion regulation method, apparatus, computer equipment and storage medium based on personalized reconfigurable music, which constructs a personalized music library corresponding to the user according to the individual differences of the user, thereby improving the pertinence and adaptability of emotion regulation of the user through music, and further realizing rapid and effective regulation of the user's emotional state through the combined playback of reconfigurable music clips.

[0005] In order to achieve the above object, the specific scheme adopted by the present invention is: an adaptive emotion regulation method based on personalized reconfigurable music, comprising the following steps:

[0006] Collecting the user's brain wave signals and extracting feature quantities from the brain wave signals;

[0007] Perform emotion recognition on the feature quantity to obtain multiple emotional states;

[0008] Play multiple different music samples to the user and test the user's emotional fluctuations;

[0009] Establishing music emotion association relationship corresponding to the user based on sample music and emotional fluctuations;

[0010] Update sample music based on music emotion association management to generate a personalized music library corresponding to the user;

[0011] Monitor the user's current mood in real time, and call the personalized music library to adjust the user's mood according to the current mood.

[0012] As a further optimization of the above-mentioned adaptive emotion regulation method based on personalized reconfigurable music, after collecting the user's brain wave signal, the brain wave signal is preprocessed, and the signal preprocessing method includes at least one of filtering, denoising and removing eye artifacts.

[0013] As a further optimization of the above-mentioned adaptive emotion regulation method based on personalized reconfigurable music, the method of extracting feature quantities from brain wave signals includes:

[0014] The wavelet packet transform method is used to divide the brain wave signal into frequency bands to obtain frequency band signals, which include delta wave, theta wave, alpha wave and beta wave;

[0015] Calculate the characteristic quantities of the frequency band signal, including power spectrum density, power spectrum peak frequency, approximate entropy, Hurst exponent, wavelet entropy and sample entropy.

[0016] When performing emotion recognition on feature quantities, a pre-trained stacked sparse autoencoder neural network model is used for emotion recognition.

[0017] As a further optimization of the above-mentioned adaptive emotion regulation method based on personalized reconfigurable music, a method of playing multiple different sample music to a user and testing the user's emotion fluctuations includes:

[0018] Collecting the user's brain wave signals within a preset reference sampling time, and extracting feature quantities as reference features;

[0019] Play multiple different sample music to the user;

[0020] Dynamically collect the user's brain wave signals and extract feature quantities as dynamic features;

[0021] Determine the user's emotional arousal based on the difference between the dynamic features and the reference features;

[0022] When the emotional arousal exceeds a preset first threshold, determining the user's emotional fluctuation;

[0023] Change the user's emotional arousal by adjusting the sample music played;

[0024] When the amount of emotional fluctuation data obtained reaches a preset second threshold, the sample music is stopped from playing.

[0025] As a further optimization of the above-mentioned adaptive emotion regulation method based on personalized reconfigurable music, a specific method for establishing a music emotion association relationship corresponding to the user based on sample music and emotion fluctuations includes:

[0026] The sample music is segmented to obtain a plurality of music clips, and a fuzzy association table between the music clips and the emotional fluctuations is generated according to the playing records of the sample music;

[0027] The fuzzy association table is optimized to obtain the music emotion association relationship.

[0028] As a further optimization of the above-mentioned adaptive emotion regulation method based on personalized reconfigurable music, when the sample music is updated, the music clips are reorganized to generate multiple personalized music, and all personalized music are combined to form a personalized music library.

[0029] Emotion regulation devices, including:

[0030] A signal acquisition module is used to collect the user's brain wave signal and extract feature quantities from the brain wave signal;

[0031] A data processing module is used to perform emotion recognition on the feature quantity to obtain multiple emotional states;

[0032] An audio playback module is used to play multiple different sample music to the user;

[0033] The data storage module is used to store sample music, personalized music library and music emotion association.

[0034] Computer equipment, including:

[0035] Memory for storing computer programs;

[0036] A processor is used to read and execute the computer program to implement the above-mentioned adaptive emotion regulation method based on personalized reconfigurable music.

[0037] The storage medium is used to store a computer program, and when the computer program is executed, the above-mentioned adaptive emotion regulation method based on personalized reconfigurable music is implemented.

[0038] Beneficial effects: The present invention constructs a personalized music library corresponding to the user according to the individual differences of the user, thereby improving the pertinence and adaptability of emotional regulation of the user through music, and further realizes rapid and effective regulation of the user's emotional state through combined playback of reconstructible music clips. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 is a flow chart of the method of the present invention;

[0040] Figure 2 is a schematic diagram of the four emotional areas. DETAILED DESCRIPTION

[0041] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0042] like Figure 1 As shown, the present invention first provides an adaptive emotion regulation method based on personalized reconfigurable music, including S1 to S6.

[0043] S1. Collect the user's brain wave signal and extract the characteristic quantity from the brain wave signal. After collecting the user's brain wave signal, the brain wave signal is preprocessed, and the signal preprocessing method includes at least one of filtering, denoising and removing eye artifacts. By preprocessing the brain wave signal, it is convenient to extract the characteristic quantity from the brain wave signal, which is convenient for subsequent use. The method of extracting the characteristic quantity from the brain wave signal includes S11 to S12.

[0044] S11. Using the wavelet packet transform method, the brain wave signal is divided into frequency bands to obtain frequency band signals, and the frequency band signals include delta wave, theta wave, alpha wave and beta wave. Each frequency band signal and the specific division method are conventional technologies in this field and will not be repeated here. The correlation between different frequency band signals and emotions is shown in Table 1.

[0045] Table 1 Correlation between frequency band signals and emotions

[0046] S12, calculate the characteristic quantities of the frequency band signal, the characteristic quantities include power spectrum density, power spectrum peak frequency, approximate entropy, Hurst exponent, wavelet entropy and sample entropy. Each characteristic quantity and the specific calculation method are also conventional techniques in this field, and will not be repeated here. In addition to power spectrum density, power spectrum peak frequency, approximate entropy, Hurst exponent, wavelet entropy and sample entropy, other characteristic quantities can also be calculated, and all characteristic quantities can be combined into the joint characteristics of the brain wave signal.

[0047] S2. Emotion recognition is performed on the feature quantity to obtain multiple emotional states. In this embodiment, when emotion recognition is performed on the feature quantity, a pre-trained stacked sparse auto-encoder neural network model (Stacked Sparse Auto-Encoder, SSAE) is used to perform emotion recognition, and the result of the recognition is the emotional state. In the process of training the stacked sparse auto-encoder neural network model, a combination of unsupervised learning and supervised learning is adopted, so that the stacked sparse auto-encoder neural network model can more accurately extract high-level abstract representations from the feature quantity, effectively improve the accuracy of classifying emotional states, and then provide a reliable basis for subsequent emotional regulation.

[0048] S3, playing a plurality of different sample music to the user and testing the user's emotional fluctuations. The method of playing a plurality of different sample music to the user and testing the user's emotional fluctuations includes S31 to S37.

[0049] S31. Collect the user's brain wave signal within a preset reference sampling time, and extract the characteristic quantity as a reference characteristic. At the beginning of playing the sample music, it is necessary to determine the user's initial brain wave signal, so that when the sample music is adjusted later, it is judged whether the user's mood has changed according to the change of the user's brain wave signal. In this embodiment, the reference sampling time is set to 20s, that is, the user's brain wave signal is collected 20s before the sample music is played, and the characteristic quantity is extracted as a reference characteristic. It should also be noted that it is best to start playing sample music to the user when the user is calm. At this time, the reference characteristics extracted are easier to judge the user's emotional fluctuations later.

[0050] S32, playing a plurality of different sample music to the user. By playing different sample music to the user, the user's emotional sensitivity to different sample music can be understood, that is, whether the user's emotions change when listening to the sample music, providing sufficient basis for subsequent adjustment of the user's emotions.

[0051] S33. Dynamically collect the user's brain wave signals and extract feature quantities as dynamic features.

[0052] S34. Determine the user's emotional arousal based on the difference between the dynamic features and the reference features. If the difference between the dynamic features and the reference features is large, the user's emotional arousal is high, and the corresponding user's emotional state has changed; if the difference between the dynamic features and the reference features is small, the user's emotional arousal is low, and the corresponding user's emotional state has not changed significantly. If the user's emotional state has changed significantly, it means that the user is more sensitive to the sample music currently being played, that is, the user has a higher emotional sensitivity to the current sample music, and his emotional state is more easily affected by the sample music. Conversely, if the user's emotional state has not changed significantly, it means that the user is not sensitive to the sample music currently being played, that is, the user has a low emotional sensitivity to the current sample music, and his emotional state is not easily affected by the sample music.

[0053] S35. When the emotional arousal exceeds a preset first threshold, determine the user's emotional fluctuation. When the emotional arousal exceeds the first threshold, it can be considered that the user's emotional state has undergone a relatively thorough change, that is, a complete conversion from one emotional state to another. The emotional fluctuation can be represented by the two emotional states before and after the change.

[0054] S36, changing the user's emotional arousal by adjusting the sample music played. By changing the user's emotional arousal, the user's emotional sensitivity to different sample music can be continuously obtained, and then the influence of different sample music on the user's emotional state can be determined. Figure 2 As shown, the user's emotional state can be changed between four emotional areas of high valence-high arousal (HVHA), high valence-low arousal (HVLA), low valence-high arousal (LVHA) and low valence-low arousal (LVLA) by adjusting the sample music played.

[0055] S37, when the amount of data on the emotional fluctuations obtained reaches a preset second threshold, stop playing the sample music. When the amount of data on the emotional fluctuations obtained reaches the preset second threshold, it means that enough data on the association between the user's emotional state and the sample music has been collected, and the playback can be stopped for subsequent processing. The second threshold can be determined according to actual needs.

[0056] S4. Establishing a music emotion association relationship corresponding to the user based on the sample music and the emotion fluctuation situation. The specific method of establishing a music emotion association relationship corresponding to the user based on the sample music and the emotion fluctuation situation includes S41 to S42.

[0057] S41, the sample music is segmented to obtain a plurality of music clips, and a fuzzy association table of the music clips and the emotional fluctuations is generated according to the playing records of the sample music. Specifically, each music clip is assigned an emotional label, and the emotional label is used to mark the corresponding relationship between the music clip and the user's emotional state, and all emotional labels and music clips are combined to form a fuzzy association table.

[0058] S42, optimizing the fuzzy association table to obtain the music emotion association relationship. Based on the fuzzy association table, considering that if there is no appropriate transition between two adjacent music clips when playing music clips, it may cause discomfort to the user and cause a chaotic change in the user's emotional state. In order to avoid this situation, it is necessary to optimize the fuzzy association table, and between two corresponding music clips with large emotional state differences, add a music clip that can smoothly transition between the two music clips, so that the user's emotional state can be adjusted more smoothly in the future.

[0059] S5. Update the sample music according to the music emotion association management to generate a personalized music library corresponding to the user. When updating the sample music, reorganize the music clips to generate multiple personalized music, and combine all personalized music to form a personalized music library. In a sample music, different music clips may have different emotional tags, that is, a sample music may be associated with multiple emotional states of the user. In order to more accurately adjust the emotional state of the user, the present invention recombines the music clips so that the combined personalized music can more efficiently adjust the emotional state of the user.

[0060] S6. Real-time monitoring of the user's current mood, and calling the personalized music library to adjust the user's mood according to the current mood. After determining the relationship between the user's emotional state and the personalized music library, the personalized music library can be called according to the user's current mood to guide and adjust the user's emotional state.

[0061] The present invention further provides an emotion regulation device, comprising a signal acquisition module, a data processing module, an audio playback module and a data storage module.

[0062] The signal acquisition module is used to collect the user's brain wave signals and extract feature quantities from the brain wave signals. In one embodiment of the present invention, the core of the signal acquisition module is the chip ADS1299 produced by TI and dedicated to EEG signal acquisition.

[0063] The data processing module is used to perform emotion recognition on the feature quantity to obtain multiple emotional states. In one embodiment of the present invention, the core of the signal processing module adopts the low-power chip MSP430.

[0064] The audio playback module is used to play multiple different sample music to the user.

[0065] The data storage module is used to store sample music, personalized music library and music emotion association.

[0066] The present invention also provides a computer device, including a memory and a processor.

[0067] Memory, used to store computer programs.

[0068] A processor is used to read and execute the computer program to implement the above-mentioned adaptive emotion regulation method based on personalized reconfigurable music.

[0069] The memory, as a carrier for storing resources, may be a read-only memory, a random access memory, a disk or an optical disk, etc. The resources stored thereon may include an operating system, a computer program, etc., and the storage method may be temporary storage or permanent storage. The operating system is used to manage and control various hardware devices and computer programs on the electronic device, which may be Windows Server, Netware, Unix, Linux, etc. In addition to the computer program that can be used to complete the adaptive emotion regulation method based on personalized reconfigurable music performed by the electronic device disclosed in any of the aforementioned embodiments, the computer program may further include a computer program that can be used to complete other specific tasks. The processor may adopt a general-purpose processor product based on X86, IA64, RISC, MIPS, ARM and other architectures.

[0070] Finally, the present invention provides a computer-readable storage medium for storing a computer program, which implements the above-mentioned adaptive emotion regulation method based on personalized reconfigurable music when executed. The above description of the disclosed embodiments enables professionals and technicians in this field to implement or use the present invention. Various modifications to these embodiments will be obvious to professionals and technicians in this field, and the general principles defined in this article can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown in this article, but will conform to the widest scope consistent with the principles and novel features disclosed in this article.

Claims

1. An adaptive emotion regulation method based on personalized reconfigurable music, characterized in that: The steps include: Collecting the user's brain wave signals and extracting feature quantities from the brain wave signals; Perform emotion recognition on the feature quantity to obtain multiple emotional states; Play multiple different music samples to the user and test the user's emotional fluctuations; Establishing music emotion association relationship corresponding to the user based on sample music and emotional fluctuations; Update sample music based on music emotion association management to generate a personalized music library corresponding to the user; Monitor the user's current mood in real time, and call the personalized music library to adjust the user's mood according to the current mood.

2. The adaptive emotion regulation method based on personalized reconfigurable music as claimed in claim 1, characterized in that: After collecting the user's brain wave signal, the brain wave signal is preprocessed, and the signal preprocessing method includes at least one of filtering, denoising and removing eye artifacts.

3. The adaptive emotion regulation method based on personalized reconfigurable music as claimed in claim 1, characterized in that: Methods for extracting features from brain wave signals include: The wavelet packet transform method is used to divide the brain wave signal into frequency bands to obtain frequency band signals, which include delta wave, theta wave, alpha wave and beta wave; Calculate the characteristic quantities of the frequency band signal, including power spectrum density, power spectrum peak frequency, approximate entropy, Hurst exponent, wavelet entropy and sample entropy.

4. The adaptive emotion regulation method based on personalized reconfigurable music as claimed in claim 1, characterized in that: When performing emotion recognition on feature quantities, a pre-trained stacked sparse autoencoder neural network model is used for emotion recognition.

5. The adaptive emotion regulation method based on personalized reconfigurable music as claimed in claim 1, characterized in that: The method of playing a plurality of different sample music to a user and testing the user's emotional fluctuations includes: Collecting the user's brain wave signals within a preset reference sampling time, and extracting feature quantities as reference features; Play multiple different sample music to the user; Dynamically collect the user's brain wave signals and extract feature quantities as dynamic features; Determine the user's emotional arousal based on the difference between the dynamic features and the reference features; When the emotional arousal exceeds a preset first threshold, determining the user's emotional fluctuation; Change the user's emotional arousal by adjusting the sample music played; When the amount of emotional fluctuation data obtained reaches a preset second threshold, the sample music is stopped from playing.

6. The adaptive emotion regulation method based on personalized reconfigurable music according to claim 1, characterized in that: The specific method of establishing a music emotion association relationship corresponding to the user based on sample music and emotional fluctuations includes: The sample music is segmented to obtain a plurality of music clips, and a fuzzy association table between the music clips and the emotional fluctuations is generated according to the playing records of the sample music; The fuzzy association table is optimized to obtain the music emotion association relationship.

7. The adaptive emotion regulation method based on personalized reconfigurable music as claimed in claim 1, characterized in that: When the sample music is updated, the music clips are reorganized to generate multiple personalized music, and all personalized music are combined to form a personalized music library.

8. An emotion regulation device, characterized in that include: A signal acquisition module is used to collect the user's brain wave signal and extract feature quantities from the brain wave signal; A data processing module is used to perform emotion recognition on the feature quantity to obtain multiple emotional states; An audio playback module is used to play multiple different sample music to the user; The data storage module is used to store sample music, personalized music library and music emotion association.

9. Computer device, characterized in that include: Memory for storing computer programs; A processor, configured to read and execute the computer program to implement the adaptive emotion regulation method based on personalized reconfigurable music as described in any one of claims 1 to 7.

10. A storage medium, characterized in that Used to store a computer program, which, when executed, implements the adaptive emotion regulation method based on personalized reconfigurable music as described in any one of claims 1 to 7.

Images