Generative AI-driven physiology-music following response intelligent emotion regulation system

Abstract

The invention discloses a generative AI-driven physiological-music following response intelligent emotion regulation system. The system mainly comprises a physiological signal acquisition module, a signal preprocessing module, an emotion real-time decoding module, a music-emotion response calibration module and a music generation and emotion regulation module. On the basis of the generative AI technology, music expectation, a music tension release mode and other music psychology theories are fused, music capable of directionally adjusting the emotional state of the user is generated through real-time accurate decoding of the emotional state of the user through multi-mode physiological signals, the user expectation is gradually established and released at the high point, roller coaster-like emotional changes of the user are caused, and the user experience is improved. And the user can subjectively obtain pleasant and satisfied positive emotions and the like.

Description

Technical Field

[0001] The present invention relates to an intelligent emotion regulation technology based on generative AI, belonging to the technical field of artificial intelligence and signal processing. Background Art

[0002] Music is the art of arranging sounds in a temporal sequence using the elements of melody, harmony, rhythm, and timbre. Unlike traditional art forms, music lacks direct plasticity and semantics, and therefore is limited in its ability to express the material world. However, music is very successful in stimulating emotions. Composers can effectively evoke a variety of emotional responses in people by manipulating the listener's expectations.

[0003] According to music psychology theory, the results of music events can be divided into two stages: before the results appear, especially when the results are uncertain, the listener mainly produces a preparatory reaction related to stress or tension, and this emotional tension will continue to accumulate before the results appear, making the listener's emotional arousal and attention reach a peak; when the results appear, the listener's expectations are realized, the emotional tension is released, and a strong sense of relaxation and satisfaction and other positive experiences will occur. The working principle of this process is similar to the principle of riding a roller coaster.

[0004] However, individuals' perception of music is highly heterogeneous, and single, fixed-style music has limited effects. In recent years, generative AI technology has made breakthroughs, enabling music generation models to understand and generate complex music structures, including melody, harmony, rhythm, etc., through deep neural networks and advanced sequence modeling techniques (such as Transformer, generative adversarial networks GAN, and variational autoencoders VAE). However, existing AI music generation systems often rely on static input or preset rules, and lack real-time perception and feedback adjustment of individual emotions and physiological states.

[0005] In the field of physiological measurement, studies have found that a variety of physiological parameters, such as breathing rate, electrocardiogram, and skin electricity, are closely related to emotional state and tension. As early as the beginning of the 20th century, lie detection experiments used physiological indicators such as breathing patterns and blood pressure changes. In recent years, the fusion of multimodal physiological signals (such as EEG, ECG, and respiration) has shown great potential in the field of affective computing, indicating that the complementarity between different signals can effectively improve the accuracy of emotion recognition. Therefore, it is feasible to integrate multimodal physiological signals, adjust the properties of AI-generated music, and then accurately and personally adjust user emotions.

[0006] Although some systems attempt to combine physiological feedback to achieve closed-loop regulation of emotions, these systems do not fully consider the complex relationship between multimodal physiological signals, and the constraint mechanism is usually missing or relatively simple. In addition, in terms of design principles, existing systems fail to combine more advanced psychological theories to achieve targeted emotion regulation. Summary of the invention

[0007] Purpose of the invention: In view of the shortcomings of the prior art, the purpose of the present invention is to propose a generative AI-driven physiological-music-following-response intelligent emotion regulation system, which is suitable for computers and mobile devices and can be implemented through data acquisition devices or wearable acquisition devices. Based on music psychology theories such as music expectation and music tension release mode, it assists users to obtain a roller coaster-like positive emotion regulation experience and creates an adaptive biofeedback system driven by maximizing user relaxation and satisfaction.

[0008] Technical solution: To achieve the above-mentioned invention object, the present invention adopts the following technical solution:

[0009] In a first aspect, the present invention provides a generative AI-driven physiological-music following response intelligent emotion regulation system, comprising:

[0010] A physiological signal acquisition module is used to collect multimodal physiological signals of users when they listen to music;

[0011] The signal preprocessing module is used to preprocess various physiological signals and initially filter out noise in the signals;

[0012] The real-time emotion decoding module is used to decode the user's emotional state in real time based on the pre-trained emotion feature recognition network;

[0013] A music-emotion response calibration module is used to generate a music-emotion response calibration curve by detecting an individual's emotional response to music of different attributes;

[0014] The music generation and emotion adjustment module is used to adaptively generate and adjust music according to the user's emotional state and the music-emotion response calibration curve based on the generative AI model, music expectation and tension release theory.

[0015] Furthermore, the multimodal physiological signals include one or more of EEG signals, ECG signals, respiratory signals, heart rate signals, body surface temperature, facial expressions and skin electrical activity signals; the denoising processing of each physiological signal in the signal preprocessing module includes a combination of one or more of high-pass filtering, low-pass filtering and wavelet threshold denoising methods.

[0016] Furthermore, in the real-time emotion decoding module, one or more physiological signals including EEG signals, ECG signals, respiratory signals, heart rate signals, body surface temperature, facial expressions and skin electrical activity signals are integrated to decode the user's emotional state in real time; fine-grained features in one or more long-term complex physiological signals including EEG signals, ECG signals, respiratory signals and skin electrical activity signals are extracted based on the CNN-LSTM network; multimodal emotion features are deeply integrated through Transformer, and other simple physiological signals are integrated through MLP to achieve real-time decoding of the user's emotional state, and obtain the emotional response E={e1,e2,...,e s}, where s is the total number of dimensions of sentiment evaluation.

[0017] Furthermore, in the music-emotion response calibration module, a typical music clip is determined by hierarchical clustering analysis of multiple attributes (such as pitch, melody, harmony, intensity, etc.) in a large-scale music dataset, and a random test music group M = {F1, F2, ..., F k}, where k is the number of clusters; in the process of playing the music group, the user's response to the nth music clip F in the music group M is measured n Specific emotional response (s is the total number of dimensions of emotion evaluation), construct the music-emotion response calibration curve

[0018] Furthermore, the music generation and emotion adjustment module captures the dynamic properties of music and the time dependence characteristics between segments based on the Transformer model, adaptively constructs constraints, generates music with coherent style, and obtains the embedded features of the music sequence. and its position encoding Among them, T is the total number of time steps of music, and d is the music feature encoding dimension.

[0019] Furthermore, the music generation and emotion adjustment module, based on the pre-trained music generation model, calibrates the user's music-emotion response curve L emotion Embedded into the encoding layer of the music generation model to achieve the follow-up response of music attribute-emotional state:

[0020] C emotion =Embedding(L emotion ),

[0021]

[0022] Where α=σ(W α [AvgPool(C emotion )])∈(0,1).

[0023] Cemotion is the conditional vector for the music generation model, (k is the number of clusters, T is the total number of music time steps) is used to achieve soft alignment of the music-emotion response calibration curve to the music time step, ⊙ represents element-by-element multiplication, P music Represents the music position code, C music represents the music sequence embedding feature, P emotion Represents the positional encoding of the music-emotion response calibration curve, X input is the model input, + means concatenating features into a vector, ∥ means merging music information and emotional information into a large input vector, α is the adaptive weight parameter, σ is the Sigmoid function, (T is the total number of time steps of the music, and s is the total number of dimensions of emotional evaluation) is used to achieve weight control of the overall influence intensity of emotional conditions.

[0024] Furthermore, the music generation and emotion adjustment module, based on the trained music generation model, generates the music according to the user's current emotion E. current and the set of previous emotional states {E1,E2,…,E current} and music psychology theory, adaptively adjust model parameters to generate music that can directional mobilize the user's emotional state:

[0025] C dynamic =Embedding(E1,E2,…,E current ),

[0026] p=f adjust (C dynamic ,Θ),

[0027]

[0028] Among them, f adjust is a learnable mapping function based on music psychology theory; Θ is the specific parameter of the trained music generation model; p is the conditional probability distribution of music attributes generated according to music psychology theory; is the loss function for generating music, T is the total number of time steps, s t is the expected emotional state at time step t, Represents a given previous state and dynamic condition C dynamic Next, generate s at time step t t probability.

[0029] In a second aspect, the present invention provides a generative AI-driven physiological-music following response intelligent emotion regulation method, comprising the following steps:

[0030] Collect multimodal physiological signals of users while listening to music;

[0031] Preprocess various physiological signals and initially filter out noise in the signals;

[0032] Decode the user's emotional state in real time based on the pre-trained physiological signal analysis network;

[0033] By detecting the emotional response of individuals to music with different attributes, a music-emotion response calibration curve is generated;

[0034] Based on the generative AI model, music expectation and tension release theory, music is adaptively generated and adjusted according to the user's emotional state and the music-emotion response calibration curve.

[0035] In a third aspect, the present invention provides a computer system comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the computer program is executed by the processor, the steps of a generative AI-driven physiological-music following response intelligent emotion regulation method described in the second aspect are implemented.

[0036] In a fourth aspect, the present invention provides a computer program product, comprising a computer program, which, when executed by a processor, implements the steps of a generative AI-driven physiological-music following response intelligent emotion regulation method described in the second aspect.

[0037] Beneficial effects: In terms of design concept, the present invention refers to the music expectation theory, combines prior knowledge and data-driven, and proposes an intelligent emotion regulation system with physiological-music follow-up response. Specifically, with the music expectation theory as the core theory for regulating emotions, user expectations are gradually established through music, and released at high points, causing roller coaster-like emotional changes in users, so that users can subjectively obtain emotions such as pleasure and satisfaction. In terms of technical implementation, the present invention proposes a new music processing and generation mode. Specifically, the music generation model captures the long-term dependency of music through a self-supervised learning module based on a generative AI model, such as Transformer, and is used to generate complex melodies, harmonies, and rhythms. In addition, through the attention mechanism embedded in the model, a constraint connection can be established between different fragments of a music passage to generate music with a coherent style. The present invention refers to the current emotions of the user quantified by multimodal physiological signals and several previous emotional states, combined with personalized emotional response labels. If the user's nervous state is still on the rise, it is chosen to continue to increase the music tension and accumulate psychological expectations; if the user's nervous state stops rising or rises more slowly, it is chosen to quickly reduce the music tension and release psychological expectations.

[0038] In summary, compared with the prior art, the present invention has the following beneficial effects: (1) the present invention uses music, which is easy to obtain and can bring aesthetic experience, to regulate user emotions in the form of a passive biofeedback system, helping users to intuitively experience the process of expectation accumulation and release, and gain relaxation and satisfaction; (2) guided by the music expectation theory, based on real-time multimodal physiological signals and generative AI large models, the present invention adaptively generates and adjusts corresponding music to achieve the precise establishment and release of user expectations. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 It is a schematic diagram of the basic principle of the present invention.

[0040] Figure 2 Schematic diagram of the system structure of an embodiment of the present invention.

[0041] Figure 3 4 is a system flow chart of an embodiment of the present invention.

[0042] Figure 4 Schematic diagram of the emotion recognition network training process in an embodiment of the present invention.

[0043] Figure 5 Schematic diagram of the music generation model training process in an embodiment of the present invention. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0045] This embodiment is implemented in accordance with the invention design and is in the form of an automated biofeedback system. The working principle is that the system generates music based on generative AI technology according to the user's current and previous emotional state to induce and guide the user's emotions. The regulation of emotions is based on the music expectation theory, where emotions refer to positive stress, tension, arousal, and expectation. Figure 1 As shown in the figure, the goal of the biofeedback system is to gradually increase the emotional intensity of users to the highest point by letting them listen to music, and then quickly release these emotions, so that users can feel relaxed and satisfied. At the same time, the interactive module can further help users understand their own psychological and physiological process of relaxation, which helps users learn the relaxation process and judge the effectiveness of relaxation methods.

[0046] This embodiment implements a generative AI-driven physiological-music following response intelligent emotion regulation system, the system structure is as follows Figure 2 As shown, the system consists of a sound output module, a physiological signal acquisition module, a signal preprocessing module, a real-time emotion decoding module, a music-emotion response calibration module, a music generation and emotion adjustment module, and a real-time interaction and storage module.

[0047] like Figure 3 As shown, the working principle of this embodiment is as follows: the sound output module interacts directly with the user and is used for the transmission, conversion and playback of music signals; the physiological signal acquisition module is used to collect multimodal physiological signals of the user in the process of listening to music in real time, including but not limited to EEG signals, ECG signals, breathing signals, heart rate signals, body surface temperature, facial expressions and skin electrical activity signals, or other types of contact or non-contact physiological signals that can represent emotional states, and transmit the data to the signal preprocessing module; the signal preprocessing module performs denoising processing on various physiological signals such as EEG signals, ECG signals, breathing signals, body surface temperature and skin electrical activity signals transmitted by the physiological signal acquisition module, so as to reduce the interference of signal noise on subsequent processing and analysis, and preliminarily filter The signal after noise removal is transmitted to the real-time emotion decoding module; the real-time emotion decoding module takes the multimodal deep learning network as the core, uses a large-scale emotion recognition physiological database for pre-training, and integrates various physiological signals based on the pre-trained emotion feature recognition network to decode the user's emotional state in real time; the music-emotion response calibration module generates a music-emotion response calibration curve for specific music attributes by detecting the individual's emotional response to music with different attributes, providing a basis for the music-based automatic emotion adjustment method; the music generation and emotion adjustment module is based on the generative AI model, music expectation and tension release theory, and adaptively generates and adjusts music according to the user's emotional state and the music-emotion response calibration curve to achieve the cumulative increase or release satisfaction of the user's emotional intensity. The real-time interaction and storage module is used to realize the real-time display and storage of physiological signal waveforms and signal quality, the real-time display and storage of user emotional state evaluation results, and the real-time display and storage of music parameter adjustment.

[0048] The sound output module converts the music electrical signal into vibration through a transducer. In some embodiments, the transducer can be a speaker built into a smart device (smartphone, computer, watch, AR or VR device), a wired speaker, a wireless speaker, a wired headset, a wireless headset, a bone conduction headset, a vibration sound transmission device, a hearing aid, or any other device or apparatus that can interact with the user to generate acoustic information.

[0049] The physiological signal acquisition module collects physiological signals of the user in real time through the sensor device, including but not limited to physiological signals such as EEG signals, ECG signals, breathing signals, heart rate signals, body surface temperature, skin electrical activity signals, or any one or more combinations of other physiological signals that can represent emotional states. In some embodiments, for a certain type of physiological signal, single-point single-form measurement or multi-point multi-form joint measurement can be used. Among them, the EEG signal acquisition device can be an EEG cap, a portable EEG headband, a distributed electrode patch, an ear clip, or a device of any form or measurement principle; the EEG signal, heart rate signal, and breathing signal acquisition device can be a distributed electrode patch, a smart fabric, a smart bracelet, a smart watch, a chest strap, a waist belt, a millimeter wave radar, or a device of any form or measurement principle; the body surface temperature acquisition device can be a device of any form or measurement principle such as a thermistor, a thermocouple, an infrared sensor, a MEMS temperature sensor, or a device; the skin electrical activity (EDA) can be a device of any form or measurement principle such as a bracelet (such as the Empatica E4 bracelet), a finger clip, a patch, a chest strap, a glove, an ear clip, a smart fabric, or a device of any form or measurement principle.

[0050] The signal preprocessing module is used to filter out electromagnetic noise and physiological artifacts during the acquisition process to achieve accurate emotion decoding. In some embodiments, each physiological signal can be high-pass filtered and low-pass filtered to filter out the baseline, power frequency interference and high-frequency noise, retain the signal in the required frequency band, and use wavelet threshold denoising to filter out other physiological artifacts when necessary. This module can be implemented based on any programming language such as C, matlab, python, java, etc. The specific preprocessing operations performed on each physiological signal can be:

[0051] (1) EEG signal: First, the EEG signal is subjected to 0.5 Hz high-pass filtering and 80 Hz low-pass filtering to filter out baseline drift and high-frequency noise in the signal; then, a band-stop filter is performed at a frequency of 50 Hz to filter out power frequency noise; finally, wavelet threshold denoising is performed to remove electrooculographic and electromyographic artifacts in the EEG signal.

[0052] (2) ECG signal and respiratory signal: First, the ECG signal and respiratory signal are subjected to 3-40 Hz band-pass filtering, and then to 50 Hz band-stop filtering.

[0053] (3) Skin electrical activity and body surface temperature: mainly smoothing filtering is performed to filter out high-frequency noise, and the window length is set to 3 points.

[0054] The filters used in the signal preprocessing module can all be FIR digital filters, which perform double-pass filtering, and realize zero-phase shift filtering by inputting the signal forward and reverse into the digital filter.

[0055] The real-time emotion decoding module performs real-time analysis on the pre-processed multimodal physiological signals based on the pre-trained emotion feature recognition network model to accurately decode the user's emotional state (including multiple indicators such as tension, arousal, and valence). In some embodiments, the model pre-training can use public physiological signal emotion databases (such as DEAP, AMIGOS, DREAMER, etc.) and self-built databases as training data sources.

[0056] Preferably, the real-time emotion decoding module extracts fine-grained features from one or more long-term complex physiological signals including EEG signals, ECG signals, respiratory signals and skin electrical activity signals based on the CNN-LSTM network; deeply fuses multimodal emotion features through Transformer, and fuses other simple physiological signals through MLP to achieve real-time decoding of the user's emotional state. Figure 4 As shown, in a preferred embodiment, the specific training process of the module can be:

[0057] (1) Segment the input signal with a 1 second window and 50% overlap to generate a data frame x with a fixed length of N t,w , and Z-normalize each data frame:

[0058]

[0059] Among them, x t,w is the vector form of the original data (such as the time series signal of EEG), μ w is the time series x t,w The mean value, σ w is the time series x t,w The standard deviation of .

[0060] (2) Convolutional neural network (CNN) is used to extract features from multimodal complex physiological signals such as EEG, ECG, respiration, and skin electricity. Specifically, the convolution kernel size of the CNN layer can be set to 3×3, with 32-64 filters per layer, ReLU activation, and Batch Normalization to maintain training stability and accelerate convergence. Each signal branch outputs a 128-dimensional feature vector.

[0061] (3) Combined with long short-term memory network (LSTM), the temporal dependencies of complex physiological signals such as EEG, ECG, respiration, and skin electricity are captured. The LSTM unit has 128 hidden units per layer, and the Dropout is set to 0.3 to prevent overfitting.

[0062] (4) In terms of multimodal fusion, first, the spatiotemporal feature vectors Feature_dim of all complex physiological signals N_modalities are concatenated into a multimodal feature matrix (N_modalities, Feature_dim). Subsequently, the Transformer architecture is used to assign weights to the multimodal features, capture the complex dependencies between different signals, and grasp the correlation and complementarity between different modal signals, so as to achieve accurate evaluation of emotional indicators such as user tension, arousal, and valence. Specifically, the number of attention heads is set to 4, and the dimension of each head is 32, to achieve modeling of inter-modal dependencies and calculate the attention weight of each modality; the feedforward neural network uses 2 fully connected layers (MLP), the hidden layer is 128-dimensional, and the activation function is ReLU; Layer Normalization is used to maintain the stability of the network.

[0063] (5) Finally, the body surface temperature data is incorporated, and the feature vectors derived from all physiological signals are passed through a fully connected layer to output the emotion-related multidimensional feature vector E = {e1, e2, ..., e s}. Among them, s is the total number of dimensions of sentiment evaluation.

[0064] (6) In terms of model training and optimization, the large-scale sentiment dataset is divided into 80% training set, 10% validation set, and 10% test set. The optimizer uses the Adam optimization algorithm. The learning rate is initially set to 0.001 and is dynamically adjusted using the learning rate scheduler. The loss function uses the mean square error (MSE) loss.

[0065] In some embodiments, the music-emotion response calibration module traverses typical music clips that can mobilize user emotions in music databases such as Lakh MIDIDataset and self-built databases, measures the user's real-time emotional response to different types of music clips, and uses it as a multi-dimensional fine-grained emotional label for the same type or specific attribute music. The specific processing flow can be as follows:

[0066] (1) For a music segment M = {M1, M2, ..., M N}, N is the total number of music clips, each clip M i Contains the following music attributes:

[0067] f music,i ={f pitch ,f rhythm ,f harmony ,f loudness ,f duration},

[0068] where f pitch is the pitch attribute, f rhythmis the melody attribute, f harmony is the harmonic property, f loudness is the sound intensity attribute, f duration For duration.

[0069] (2) In order to eliminate the dimensional differences of different features, Z normalization is used to process each feature. For the i-th music clip:

[0070]

[0071] Among them, μ j and σ j are the mean and standard deviation of the j-th music feature respectively.

[0072] (3) Perform hierarchical clustering analysis based on music attributes, and use average link as the aggregation strategy to obtain k clusters (classes). For the nth cluster P n , according to the minimum distance metric principle, traverse the music m in the cluster j ∈P n , search for the cluster center point, and get the typical music closest to the center point attribute:

[0073]

[0074] Among them, d(m j ,P n ) represents m j With cluster P n distance; generate test music group M = {F1, F2, ..., F k}, k is the number of clusters.

[0075] Traverse the test music group M and determine the user's response to the nth typical music clip F n Specific emotional response I n , construct the music-emotion response calibration curve:

[0076]

[0077] Among them, k is the number of clusters, and s is the total number of dimensions of sentiment evaluation.

[0078] The music generation and emotion adjustment module takes music expectation theory as the core of emotion adjustment. Based on the pre-trained music generation model, it generates music that can significantly mobilize the user's emotional state by jointly analyzing the user's current and past emotional states and individualized music-emotion response labels, and establishes the user's expectations to cause the user's roller coaster-like emotional changes. Specifically, in some embodiments, Figure 5 As shown in the figure, the module is divided into three processes: upstream, midstream and downstream.

[0079] (1) The upstream tasks mainly involve the generation of music. Based on the Transformer model, the dynamic properties of music and the time dependence characteristics between fragments can be captured, and constraints can be adaptively constructed to generate music with a coherent style. For example, this process parses the music MIDI file into discrete tokens, encodes the music event sequence into an embedding vector, and based on the Transformer model and the embedded multi-head attention mechanism, self-supervised capture and learning of the short- and long-range dynamic change laws and generation rules of music attributes in music databases such as Lakh MIDIDataset and self-built databases, in order to achieve continuous generation, real-time adjustment and smooth transition of music, and obtain the embedding features of the music sequence. and its position encoding Among them, T is the total number of time steps of music, and d is the music feature encoding dimension.

[0080] (2) The midstream task is used to construct an accurate mapping between music and emotions. This process is based on the music generation Transformer model trained by the upstream task, and uses the individualized music-emotion response calibration curve L emotion To supplement the input, add the conditional vector C emotion =Embedding(L emotion )(The Embedding operation will input data L music Converted into a low-dimensional dense vector representation for subsequent sequence modeling by the Transformer model) and combined it with the music position encoding P music Dynamically aligned and embedded into the Transformer encoding layer of the upstream model:

[0081]

[0082] α=σ(W α [AvgPool(C emotion )])∈(0,1).

[0083] Among them, LayerNorm is a layer normalization operation, which is used to normalize and aggregate features in space; (k is the number of clusters, T is the total number of music time steps) is used to achieve soft alignment of the music-emotion response calibration curve to the music time step, ⊙ represents element-by-element multiplication, P emotion Represents the positional encoding of the music-emotion response calibration curve, X input is the model input, + means concatenating features into vectors, ∥ means merging music information and emotional information into a large input vector, α is an adaptive weight parameter used to prevent emotional conditions from being overly dominant; AvgPool is an average pooling operation used to extract the global emotional intensity in the time dimension; σ is a Sigmoid function, (T is the total number of time steps of the music, and s is the total number of dimensions of emotional evaluation) is used to achieve weight control of the overall influence intensity of emotional conditions.

[0084] This task enables the model to adaptively analyze music (groups) that arouse the user's emotions, and fine-tune the model based on the analysis results to achieve accurate follow-up response of music attributes-emotional state.

[0085] (3) The downstream task is to analyze the user’s emotional state based on music psychology theory and clarify the direction of music adjustment. This process is based on the user’s current emotion E transmitted by the real-time emotion decoding module. current and the set of previous emotional states {E1,E2,…,E current} is input, and the control vector C of the Transformer model is obtained through the Embedding operation dynamic :

[0086] C dynamic =Embedding(E1,E2,…,E current ).

[0087] Then, based on the model constructed in steps (1) to (2), the model parameters are adaptively adjusted according to music psychology theory to generate music that can directionally mobilize the user's emotional state:

[0088] p=f adjust (C dynamic ,Θ),

[0089]

[0090] Among them, f adjust It is a learnable mapping function based on music psychology theory. Its main functions and effects are as follows: Figure 1 As shown, specifically: when it is determined that the user's nervous state is still on the rise, the fine-tuning model outputs the corresponding music to continue to accumulate expectations; when the user's nervous state stops rising or rises more slowly, the music generation parameters are converted to release the user's expectations; Θ is the specific parameter of the Transformer model obtained after training in steps (1) to (2); p is the conditional probability distribution of music attributes generated according to music psychology theory; is the loss function for generating music, T is the total number of time steps, s t is the expected emotional state at time step t. Specifically, Represents a given previous state and dynamic condition C dynamic Next, generate s at time step t t probability.

[0091] The real-time interaction and storage module includes a user interaction and operation interface, as well as a storage medium for storing physiological data and music parameters generated during use. The functions implemented by this module include: real-time display and storage of physiological signal waveforms and signal quality, real-time display and storage of user emotional state assessment results, and real-time display and storage of music parameter adjustments. Specifically, this module can be implemented based on various smart terminals such as mobile phones, tablets, computers, watches, AR or VR headsets, and can also be configured in a remote cloud server.

[0092] The embodiment of the present invention also discloses a generative AI-driven physiological-music follow-response intelligent emotion regulation method, comprising the following steps: collecting multimodal physiological signals of users during the process of listening to music; preprocessing various physiological signals to preliminarily filter out noise in the signals; decoding the user's emotional state in real time based on the pre-trained physiological signal analysis network; generating a music-emotion response calibration curve by detecting the individual's emotional response to music with different attributes; based on the generative AI model, music expectation and tension release theory, adaptively generating and adjusting music according to the user's emotional state and the music-emotion response calibration curve. The specific implementation details of each step refer to the implementation of the system module of the aforementioned embodiments, which will not be repeated here.

[0093] An embodiment of the present invention also discloses a computer system, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, the steps of a generative AI-driven physiological-music following response intelligent emotion regulation method are implemented.

[0094] An embodiment of the present invention also discloses a computer program product, including a computer program, which, when executed by a processor, implements the steps of a generative AI-driven physiological-music follow-response intelligent emotion regulation method.

[0095] The embodiments are only for illustrating the technical idea of ​​the present invention and cannot be used to limit the protection scope of the present invention. Any changes made on the basis of the technical solution in accordance with the technical idea proposed by the present invention shall fall within the protection scope of the present invention.

Claims

1. A generative AI-driven physiological-music-following intelligent emotion regulation system, characterized in that: include: A physiological signal acquisition module is used to collect multimodal physiological signals of users when they listen to music; The signal preprocessing module is used to preprocess various physiological signals and initially filter out noise in the signals; The real-time emotion decoding module is used to decode the user's emotional state in real time based on the pre-trained emotion feature recognition network; A music-emotion response calibration module is used to generate a music-emotion response calibration curve by detecting an individual's emotional response to music of different attributes; The music generation and emotion adjustment module is used to adaptively generate and adjust music according to the user's emotional state and the music-emotion response calibration curve based on the generative AI model, music expectation and tension release theory.

2. According to claim 1, a generative AI-driven physiological-music following response intelligent emotion regulation system is characterized in that: The multimodal physiological signals include one or more of EEG signals, ECG signals, respiratory signals, heart rate signals, body surface temperature, facial expressions and skin electrical activity signals; the denoising processing of each physiological signal in the signal preprocessing module includes a combination of one or more of high-pass filtering, low-pass filtering and wavelet threshold denoising methods.

3. According to claim 1, a generative AI-driven physiological-music following response intelligent emotion regulation system is characterized in that: In the real-time emotion decoding module, one or more physiological signals including EEG signals, ECG signals, breathing signals, heart rate signals, body surface temperature, facial expressions and skin electrical activity signals are integrated to decode the user's emotional state in real time; fine-grained features in one or more long-term complex physiological signals including EEG signals, ECG signals, breathing signals and skin electrical activity signals are extracted based on the CNN-LSTM network; Through the deep fusion of multimodal emotional features through Transformer, and the fusion of other simple physiological signals through MLP, the real-time decoding of the user's emotional state is realized, and the emotional response E = {e1, e2, ..., e s }, where s is the total number of dimensions of sentiment evaluation.

4. According to claim 1, a generative AI-driven physiological-music following response intelligent emotion regulation system is characterized in that: In the music-emotion response calibration module, a typical music clip is determined by hierarchical clustering analysis of multiple attributes in a large-scale music data set, and a random test music group M = {F1, F2, ..., F k }, where k is the number of clusters; in the process of playing the music group, the user's response to the nth music clip F in the music group M is measured n Specific emotional response Constructing music-emotion response calibration curve Where s is the total number of emotion evaluation dimensions.

5. According to claim 1, a generative AI-driven physiological-music following response intelligent emotion regulation system is characterized in that: The music generation and emotion adjustment module captures the dynamic properties of music and the time dependency characteristics between segments based on the Transformer model, adaptively constructs constraints, and generates music with a coherent style.

6. A generative AI-driven physiological-music following response intelligent emotion regulation system according to claim 1, characterized in that: The music generation and emotion adjustment module, based on the pre-trained music generation model, calibrates the user's music-emotion response curve L emotion ={I1,I2,...,I k } is embedded into the encoding layer of the music generation model to achieve the follow-up response of music attribute-emotional state: C em otion=Embedding(L emotion ), Where α=σ(W α [AvgPool(C emotion )])∈(0,1),C em otion is the conditional vector of the music generation model, It is used to realize the soft alignment of the music-emotion response calibration curve to the music time step, ⊙ represents element-by-element multiplication, k is the number of clusters, T is the total number of music time steps, P music Represents the music position code, C music represents the music sequence embedding feature, P emotion Represents the positional encoding of the music-emotion response calibration curve, X input is the model input, + means concatenating features into a vector, ∥ means merging music information and emotional information into a large input vector, α is the adaptive weight parameter, σ is the Sigmoid function, It is used to achieve weight control of the overall impact intensity of emotional conditions, and s is the total number of dimensions of emotional evaluation.

7. A generative AI-driven physiological-music-following-response intelligent emotion regulation system according to claim 1, characterized in that: The music generation and emotion adjustment module is based on the trained music generation model and the user's current emotion E current and the set of previous emotional states {E1,E2,…,E current } and music psychology theory, adaptively adjust model parameters to generate music that can directional mobilize the user's emotional state: C dynamic =Embedding(E1,E2,…,E current ), p=f adjust (C dynamic ,I), Among them, f adjust is a learnable mapping function based on music psychology theory; Θ is the specific parameter of the trained music generation model; p is the conditional probability distribution of music attributes generated according to music psychology theory; is the loss function for generating music, T is the total number of time steps of the music, s t is the expected emotional state at time step t, Represents a given previous state and dynamic condition C dynamic Next, generate s at time step t t probability.

8. A generative AI-driven physiological-music following response intelligent emotion regulation method, characterized in that: The steps include: Collect multimodal physiological signals of users while listening to music; Preprocess various physiological signals and initially filter out noise in the signals; Decode the user's emotional state in real time based on the pre-trained physiological signal analysis network; By detecting the emotional response of individuals to music with different attributes, a music-emotion response calibration curve is generated; Based on the generative AI model, music expectation and tension release theory, music is adaptively generated and adjusted according to the user's emotional state and the music-emotion response calibration curve.

9. A computer system comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the computer program is executed by a processor, the steps of a generative AI-driven physiological-music-following-response intelligent emotion regulation method according to claim 8 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of a generative AI-driven physiological-music-following-response intelligent emotion regulation method according to claim 8 are implemented.

Images