A neural network-based real-time recommendation system for decompression music

Abstract

The application belongs to the field of music recommendation generation, and specifically discloses a real-time stress-relief music recommendation system based on a neural network, which comprises an electroencephalogram signal acquisition module, an electroencephalogram signal preprocessing module, an electroencephalogram feature extraction module, an emotion recognition module, a music emotion anchoring module, a recommendation matching module, a closed-loop dynamic adjustment module and an audio output feedback module.The scheme adopts three-dimensional frequency-spatiotemporal convolution and inflation causal time convolution to extract local spatial spectral features and long-range time dependence of electroencephalogram in parallel, introduces a window probability sparse self-attention mechanism to reduce the computational complexity to meet the real-time requirement, applies model predictive control theory to music recommendation, dynamically adjusts the playing strategy by solving a multi-objective optimization problem in a rolling time domain, forms a perception, execution and feedback closed loop, and realizes high-precision, low-delay, self-adaptive and continuously-adjusted stress-relief music recommendation.

Description

Technical Field

[0001] This invention relates to the field of music recommendation, specifically to a real-time recommendation system for stress-relieving music based on neural networks. Background Technology

[0002] With increasing competition in modern society, negative emotions such as anxiety and depression have become major threats to public mental health. Music, as a non-invasive means of emotion regulation, has been proven to effectively influence the autonomic nervous system and the limbic system of the brain. Existing mainstream systems, such as collaborative filtering and content-based recommendation, heavily rely on users' historical playback behavior. These static preferences cannot reflect the dynamic needs of users due to changes in stress levels at specific moments, leading to recommendation results that often contradict the user's true purpose of relieving stress through music. Existing emotion recognition methods involving EEG, such as the baseline methods on the DEAP dataset, mostly extract only statistical features such as power spectral density or differential entropy, failing to utilize three-dimensional convolutional kernels to simultaneously capture transient changes and spatial synchronicity on the time axis, spatial channel axis, and frequency axis. Summary of the Invention

[0003] To address the above issues and overcome the shortcomings of existing technologies, this invention provides a real-time stress-relief music recommendation system based on neural networks. Addressing the problem of insufficient multi-dimensional feature extraction from EEG signals, this invention constructs a dual-path parallel network: Path one uses a three-dimensional frequency-spatiotemporal convolutional kernel to simultaneously extract the instantaneous frequency features, spatial distribution features, and temporal evolution features of the EEG signal; Path two employs a stacked dilated causal convolutional network to exponentially expand the receptive field without increasing computational load, capturing long-range temporal dependencies. To address the low computational efficiency and high latency of the Transformer algorithm, this invention introduces a window probability sparse self-attention mechanism. By limiting the attention calculation range to a local time window and combining it with query sparsity evaluation, computational complexity is reduced, meeting real-time requirements. Addressing the lack of dynamic optimization and closed-loop feedback in music recommendation, this invention innovatively introduces model predictive control into the field of music recommendation. Using the user's current emotion vector as the state variable and the acoustic feature adjustment of the music as the control variable, a multi-objective cost function is constructed, including emotion tracking error, control energy consumption, and control increment. The optimal control sequence is then solved through rolling time domain calculation.

[0004] The technical solution adopted by the present invention is as follows: The present invention provides a real-time recommendation system for stress-relieving music based on neural networks. The real-time recommendation system for stress-relieving music based on neural networks includes an EEG signal acquisition module, an EEG signal preprocessing module, an EEG feature extraction module, an emotion recognition module, a music emotion anchoring module, a recommendation matching module, a closed-loop dynamic adjustment module, and an audio output feedback module.

[0005] The EEG signal acquisition module uses a wearable multi-channel dry electrode EEG device to acquire EEG signals. After analog-to-digital conversion, the EEG signals are wirelessly transmitted to the EEG signal preprocessing module in the form of data packets.

[0006] The EEG signal preprocessing module first performs bandpass and notch filtering to denoise the EEG signal, outputting a clean time-domain signal, and then sends it to independent component decomposition to remove artifacts of electrooculography and electromyography, reconstructing the EEG, segmenting the EEG, performing short-time Fourier transform on each segment, and generating a three-dimensional time-spectrum tensor.

[0007] The EEG feature extraction module extracts deep emotional semantic features contained in the three-dimensional time-frequency tensor through a dual-path parallel deep network. One path focuses on the local time-frequency-space joint mode, while the other path focuses on the global long-range time dependence. The deep emotional semantic features of the two paths are fused to obtain the EEG deep fusion vector.

[0008] The emotion recognition module performs global context modeling on the deep fusion vector of EEG, and regresses the continuous coordinate values ​​of the user in the two-dimensional emotion space in real time to obtain the user's real-time emotion vector.

[0009] The music emotion anchoring module offline analyzes the audio signal of each track in the music library, extracts its underlying acoustic physical properties, and maps them to emotional coordinates consistent with the user's emotional space to obtain music emotion anchor points.

[0010] The recommendation matching module calculates the psychoacoustic distance between the user's real-time emotional vector and the emotional anchor point of each song, and sorts the candidate songs that best match the user's current emotional state according to the degree of closeness.

[0011] The closed-loop dynamic adjustment module transforms music recommendation into a finite-time-domain rolling optimization control problem by continuously monitoring user emotional feedback and dynamically adjusting the playback strategy.

[0012] The audio output feedback module converts digital audio signals into analog signals to drive the playback device to output sound, while monitoring the playback progress and triggering a new round of control calculations at key time points.

[0013] Furthermore, the EEG signal preprocessing module includes a filtering and denoising unit, an artifact removal unit, a data segmentation and enhancement unit, and a time-frequency transformation unit;

[0014] The filtering and denoising unit, through the synergistic effect of DC removal, bandpass filtering, and notch filtering, suppresses baseline drift, high-frequency electromyography (EMG) interference, and power frequency interference, while maintaining waveform distortion-free output with the help of zero-phase technology, producing a clean time-domain signal. Specifically, it removes DC offset from the EEG signal using the following formula: ;

[0015] In the formula, After removing the DC component The amplitude of the EEG signal at any given moment. yes The amplitude of the raw EEG signal at that moment. This is the total number of sampling points. It is the sampling point number;

[0016] A zero-phase FIR filter is used, and bandpass filtering is employed to remove high-frequency electromyographic noise and extremely low-frequency drift.

[0017] A notch filter is used to eliminate power frequency interference and obtain a filtered signal.

[0018] The artifact removal unit uses independent component analysis to perform blind source separation on the pure time-domain signal to obtain an independent component matrix and a mixture matrix. By calculating the Pearson correlation coefficient between each independent component and the standard reference signal, it identifies and removes the electrooculogram (EOG) and electromyogram (EMG) artifact components. The remaining components are projected back into the signal space through the inverse matrix of the mixture matrix to obtain the pure EEG signal.

[0019] The data segmentation enhancement unit uses a sliding window to segment the pure EEG signal to obtain segmented signals, and all segmented signals constitute a sample set.

[0020] The time-frequency transformation unit performs a short-time Fourier transform on each segment of the signal, mapping the one-dimensional time series into a two-dimensional time-frequency spectrum. The calculation formula for the short-time Fourier transform is as follows: ;

[0021] In the formula, It is a two-dimensional time-frequency spectrum. It is the window length. It is the Hamming window function. It is a frequency index. It is the time offset within the window.

[0022] The EEG feature extraction module includes a three-dimensional frequency-spatiotemporal convolutional network unit, a dilated causal temporal convolutional network unit, and a dual-path feature fusion unit;

[0023] The three-dimensional frequency-spatiotemporal convolutional network unit maps the preprocessed two-dimensional temporal spectrum and the electrode spatial topological position into a three-dimensional tensor to obtain spatial-frequency fusion features;

[0024] The 3D convolution kernel slides simultaneously in three dimensions to perform convolution, and the convolution formula used is as follows: ;

[0025] In the formula, It is the output feature map in channel offset Frequency offset Time offset The value at that location, , and The dimensions of the convolution kernel are defined by its size in the channels, frequency, and time dimensions. These are learnable convolutional kernel weights. It is a bias term;

[0026] The output feature map is compressed by convolution, and information from various dimensions is fused to generate an intermediate feature representation.

[0027] The dilated causal temporal convolutional network unit captures the evolution trend of emotions over a long time axis. It employs a dilated causal convolutional structure to ensure that the output at the current moment depends only on information from historical moments, thus obtaining temporal depth features. The discrete calculation formula for dilated causal convolution is as follows: ;

[0028] In the formula, It is the first The output value of each neuron It is the kernel size. It is the kernel element index. These are the convolution kernel weights. Indicates the time step from the current time. Forward offset The input value of the step, It is the expansion factor, which increases exponentially with the number of network layers;

[0029] The dual-path feature fusion unit concatenates the spatial-frequency fusion features output by the three-dimensional frequency-spatiotemporal convolutional network unit with the temporal depth features output by the dilated causal temporal convolutional network unit along the feature axis, and compresses them to a fixed dimension through a fully connected mapping layer to generate the final EEG depth fusion vector.

[0030] The emotion recognition module maps the deep fusion vector of EEG to a two-dimensional emotion space, including a location encoding unit, a window probability sparse self-attention unit, and an emotion regression output unit.

[0031] The location encoding unit uses sine-cosine absolute location encoding to inject temporal information into the EEG deep fusion vector, and the formula used is as follows: ; ;

[0032] In the formula, It is the sequence position index of the deep EEG fusion vector. It is a dimensional index. It is the feature dimension;

[0033] The window probability sparse self-attention unit obtains the query matrix, key matrix, and value matrix from the input sequence through linear projection.

[0034] Window constraint: Set a fixed window radius of R, and calculate attention for each query only with its 2R nearest neighbors to filter out irrelevant noise from distant locations;

[0035] Probabilistic sparsity: For each query vector, calculate its attention score on all keys within the window, retain only the top u key-value pairs with the highest scores, and set a sparsity threshold.

[0036] The formula for calculating the output of single-head attention is defined as follows: ;

[0037] In the formula, It is the query matrix after window mask sparsification. It is the query matrix after probability sparsification. It is the dimension of the key vector. It is a single-head attention output. , , These are the query matrix, key matrix, and value matrix. This indicates transpose.

[0038] The emotion regression output unit maps the global context vector output by the Transformer encoder through two fully connected network layers. The output layer uses the Sigmoid activation function to calculate valence and arousal, respectively. The formulas for calculating valence and arousal are as follows: ; ;

[0039] In the formula, It is valence. It's wakefulness. It is the Sigmoid activation function. It is a valence weight matrix. It is the wake-up weight matrix. It is a valence bias term. It is the arousal bias term. It is the global context vector;

[0040] The music emotion anchoring module constructs a high-dimensional acoustic feature vector for the music database and maps it to an emotion anchoring space that is isomorphic to the user's emotional space. It includes a low-level acoustic feature extraction unit and a music emotion regression mapping unit.

[0041] The underlying acoustic feature extraction unit performs frame-by-frame windowing processing on the original audio PCM stream to extract audio features, including beat rate, spectral centroid, Mel frequency cepstral coefficients, and reverberation time.

[0042] The music emotion regression mapping unit constructs an auxiliary regression network that maps the extracted acoustic feature vectors into emotion coordinates. This mapping network adopts a pre-training-fine-tuning paradigm and uses a large-scale manually labeled dataset as supervision.

[0043] The recommendation matching module calculates the weighted Euclidean distance between the user's real-time emotion vector and the music emotion anchor points of the tracks in the music library as the emotion distance, and selects the K tracks with the smallest emotion distance as the initial candidate set.

[0044] The closed-loop dynamic adjustment module upgrades music recommendation from static matching to a dynamic adjustment process. It uses user emotions as the controlled variable and music playback parameters as the manipulated variable to construct a finite-time domain rolling optimization problem, including a system state-space modeling unit and a rolling time domain solution and execution unit.

[0045] The system state space modeling unit uses a recurrent neural network to establish an emotion state predictor, and defines the input vector at time k, including the user's real-time emotion vector at the previous time, music track switching instructions, playback volume gain coefficient, environmental noise interference, and user autonomous operation.

[0046] The emotion state predictor recursively predicts the user's emotional response at future moments, using the following formula: ;

[0047] In the formula, It is a prediction of user sentiment response. It predicts the time domain. It is the length of the input sequence;

[0048] The rolling time-domain solver and execution unit performs the following operations in each control cycle:

[0049] The user's real-time emotion vector is obtained by the emotion recognition module, the future system output is predicted by the RNN surrogate model, and the above optimization problem is solved by calling the sequence quadratic programming algorithm to obtain the optimal control sequence.

[0050] Only the first element of the optimal sequence is sent to the audio output module;

[0051] When the next moment arrives, the state is resampled, and the above steps are repeated to achieve closed-loop dynamic tracking and compensation.

[0052] The audio output feedback module executes control commands, decodes digital audio signals into analog signals to drive the speaker, and monitors the playback progress in real time. When the song reaches the end, it actively requests the next control command from the MPC controller to ensure that there is no delay between song playback intervals.

[0053] The beneficial effects achieved by the present invention using the above solution are as follows:

[0054] (1) To address the problem of insufficient multidimensional feature extraction of EEG signals, this invention constructs a dual-path parallel network: Path 1 uses a three-dimensional frequency-spatiotemporal convolution kernel to simultaneously extract the instantaneous frequency features, spatial distribution features and temporal evolution features of EEG signals; Path 2 uses a stacked dilated causal convolutional network to exponentially expand the receptive field without increasing the computational load and capture long-term temporal dependence.

[0055] (2) To address the issues of low computational efficiency and high latency in emotion recognition of Transformer, this invention introduces a window probability sparse self-attention mechanism. By limiting the attention computation range to a local time window and combining it with Query sparsity evaluation, the computational complexity is reduced and the real-time requirements are met.

[0056] (3) In view of the lack of dynamic optimization and closed-loop feedback in music recommendation, this invention innovatively introduces model predictive control into the field of music recommendation. The current emotional vector of the user is used as the state variable and the adjustment of the acoustic features of the music is used as the control variable. A multi-objective cost function including emotional tracking error, control energy consumption and control increment is constructed, and the optimal control sequence is solved by rolling time domain. Attached Figure Description

[0057] Figure 1 The present invention provides a module connection diagram for a real-time stress-relieving music recommendation system based on a neural network.

[0058] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof. Detailed Implementation

[0059] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0060] Example 1, see Figure 1This invention provides a real-time stress-relief music recommendation system based on neural networks. The system includes an EEG signal acquisition module, an EEG signal preprocessing module, an EEG feature extraction module, an emotion recognition module, a music emotion anchoring module, a recommendation matching module, a closed-loop dynamic adjustment module, and an audio output feedback module.

[0061] The EEG signal acquisition module uses a wearable multi-channel dry electrode EEG device, with electrodes laid out according to the international 10-20 system, a sampling frequency fs≥256Hz, and a quantization bit depth of 16 bits to acquire EEG signals. After analog-to-digital conversion, the EEG signals are wirelessly transmitted to the preprocessing module in the form of data packets.

[0062] The EEG signal preprocessing module standardizes the raw signal into a high signal-to-noise ratio tensor suitable for deep network input.

[0063] The EEG feature extraction module extracts deep emotional semantic features contained in EEG signals through a dual-path parallel deep network. One path focuses on the local time-frequency-space joint mode, while the other path focuses on the global long-range time dependence. The deep emotional semantic features of the two paths are fused to obtain the EEG deep fusion vector.

[0064] The emotion recognition module performs global context modeling on the deep fusion vector of EEG, and regresses the continuous coordinate values ​​of the user in the two-dimensional emotion space in real time to obtain the user's real-time emotion vector.

[0065] The music emotion anchoring module offline analyzes the audio signal of each track in the music library, extracts its underlying acoustic physical properties, and maps them to emotional coordinates consistent with the user's emotional space to obtain music emotion anchor points.

[0066] The recommendation matching module calculates the emotional distance between the user's real-time emotional vector and the emotional anchor point of each song, and sorts the candidate songs that best match the user's current emotional state according to the degree of closeness.

[0067] The closed-loop dynamic adjustment module transforms music recommendation into a finite-time-domain rolling optimization control problem by continuously monitoring user emotional feedback and dynamically adjusting the playback strategy.

[0068] The audio output feedback module converts digital audio signals into analog signals to drive the playback device to output sound, while monitoring the playback progress and triggering a new round of control calculations at key time points.

[0069] Example 2, see Figure 1 This embodiment is based on the above embodiment. The EEG signal preprocessing module includes a filtering and denoising unit, an artifact removal unit, a data segmentation and enhancement unit, and a time-frequency transformation unit.

[0070] The filtering and denoising unit specifically operates by removing DC offset from the EEG signal, using the following formula: ;

[0071] In the formula, After removing the DC component The amplitude of the EEG signal at any given moment. yes The amplitude of the raw EEG signal at that moment. This is the total number of sampling points. It is the sampling point number;

[0072] A zero-phase FIR filter is used to retain the frequency band from 0.5Hz to 50Hz, and bandpass filtering is used to remove high-frequency electromyographic noise and extremely low-frequency drift.

[0073] A notch filter is set at 50Hz. By filtering the signal through the notch filter, the power frequency interference is eliminated and the filtered signal is obtained.

[0074] The artifact removal unit uses independent component analysis to perform blind source separation on the filtered signal, obtaining an independent component matrix and a mixture matrix. By calculating the Pearson correlation coefficient between each independent component and the standard reference signal, it identifies and removes electrooculography (EOG) and electromyography (EMG) artifact components. The remaining components are projected back into the signal space through the inverse matrix of the mixture matrix to obtain a pure EEG signal.

[0075] The data segmentation enhancement unit uses a sliding window with a length of 2 seconds and an overlap rate of 50% to segment the pure EEG signal to obtain segmented signals, and all segmented signals constitute a sample set.

[0076] The time-frequency transformation unit performs a short-time Fourier transform on each segment of the signal, mapping the one-dimensional time series to a two-dimensional time-frequency spectrum, which serves as the input to the subsequent three-dimensional convolutional network. The calculation formula for the short-time Fourier transform is as follows: ;

[0077] In the formula, It is a two-dimensional time-frequency spectrum. It is the window length. It is the Hamming window function. It is a frequency index. It is the time offset within the window.

[0078] Example 3, see Figure 1 This embodiment is based on the above embodiment, and the EEG feature extraction module includes a three-dimensional frequency-spatiotemporal convolutional network unit, a dilated causal temporal convolutional network unit, and a dual-path feature fusion unit;

[0079] The three-dimensional frequency-spatiotemporal convolutional network unit maps the preprocessed two-dimensional temporal spectrum map and the electrode spatial topological position into channel dimension, frequency dimension, and time dimension to obtain spatial-frequency fusion features.

[0080] The 3D convolution kernel slides simultaneously in three dimensions to perform convolution, and the convolution formula used is as follows: ;

[0081] In the formula, It is the output feature map in channel offset Frequency offset Time offset The value at that location, , and The dimensions of the convolution kernel are 3, 8, and 5 in the channel, frequency, and time dimensions, respectively. In this scheme, we use 3, 8, and 5, respectively. These are learnable convolutional kernel weights. It is a bias term;

[0082] The output feature map is compressed using a 1×1 1×1 convolution to fuse information from various dimensions and generate an intermediate feature representation.

[0083] The dilated causal temporal convolutional network unit captures the evolution trend of emotions over a long time axis. It employs a dilated causal convolutional structure to ensure that the output at the current moment depends only on information from historical moments, avoiding the leakage of future information and obtaining temporal depth features. The discrete calculation formula for dilated causal convolution is as follows: ;

[0084] In the formula, It is the first The output value of each neuron It is the kernel size. It is the kernel element index. These are the convolution kernel weights. Indicates the time step from the current time. Forward offset The input value of the step, It is the expansion factor, which increases exponentially with the number of network layers;

[0085] The dual-path feature fusion unit concatenates the spatial-frequency fusion features output by the three-dimensional frequency-spatiotemporal convolutional network unit with the temporal depth features output by the dilated causal temporal convolutional network unit along the feature axis, and compresses them to a fixed dimension through a fully connected mapping layer to generate the final EEG depth fusion vector, which contains a holographic representation of emotion-induced EEG.

[0086] Example 4, see Figure 1This embodiment is based on the above embodiment. The emotion recognition module quickly maps the deep fusion vector of EEG to a two-dimensional emotion space, including a location encoding unit, a window probability sparse self-attention unit, and an emotion regression output unit.

[0087] The location encoding unit uses sine-cosine absolute location encoding to inject temporal information into the EEG deep fusion vector, and the formula used is as follows: ; ;

[0088] In the formula, It is the sequence position index of the deep EEG fusion vector. It is a dimensional index. It is the feature dimension;

[0089] The window probability sparse self-attention unit obtains the query matrix, key matrix, and value matrix from the input sequence through linear projection.

[0090] Window constraint: Set a fixed window radius of R, and calculate attention for each query only with its 2R nearest neighbors to filter out irrelevant noise from distant locations;

[0091] Probabilistic sparsity is achieved by calculating the attention score of each query vector on all keys within the window, retaining only the top u key-value pairs with the highest scores, setting the sparsity threshold to 0.25, and setting the rest to zero.

[0092] The formula for calculating single-head attention output is as follows: ;

[0093] In the formula, It is the query matrix after window mask sparsification. It is the query matrix after probability sparsification. It is the dimension of the key vector. It is a single-head attention output. , , These are the query matrix, key matrix, and value matrix. This indicates transposition, and the multi-head mechanism executes it in parallel 8 times, concatenating the results for output.

[0094] The emotion regression output unit maps the global context vector output by the Transformer encoder through two fully connected network layers. The output layer uses the Sigmoid activation function to calculate valence and arousal, respectively. The formulas for calculating valence and arousal are as follows: ; ;

[0095] In the formula, It is valence. It's wakefulness. It is the Sigmoid activation function. It is a valence weight matrix. It is the wake-up weight matrix. It is a valence bias term. It is the arousal bias term. It is the global context vector.

[0096] Example 5, see Figure 1 This embodiment is based on the above embodiment. The music emotion anchoring module constructs a high-dimensional acoustic feature vector for the music database and maps it to an emotion anchoring space that is isomorphic to the user's emotional space. It includes a low-level acoustic feature extraction unit and a music emotion regression mapping unit.

[0097] The underlying acoustic feature extraction unit performs frame-by-frame windowing processing on the original audio PCM stream, with 2048 sampling points per frame and a frame shift of 512 points, and extracts the following features:

[0098] Beat rate: The number of beats per minute is detected by the autocorrelation function;

[0099] Spectral centroid: measures the brightness of audio, calculated using the following formula: ;

[0100] In the formula, It is the first Frequency values ​​at each frequency point It is the corresponding spectral amplitude. It is the total number of frequency points. It is the centroid of the spectrum;

[0101] Mel frequency cepstral coefficients: Extract the first 13 dimensions of coefficients to characterize timbre;

[0102] Reverberation time: Measures the time required for sound to decay by 60 dB;

[0103] The music emotion regression mapping unit constructs an auxiliary regression network that maps the extracted acoustic feature vectors into emotion coordinates. This mapping network adopts a pre-training-fine-tuning paradigm and uses a large-scale manually labeled dataset as supervision.

[0104] Example 6, see Figure 1 This embodiment is based on the above embodiment. The recommendation matching module calculates the weighted Euclidean distance between the user's real-time emotion vector and the music emotion anchor points of the tracks in the music library as the emotion distance, and selects the 10 tracks with the smallest emotion distance as the initial candidate set.

[0105] Example 7, see Figure 1 This embodiment is based on the above embodiment. The closed-loop dynamic adjustment module upgrades music recommendation from static matching to a dynamic adjustment process. Drawing on predictive control concepts from industrial process control, it uses user emotion as the controlled variable and music playback parameters as manipulated variables to construct a finite-time rolling optimization problem, including a system state-space modeling unit.

[0106] The system state space modeling unit uses a recurrent neural network to establish an emotion state predictor, and defines the input vector at time k, including the user's real-time emotion vector at the previous time, music track switching instructions, playback volume gain coefficient, environmental noise interference, and user autonomous operation.

[0107] The emotion state predictor recursively predicts the user's emotional response at future moments, using the following formula: ;

[0108] In the formula, It is a prediction of user sentiment response. It predicts the time domain. It is the length of the input sequence;

[0109] The rolling time-domain solution and execution performs the following operations in each control cycle:

[0110] The user's real-time emotion vector is obtained by the emotion recognition module, the future system output is predicted by the RNN surrogate model, and the above optimization problem is solved by calling the sequence quadratic programming algorithm to obtain the optimal control sequence.

[0111] Only the first element of the optimal sequence is sent to the audio output module;

[0112] When the next moment arrives, the state is resampled, and the above steps are repeated to achieve closed-loop dynamic tracking and compensation.

[0113] Example 8, see Figure 1 This embodiment is based on the above embodiment. The audio output feedback module executes control commands to decode digital audio signals into analog signals to drive the speaker. At the same time, it monitors the playback progress in real time. When the song reaches the end, it actively requests the next control command from the MPC controller to ensure that there is no delay between song playback intervals.

[0114] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0115] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

[0116] The present invention and its embodiments have been described above. This description is not restrictive, and the accompanying drawings are only one embodiment of the present invention; the actual structure is not limited thereto. In conclusion, if those skilled in the art are inspired by this description and design similar structures and embodiments without departing from the spirit of the invention, such designs should fall within the protection scope of the present invention.

Claims

1. A real-time stress-relieving music recommendation system based on neural networks, characterized in that, The system includes an EEG signal acquisition module, an EEG signal preprocessing module, an EEG feature extraction module, an emotion recognition module, a music emotion anchoring module, a recommendation matching module, a closed-loop dynamic adjustment module, and an audio output feedback module. The EEG signal acquisition module uses a wearable multi-channel dry electrode EEG device to acquire EEG signals. After analog-to-digital conversion, the EEG signals are wirelessly transmitted to the EEG signal preprocessing module in the form of data packets. The EEG signal preprocessing module first performs bandpass and notch filtering to denoise the EEG signal, outputting a clean time-domain signal, and then sends it to independent component decomposition to remove artifacts of electrooculography and electromyography, reconstructing the EEG, segmenting the EEG, performing short-time Fourier transform on each segment, and generating a three-dimensional time-spectrum tensor. The EEG feature extraction module extracts deep emotional semantic features contained in the three-dimensional time-spectrum tensor through a dual-path parallel deep network, and fuses the deep emotional semantic features of the two paths to obtain an EEG deep fusion vector. The emotion recognition module performs global context modeling on the deep fusion vector of EEG, and regresses the continuous coordinate values ​​of the user in the two-dimensional emotion space in real time to obtain the user's real-time emotion vector. The music emotion anchoring module offline analyzes the audio signal of each track in the music library, extracts its underlying acoustic physical properties, and maps them to emotional coordinates consistent with the user's emotional space to obtain music emotion anchor points. The recommendation matching module calculates the psychoacoustic distance between the user's real-time emotional vector and the emotional anchor point of each song, and sorts the candidate songs that best match the user's current emotional state according to the degree of closeness. The closed-loop dynamic adjustment module transforms music recommendation into a finite-time-domain rolling optimization control problem by continuously monitoring user emotional feedback and dynamically adjusting the playback strategy. The audio output feedback module converts digital audio signals into analog signals to drive the playback device to output sound, while monitoring the playback progress and triggering a new round of control calculations at key time points.

2. The real-time stress-relieving music recommendation system based on neural networks according to claim 1, characterized in that, The EEG signal preprocessing module includes a filtering and noise reduction unit, an artifact removal unit, a data segmentation and enhancement unit, and a time-frequency transformation unit. The filtering and denoising unit, through the combined effects of DC removal, bandpass and notch filtering, and with the help of zero-phase technology, outputs a clean time-domain signal. A zero-phase FIR filter is used, and bandpass filtering is employed to remove high-frequency electromyographic noise and extremely low-frequency drift. A notch filter is used to eliminate power frequency interference and obtain a filtered signal. The artifact removal unit uses independent component analysis to perform blind source separation on the pure time-domain signal to obtain an independent component matrix and a mixture matrix. By calculating the Pearson correlation coefficient between each independent component and the standard reference signal, it identifies and removes the electrooculogram (EOG) and electromyogram (EMG) artifact components. The remaining components are projected back into the signal space through the inverse matrix of the mixture matrix to obtain the pure EEG signal. The data segmentation enhancement unit uses a sliding window to segment the pure EEG signal to obtain segmented signals, and all segmented signals constitute a sample set. The time-frequency conversion unit performs a short-time Fourier transform on each segmented signal, mapping the one-dimensional time series into a two-dimensional time-frequency spectrum.

3. The real-time stress-relieving music recommendation system based on neural networks according to claim 1, characterized in that, The EEG feature extraction module includes a three-dimensional frequency-spatiotemporal convolutional network unit, a dilated causal temporal convolutional network unit, and a dual-path feature fusion unit; The three-dimensional frequency-spatiotemporal convolutional network unit maps the preprocessed two-dimensional temporal spectrum and the electrode spatial topological position into a three-dimensional tensor to obtain spatial-frequency fusion features; The three-dimensional convolution kernel slides simultaneously in three dimensions to perform convolution to obtain the output feature map. The output feature map is then compressed through convolution operations to fuse information from each dimension and generate an intermediate feature representation.

4. The real-time stress-relieving music recommendation system based on neural networks according to claim 3, characterized in that, The dilated causal temporal convolutional network unit captures the evolution trend of emotions over a long time axis. It employs a dilated causal convolutional structure to obtain temporal depth features. The discrete calculation formula for dilated causal convolution is as follows: ； In the formula, It is the first The output value of each neuron It is the kernel size. It is the kernel element index. These are the convolution kernel weights. Indicates the time step from the current time. Forward offset The input value of the step, It is the expansion factor, which increases exponentially with the number of network layers; The dual-path feature fusion unit concatenates spatial-frequency fusion features and temporal depth features along the feature axis, and compresses them to a fixed dimension through a fully connected mapping layer to generate the final EEG deep fusion vector.

5. The real-time stress-relieving music recommendation system based on neural networks according to claim 1, characterized in that, The emotion recognition module maps the deep fusion vector of EEG to a two-dimensional emotion space, including a location encoding unit, a window probability sparse self-attention unit, and an emotion regression output unit. The location encoding unit uses sine-cosine absolute location encoding to inject temporal information into the EEG deep fusion vector; The window probability sparse self-attention unit obtains the query matrix, key matrix, and value matrix from the input sequence through linear projection. Window constraint: Set a fixed window radius of R, and calculate attention for each query only with its 2R nearest neighbors to filter out irrelevant noise from distant locations; Probabilistic sparsity is achieved by calculating the attention score of each query vector on all keys within the window, retaining only the top u key-value pairs with the highest scores, and setting a sparsity threshold.

6. The real-time stress-relieving music recommendation system based on neural networks according to claim 5, characterized in that, The emotion regression output unit maps the global context vector output by the Transformer encoder through two fully connected network layers. The output layer uses the Sigmoid activation function to calculate valence and arousal, respectively. The formulas for calculating valence and arousal are as follows: ； ； In the formula, It is valence. It's wakefulness. It is the Sigmoid activation function. It is a valence weight matrix. It is the wake-up weight matrix. It is a valence bias term. It is the arousal bias term. It is the global context vector.

7. A real-time stress-relieving music recommendation system based on neural networks according to claim 1, characterized in that, The music emotion anchoring module constructs a high-dimensional acoustic feature vector for the music database and maps it to an emotion anchoring space that is isomorphic to the user's emotional space. It includes a low-level acoustic feature extraction unit and a music emotion regression mapping unit. The underlying acoustic feature extraction unit performs frame-by-frame windowing processing on the original audio PCM stream to extract audio features, including beat rate, spectral centroid, Mel frequency cepstral coefficients, and reverberation time. The music emotion regression mapping unit constructs an auxiliary regression network that maps the extracted acoustic feature vectors into emotion coordinates. This mapping network adopts a pre-training-fine-tuning paradigm and uses a large-scale manually labeled dataset as supervision.

8. A real-time stress-relieving music recommendation system based on neural networks according to claim 1, characterized in that, The recommendation matching module calculates the weighted Euclidean distance between the user's real-time emotion vector and the musical emotion anchor points of tracks in the music library as the emotion distance, and selects the K tracks with the smallest emotion distance as the initial candidate set.

9. A real-time stress-relieving music recommendation system based on a neural network according to claim 1, characterized in that, The closed-loop dynamic adjustment module upgrades music recommendation from static matching to a dynamic adjustment process. It uses user emotions as the controlled variable and music playback parameters as the manipulated variable to construct a finite-time domain rolling optimization problem, including a system state-space modeling unit and a rolling time domain solution and execution unit. The system state space modeling unit uses a recurrent neural network to establish an emotion state predictor, and defines the input vector at time k, including the user's real-time emotion vector at the previous time, music track switching instructions, playback volume gain coefficient, environmental noise interference, and user autonomous operation. The emotion state predictor recursively predicts the user's emotional response at future moments, using the following formula: ； In the formula, It is a prediction of user sentiment response. It predicts the time domain. It is the length of the input sequence; The rolling time-domain solver and execution unit performs the following operations in each control cycle: The user's real-time emotion vector is obtained by the emotion recognition module, the future system output is predicted by the RNN surrogate model, and the above optimization problem is solved by calling the sequence quadratic programming algorithm to obtain the optimal control sequence. Only the first element of the optimal sequence is sent to the audio output module; When the next moment arrives, the state is resampled, and the above steps are repeated to achieve closed-loop dynamic tracking and compensation.

10. A real-time stress-relieving music recommendation system based on a neural network according to claim 1, characterized in that, The audio output feedback module executes control commands, decodes digital audio signals into analog signals to drive the speaker, and monitors the playback progress in real time. When the song reaches the end, it actively requests the next control command from the MPC controller.

Images