Intelligent audio recommendation control method and system

Abstract

The invention relates to the technical field of music recommendation, in particular to an intelligent audio recommendation control method and system. The method comprises the following steps: detecting a real-time face image of a user in an initial music playing process, carrying out multi-period emotion evolution modeling, and constructing a user emotion state evolution curve; identifying the initially played music, and performing emotion resonance matching based on the emotional state evolution curve of the user to obtain emotion resonance response data; constructing a global music feature map of the user music library; performing deep rhythm demand analysis on the emotion resonance response data according to the global music feature map, and extracting an emotion demand adaptive music set; adaptive music recommendation decision making is carried out based on the emotion demand adaptive music set, and recommendation control execution is carried out. The music emotion demand of the user is predicted through accurate user emotion perception, adaptive music is efficiently and accurately recommended, the satisfaction degree of the user is improved, and good auditory experience is provided.

Description

Technical Field

[0001] This invention relates to the field of music recommendation technology, and in particular to an intelligent audio recommendation control method and system. Background Technology

[0002] With the widespread adoption of the internet and mobile devices, digital music has become an indispensable form of entertainment in people's daily lives. Personalized music recommendation technology, as a key means to enhance user experience and meet diverse listening needs, has received widespread attention and rapid development. Traditional music recommendation methods are mostly based on users' historical behavioral data, such as playback history, collection preferences, and song characteristics, using algorithms such as collaborative filtering and content filtering for recommendations. While these methods improve the accuracy of recommendations to some extent, they often overlook the influence of the user's current emotional state on music selection, resulting in recommendations that lack emotional resonance and real-time interactivity, failing to meet users' higher demands for personalized experiences.

[0003] With the continuous advancement of emotion computing, natural language processing, and multimodal data analysis technologies, intelligent audio recommendation control has gradually become a research hotspot. These recommendation methods collect multi-source emotional information such as facial expressions, voice tone, and physiological signals to identify the user's emotional state in real time. Combined with the user's music preferences and environmental background, they dynamically adjust recommendation strategies to achieve music recommendations that better meet the user's emotional needs. Compared to traditional methods, emotion-aware recommendation not only enhances the system's understanding of the user's subjective experience but also improves the interactivity of the recommendations and user satisfaction.

[0004] However, personalized music recommendation based on emotion perception faces numerous challenges in practical applications. How to effectively integrate emotional information with music content and construct a robust and efficient recommendation model is a key technical problem that urgently needs to be solved. Furthermore, processing large amounts of multimodal emotion data in real time and providing low-latency recommendations places higher demands on system performance and algorithm design; therefore, a more intelligent music recommendation method is needed. Summary of the Invention

[0005] To address the aforementioned technical problems, this invention proposes an intelligent audio recommendation control method and system, thereby resolving at least one of the aforementioned technical problems.

[0006] To achieve the above objectives, the present invention provides an intelligent audio recommendation control method, comprising the following steps: Step S1: Detect real-time facial images of the user during the initial music playback process, perform multi-time period emotion evolution modeling, and construct the user's emotional state evolution curve; Step S2: Identify the initial music played, perform emotional resonance matching based on the user's emotional state evolution curve, and obtain emotional resonance response data; Step S3: Construct a global music feature map of the user's music library; Step S4: Perform deep rhythm demand analysis on the emotional resonance response data based on the global music feature map, and extract the music set that matches the emotional demand. Step S5: Make adaptive music recommendation decisions based on the music set adapted to emotional needs, and implement recommendation control.

[0007] This specification provides an intelligent audio recommendation control system for executing the intelligent audio recommendation control method described above, comprising: The emotional state evolution module is used to detect real-time facial images of users during the initial music playback process, perform multi-time period emotional evolution modeling, and construct user emotional state evolution curves. The emotional resonance module is used to identify the initial music played, perform emotional resonance matching based on the user's emotional state evolution curve, and obtain emotional resonance response data. The music feature graph module is used to construct a global music feature graph for the user's music library; The emotional demand adaptation module is used to perform deep rhythmic demand analysis on the emotional resonance response data based on the global music feature map and extract the emotional demand adaptation music set. The adaptive music recommendation module is used to make adaptive music recommendation decisions based on the music set adapted to emotional needs, and to perform recommendation control execution.

[0008] The beneficial effects of this invention are as follows: By using micro-expression recognition technology (such as AU recognition, eyelid movement, and mouth corner tension), it can accurately capture users' natural reactions to music in real time, avoiding subjective feedback bias. Compared to static emotion tags, this step achieves continuous modeling of emotion evolution curves over multiple time periods, revealing the emotional flow of users throughout the entire music segment playback process and improving the ability to model responses to "musical emotion trigger points." Combined with time series analysis methods, it reduces the probability of single-frame misjudgment, improving the robustness of emotion recognition and laying a precise foundation for subsequent resonance analysis. Through resonance analysis between spectral features (such as beat, intensity, and mode) and user emotional evolution trends, a "psychological-music" coupling model is formed, enabling music recommendation to move beyond shallow preference matching and instead achieve deep adaptation based on an empathic resonance mechanism. Different users respond significantly to the same music; this step quantifies individualized "emotional resonance response factors," providing high-dimensional, multi-feature support for personalized recommendations. Through a multi-time-point matching mechanism, it can support more precise music feature preference modeling, such as "intro preference" and "climax stimulation." By extracting fundamental frequencies, rhythms, timbre, and emotional trigger frequencies from music, and combining this with the emotional polarity, semantic intensity, and thematic structure of lyrics, a multi-dimensional musical emotional profile is constructed for each track. Integrating audio features and textual semantic features breaks through the limitations of single-dimensional matching, forming a cross-modal music feature map, enhancing the recommendation system's ability to understand the "intrinsic emotional content" of music. This map helps to trace the changing trends of users' historical preferences in dimensions such as emotion, rhythm, and style, providing a foundation for predicting future preference changes. The rhythm, tempo, and intensity requirements in the emotional resonance response factors can accurately locate the optimal music "adjustment parameters" for the user's current state. By calculating the similarity of the global music feature map, music that best resonates with the current emotional state can be selected, satisfying the "current need" psychological state adaptation. Compared to static recommendations based on tag classification or user profiles, this mechanism dynamically responds to user states, improving the timeliness and individual perception of recommendations. By locating the user's geographical environment (such as commuting, exercise, and resting places) and behavioral states (such as step frequency and screen interaction frequency), the matching degree between recommended music and life scenarios can be significantly improved. Music's effects vary drastically across different scenarios. This step, through real-time state modeling, ensures semantic consistency between recommended results and the usage context, leading to a more natural auditory experience. By continuously tracking the relationship between user behavior, emotion, and music, the evolution of user preferences is analyzed, providing predictive basis for long-term model optimization. By combining emotional response factors and behavioral scenario patterns, recommended results are no longer static but adaptively adjusted based on the user's current multiple needs. Explicit feedback (favorites, ratings) and implicit feedback (playtime, skipping rate) jointly participate in model training, improving the system's long-term accuracy.Through technologies such as reinforcement learning and self-supervised learning, the system can continuously iterate and optimize itself according to the user's emotional cycle, scene changes, and music aesthetic trends, forming a truly intelligent music recommendation agent with "empathic cognition ability". Attached Figure Description

[0009] Figure 1 This is a flowchart illustrating the steps of an intelligent audio recommendation control method according to the present invention. Figure 2 This is a detailed flowchart illustrating the implementation steps of step S1. Figure 3 This is a detailed flowchart illustrating the implementation steps of step S2; Figure 4 This is a flowchart illustrating the detailed implementation steps of step S3. Detailed Implementation

[0010] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0011] This application provides an intelligent audio recommendation control method and system. The executing entities of the intelligent audio recommendation control method and system include, but are not limited to, mechanical equipment, data processing platforms, cloud server nodes, network upload devices, etc., which can be considered as general computing nodes in this application. The data processing platform includes, but is not limited to, at least one of an audio image management system, an information management system, and a cloud data management system.

[0012] Please see Figures 1 to 4 This invention provides an intelligent audio recommendation control method, comprising the following steps: Step S1: Detect real-time facial images of the user during the initial music playback process, perform multi-time period emotion evolution modeling, and construct the user's emotional state evolution curve; Step S2: Identify the initial music played, perform emotional resonance matching based on the user's emotional state evolution curve, and obtain emotional resonance response data; Step S3: Construct a global music feature map of the user's music library; Step S4: Perform deep rhythm demand analysis on the emotional resonance response data based on the global music feature map, and extract the music set that matches the emotional demand. Step S5: Make adaptive music recommendation decisions based on the music set adapted to emotional needs, and implement recommendation control.

[0013] In the embodiments of the present invention, see Figure 1 The diagram below illustrates the steps of an intelligent audio recommendation control method according to the present invention. In this example, the steps of the intelligent audio recommendation control method include: Step S1: Detect real-time facial images of the user during the initial music playback process, perform multi-time period emotion evolution modeling, and construct the user's emotional state evolution curve; In this embodiment, visual perception is used to capture the user's natural emotional response when music is first played, constructing their initial emotional state and developing a continuous dynamic model. First, the camera module of the user's terminal device (such as a smartphone, PC, or head-mounted camera) is invoked to capture a real-time facial image frame sequence within the first 5 seconds after the user clicks the "play" button. The image sampling frequency is set to 15 frames per second (15 FPS) to maintain the temporal continuity of facial expression changes without significantly interfering with the user's device resources. The image data is input into a pre-trained multi-expression recognition model (such as one based on EmotionNet or FaceAction Units CNN), which can capture facial action units (AUs) from changes in facial key points, such as AU12 (upturned corners of the mouth) and AU4 (converged eyebrows). Specifically for micro-expression recognition, a short-term facial transient recognition network (such as STMM or FACS+GRU models) is introduced, focusing on facial expression segments that change rapidly within 0.04 to 0.2 seconds. These changes are often a true projection of emotional responses and have strong value for emotional diagnosis. Micro-expression recognition models use a time-sliding window mechanism and joint probability estimation of action units to determine whether a user exhibits subtle emotional movements such as "suppressed smile," "sudden frown," or "slight fear."

[0014] Each frame of the recognition result is assigned an emotion label (such as basic emotions like happiness, sadness, surprise, and disgust), and the corresponding timestamp and facial confidence score are recorded to form a preliminary emotion time series. To build a more robust emotion evolution model, the emotional states of consecutive frames are smoothed and their dynamic change trends are analyzed. Specifically, the exponential moving average (EMA) is used to denoise the emotion intensity signal, and then three key indicators—"emotion intensity gradient," "emotion state transition rate," and "emotion duration stability"—are calculated using a sliding window (e.g., every 10 seconds). These emotion dynamics parameters together constitute the input for modeling the user's multi-time-segment emotion evolution. The user's "emotion state evolution curve" is plotted on the timeline, visually presenting the trajectory of emotion intensity changes over time in a line graph format. For example, if a user's smile intensifies when playing the first chorus, or their micro-expression suddenly turns to surprise when the bridge appears, it may mean that this music evokes a strong emotional resonance. This emotion evolution curve will serve as one of the core perceptual bases for personalized music recommendation, driving the dynamic adjustment of subsequent music rhythm matching, resonance intensity calculation, and candidate music selection strategies.

[0015] Step S2: Identify the initial music played, perform emotional resonance matching based on the user's emotional state evolution curve, and obtain emotional resonance response data; In this embodiment, the dynamic matching relationship between the acoustic characteristics of music and the user's emotional response is analyzed to uncover the resonance mechanism between emotion and music, thereby improving the accuracy of emotion-based music recommendation strategies. First, a spectral analysis is performed on the music initially played by the user. The Short-Time Fourier Transform (STFT) method is used to divide the audio signal into windows of 2048 sampling points (approximately 46ms, sampling rate 44.1kHz), with a window overlap rate set to 50% to ensure a balance between time and frequency resolution. The spectral data is then converted using Power Spectral Density (PSD) to extract the beat frequency and pitch frequency components of the music.

[0016] Rhythm extraction uses an autocorrelation function and a beat tracking algorithm to capture rhythmic cycles in music in real time, typically within a range of 60 to 180 BPM (Beats Per Minute), matching most popular music rhythms. Pitch component analysis utilizes a pitch detection algorithm combined with a tonal centroid model to identify the fundamental frequency and overtones, revealing melodic clues and tonal changes. The spectral analysis results are presented as a time series to form a "musical rhythm feature curve" and a "pitch frequency component curve." The "emotional state evolution curve" constructed by the user in step S1 is synchronously aligned with the musical rhythm feature curve, achieving high-precision matching within ±0.5 seconds based on timestamp accuracy. The emotion-music resonance mining module employs a strategy combining temporal correlation analysis (such as the Dynamic Time Warping (DTW) algorithm) and mutual information (MI) measures to quantify the resonance between emotional intensity gradients and musical rhythm changes. This method effectively reveals the dynamic coupling relationship between user emotional fluctuations and changes in musical rhythm at multiple time points, such as whether sudden beat changes or melodic climaxes trigger emotional escalation or transitions. It introduces multi-time-point emotional resonance matching calculations, extracting key time nodes for different structural sections of the music (such as verse, chorus, and bridge), and matching the peaks and troughs on the user's emotional curve with the dynamic changes in musical rhythm and pitch. During the calculation, the Pearson Correlation Coefficient is used to measure the linear correlation between emotional response and musical features, while a weighted average mechanism is combined to emphasize periods of strong emotional shifts. Finally, by synthesizing the matching results from all time points, an "emotional resonance response factor" is generated, representing the user's ability to respond to the current music. Its value is typically standardized between 0 and 1, with higher values ​​indicating stronger synchronization between emotion and musical rhythm and melody.

[0017] Step S3: Construct a global music feature map of the user's music library; In this embodiment, multimodal music feature information from the user's local or cloud-based music library is comprehensively deconstructed. Through deep acoustic analysis combined with lyric semantic understanding, a structured global music feature map is constructed, laying a solid data foundation for subsequent emotion matching and personalized recommendations. The process first uses an audio parsing module to automatically scan the user's device music directory, identifying and reading all audio files. Supported formats include mainstream encoding formats such as MP3, WAV, and FLAC. The identification stage employs a combination of file metadata and audio fingerprint algorithms (such as Chromaprint and AcousticID) to ensure a file identification accuracy rate of over 99%.

[0018] Entering the acoustic feature analysis stage, multi-level time-frequency domain signal decomposition is performed on each audio file. First, short-time Fourier transform (STFT) is used to extract fundamental frequency information, analyzing pitch and its variation trends. The fundamental frequency detection accuracy reaches ±5Hz, ensuring accurate capture of melodic clues. Subsequently, overtone analysis is performed, extracting overtone structure parameters through harmonic frequency detection to reflect the richness and texture of the timbre. Furthermore, timbre envelope features are extracted using Mel-frequency cepstral coefficients (MFCC), covering 12 to 20 coefficients to meticulously depict instrument sounds, singer vocals, and the overall musical atmosphere. Temporal features such as volume envelope, rhythmic intensity, and instantaneous energy fluctuations are also simultaneously acquired, with a window length typically of 512 sampling points and an overlap rate of 75% to ensure temporal resolution. The lyrics semantic feature parsing module, for audio files with lyrics (LRC or embedded ID3 tags), uses natural language processing (NLP) techniques to analyze emotional polarity and intensity. The algorithm utilizes pre-trained sentiment analysis models (such as BERT sentiment classifier or sentiment dictionary model) to segment, tag, and parse the lyrics text, quantifying sentiment polarity (positive, negative, neutral) and sentiment intensity (continuous values ​​in the 0-1 range). For sentiment gradient recognition, the model detects progressive and transitional phrases in the lyrics, locating points of emotional shifts to facilitate subsequent multi-timeframe sentiment matching. A sentiment vector space for the lyrics is constructed, using word embedding techniques (Word2Vec or GloVe) to convert each sentence or paragraph into a high-dimensional vector, forming a structured sentiment semantic graph. Combining the emotional triggering frequency bands and emotional transition points from the acoustic features, a cross-modal fusion algorithm (such as multimodal deep fusion networks or collaborative attention mechanisms) integrates the audio features with the lyric sentiment vectors to achieve global music fitting. This fusion model considers both the physical features of the audio and the emotional information of the lyrics, comprehensively reflecting the emotional expression and triggering mechanisms of the music. Generate a global music feature map of the user's music library, stored in the form of a matrix or graph database, containing the fundamental spectrum, overtone structure, timbre envelope, rhythmic features, and lyric emotion vectors and their corresponding time points for each song.

[0019] Step S4: Perform deep rhythm demand analysis on the emotional resonance response data based on the global music feature map, and extract the music set that matches the emotional demand. In this embodiment, a deep rhythmic demand analysis is conducted based on the user's emotional resonance response factor. This factor reflects the resonance between the user's current emotional state and the rhythm and melody of the music being played, reflecting the user's specific preferences in terms of rhythm frequency, beat intensity, and melody variation. During the analysis, the emotional resonance response factor is decomposed into multiple dimensions, including rhythmic synchronization, emotional intensity gradient, and music energy response. Time-frequency analysis methods such as wavelet transform are used to refine the multi-scale features of rhythmic changes, accurately capturing the user's needs for tempo (BPM range), rhythmic fluctuation amplitude, and beat stability. Experimental parameters show that tempo preference is usually concentrated in the 60-150 BPM range, rhythmic demand tends to be in the mid-to-high frequency range when emotions are intense, and rhythmic stability demand corresponds to a rhythmic fluctuation rate of less than 0.15. Using the above rhythmic demand characteristics, similarity calculation is performed in the global music feature map of the user's music library. Each piece of music in the global map is represented by a multi-dimensional feature vector, covering pitch frequency components, rhythmic features, timbre envelope, and lyric emotional vectors. Similarity calculation employs a weighted cosine similarity method, where the weight coefficients are dynamically adjusted based on the user's current emotional response factors, emphasizing the matching weight of rhythm and emotion-related features. In practice, the similarity threshold is typically set above 0.75 to ensure a high degree of alignment between the recommended music and the user's rhythmic needs.

[0020] This study introduces an Approximate Nearest Neighbor (ANN) algorithm to improve the retrieval efficiency of large-scale music libraries. Combined with inverted indexes and tree-structured indexing methods, it can quickly locate matching music from thousands to tens of thousands of tracks within seconds. During the matching process, music features associated with negative emotional responses are filtered out to avoid recommending tracks unsuitable for the user's current emotional state. Based on similarity scores and rhythm requirements, an emotionally appropriate music set is automatically generated. This set not only covers music that satisfies rhythm and melody preferences but also incorporates lyric sentiment vectors to ensure consistency in emotional expression. In the experimental validation phase, tests were conducted with 300 users. User feedback showed that the emotional relevance of the matched music improved by over 20%, the completion rate increased by 15%, and the skip rate decreased by 12%, significantly optimizing the personalized recommendation experience.

[0021] Step S5: Make adaptive music recommendation decisions based on the music set adapted to emotional needs, and implement recommendation control.

[0022] In this embodiment, real-time scene music demand information is received, including rhythm preferences, timbre tendencies, lyric emotional attributes, and music energy levels, and then fused and matched with a generated set of music adapted to emotional needs. A multi-objective optimization strategy is employed during the fusion process, using a weighted linear combination model to balance the weights of the two demand vectors. The weights are dynamically adjusted based on the user's historical behavior data and real-time emotional fluctuations. For example, when the user is in an extreme emotional state, the weight of the emotional demand is set to 0.7, and the weight of the scene demand is 0.3; in a stable state, both are balanced at 0.5. This model ensures that the recommendation result matches both the user's immediate emotion and the current environment. Based on the fused music demand features, a reinforcement learning method is used to achieve adaptive recommendation decisions. Specifically, a Deep Q-Network (DQN) is used as the core algorithm, taking the user's state (emotional state, scene state) as environmental input, the recommended action corresponding to music selection, and the reward function design combining explicit user feedback (rating, number of favorites) and implicit feedback (playback duration, skip rate, number of repeated plays). For example, a playback duration exceeding 3 minutes is considered a positive reward, and more than 2 skips are considered a negative reward. During training, an experience replay mechanism was employed to buffer the most recent 1000 user feedback data points, ensuring the stability and generalization ability of the model training. Experimental data shows that after 1000 rounds of iterative training, the model's average recommendation accuracy increased to over 90%, and user satisfaction increased by 20%.

[0023] To ensure real-time responsiveness of recommendations, the model employs online incremental learning technology, fusing new feedback data in real time and dynamically adjusting strategy weights to avoid recommendation bias and overfitting. A threshold monitoring mechanism is implemented; when user satisfaction shows a downward trend, the strategy adjustment module is triggered to automatically readjust weights or switch exploration strategies to maintain recommendation quality and diversity. An intelligent music recommendation agent is constructed, continuously learning from user feedback and environmental changes to gradually improve personalized recommendation strategies. The agent integrates multiple technologies, including emotion perception, multimodal data fusion, reinforcement learning, and behavioral analysis, forming a closed-loop optimization system. In experimental environments, this agent can achieve second-level recommendation decision updates in various user scenarios, improving overall user retention by 15% and increasing daily playback time by approximately 25 minutes, significantly optimizing user experience and performance.

[0024] In this embodiment, see Figure 2 The diagram below illustrates the detailed implementation steps of step S1. In this embodiment, the detailed implementation steps of step S1 include: Detects real-time facial images of the user during the initial music playback process; Adaptive contrast optimization is performed on the real-time facial image to generate an adaptively optimized facial image; Multimodal physiological state response perception is performed based on adaptively optimized facial images, and a multimodal physiological state perception matrix is ​​constructed. Decode the potential emotional state of the multimodal physiological state perception matrix to generate potential emotional fluctuation features of users. The potential emotional fluctuation characteristics of users are decomposed in a multi-level manner to extract the emotional intensity gradient, emotional transition rate and emotional stability. Based on the gradient of emotional intensity, the rate of emotional transition, and the stability of emotional persistence, a multi-period emotional evolution model is constructed to build a curve of user emotional state evolution.

[0025] In this embodiment, during the initial stage when the user starts playing music, real-time facial images of the user are captured via a front-facing camera or other image acquisition devices. This process emphasizes low latency and high-resolution synchronous acquisition capabilities to ensure image quality and time consistency in subsequent analysis. A frame rate of 30fps and a resolution of at least 720p (1280x720 pixels) are generally recommended to ensure that dynamic facial expressions and micro-expressions are not missed. The real-time image acquisition process requires calling a face detection module (e.g., HOG + SVM method based on MTCNN or Dlib) to accurately locate and crop the face region in each frame, and to perform image standardization processing, including image size normalization, color space conversion (RGB to grayscale or YCbCr), and illumination compensation. To prevent user head movements from affecting image quality, a head pose estimation module (e.g., using OpenFace or MediaPipe technology) can be integrated to automatically remove or resample frames that deviate too much from the standard pose. Subsequent image enhancement and emotion recognition establish a clear and stable foundation of facial data. After obtaining the original facial image, image enhancement is required to improve the accuracy of subsequent emotional feature extraction. Adaptive contrast optimization is a key preprocessing step, with common methods including CLAHE (Contrast Limited Adaptive Histogram Equalization) and adaptive gamma correction. CLAHE processes the contrast of local areas in the image by dividing it into blocks, avoiding noise expansion by limiting contrast gain, and is suitable for enhancing subtle facial features (such as subtle eyebrow movements or slight upturns of the mouth). In its implementation, the image is divided into 8x8 blocks, the local histograms are redistributed, and bilinear interpolation is performed to synthesize a complete image. Gamma correction is used to balance the brightness of facial images under different lighting conditions, improving the visibility of details in dark areas. In experiments, a gamma value between 0.5 and 1.5 is generally chosen, adaptively adjusted according to the average brightness of the image. This step also needs to be combined with a skin color detection algorithm to avoid non-facial areas affecting the contrast optimization results. After image enhancement, the optimized image is evaluated for quality (e.g., using the SSIM structural similarity index or the Laplacian variance index for image sharpness) to ensure that the enhanced image meets the sharpness requirements for subsequent feature extraction.

[0026] Multimodal features characterizing a user's physiological and emotional state are extracted from enhanced facial images to construct a unified physiological state perception matrix. This matrix integrates static image features and temporal variation features from dynamic image sequences. Static features mainly include local muscle motion units (AUs) based on facial expression recognition, such as AU12 (smiling) and AU4 (frowning). These features are typically extracted using tools like OpenFace or AFDEX, with the results represented as a feature vector (approximately 17-35 dimensions) per frame. Dynamic image features are primarily obtained through optical flow methods (such as the Farneback algorithm or the RAFT depth model) to capture the dynamic change speed and direction of facial muscles over time. To enhance emotion discrimination, skin color change frequencies extracted based on Eulerian Video Magnification (EVM) can also be integrated to estimate heart rate change trends (BVP signal). When constructing the multimodal matrix, all types of features undergo unified normalization (e.g., Min-Max normalization or Z-score normalization) and are stacked according to time series to generate a three-dimensional tensor (number of frames × feature types × feature dimensions). This matrix forms the basis for subsequent latent emotion decoding and modeling, reflecting the trajectory of the user's facial physiological state changes during the initial stage of music playback. After constructing the multimodal physiological state perception matrix, an emotion decoding model is needed to map it to the user's latent emotional state. Typically, temporal modeling networks from deep learning architectures, such as Bi-LSTM or Transformer structures, are used to capture the evolution of facial dynamics over time. The model input is the multimodal feature matrix, and the output is a sequence of continuous or discrete emotion labels (e.g., two-dimensional emotion space coordinates based on the Valence-Arousal model, or classification results of basic emotions such as anger, sadness, and happiness). In experimental settings, the training set should include video data with emotion annotations, such as the AFEW, SEMAINE, or DEAP datasets. The output of the emotional state includes not only the estimate at the current moment but also the fluctuation trend over time, such as a continuous increase or sharp decrease in the Valence value. To avoid misjudgment, a sliding window smoothing algorithm (such as Savitzky-Golay filtering) is used to smooth the decoding results in the short term and extract trend features. Ultimately, a set of feature vectors describing a user's potential emotional fluctuations will be generated, including metrics such as timestamps, emotional coordinates, fluctuation amplitude, and frequency of change.

[0027] After analyzing the potential emotional fluctuation characteristics, a multi-level decomposition is needed to extract more detailed emotional dynamic indicators. First, the emotional intensity gradient represents the steepness of emotional change, which can be calculated by taking the first derivative of the Valence or Arousal curve. In experiments, if the gradient exceeds 0.6 (based on normalized values) within a certain time period, it indicates a drastic emotional change. Second, the emotional transition rate refers to the time required to transition from one emotional state to another, defined as the interval between two major emotional category switches, commonly measured in seconds. For example, the transition rate is the number of frames taken to switch from "happy" to "sad" multiplied by the frame rate. Third, emotional persistence stability measures the duration and stability of an emotional state over time, usually measured by variance. If an emotional state persists for more than 30% of its average duration and the fluctuation is less than a set threshold (e.g., 0.15 standard deviation), it is considered a stable emotional state. These three indicators constitute the core of multi-level emotional behavior characteristics, helping subsequent personalized recommendation models understand users' emotional acceptance patterns and fluctuation structures during music playback. After extracting complete emotional dynamics, time series modeling methods are used to visualize and model the evolution trend of user emotions. This modeling process typically involves constructing short-term emotional state curves using sliding windows (e.g., 30-second windows) and connecting them into a complete evolution curve using a nested multi-window structure. The model can take the form of weighted moving average (WMA), exponential smoothing (EMA), or prediction modeling based on Kalman filtering to reduce jitter in emotion estimation. The vertical axis of the evolution curve represents the Valence / Arousal value, and the horizontal axis represents the timestamp. Information such as emotional jump points, stable phases, and regression zones can be embedded in the curve, and emotional changes can be associated with event markers (e.g., playing a climax segment). Evaluation metrics for the curve structure in the experiment include curve smoothness, sensitivity to change (the degree of response to music changes), and predictive consistency (consistency with user subjective feedback). The final generated emotional state evolution curve serves as a personalized "fingerprint" of the user's emotional response, used as a matching factor reference for the next step of the personalized music recommendation model. Through cluster analysis of the curve shape, users' music emotion-sensitive, stable, or delayed-response behavioral preferences can also be categorized.

[0028] In this embodiment, the specific steps for constructing a multimodal physiological state perception matrix based on adaptively optimized facial images for multimodal physiological state response perception are as follows: Perform dynamic eye optical flow tracking on adaptively optimized facial images to extract the user's eye movement trajectory; The user's pupil constriction frequency and blink count are calculated based on adaptively optimized facial images; Multidimensional temporal state mining of the user's eyeball drift trajectory, pupil contraction frequency and blink count is performed to generate user temporal eyeball state features. The facial muscle tension and relaxation micro-changes are calculated on the adaptively optimized facial image to extract the facial muscle tension and relaxation degree. Based on facial muscle tension and relaxation, deep micro-expression recognition is performed to obtain the user's real-time micro-expression features; Based on collecting users' heart rate parameters using smart wearable devices; The user's heart rate parameters are analyzed for changes over multiple eras to generate a user heart rate fluctuation curve; The system performs time-stamp synchronization calibration on the user's heart rate fluctuation curve, real-time micro-expression features, and temporal eye state features, and performs multimodal physiological state response perception to construct a multimodal physiological state perception matrix.

[0029] In this embodiment, based on a contrast-optimized facial image sequence, dynamic eye tracking begins, focusing on extracting the user's eye movement trajectory during music playback. This task is primarily achieved using a combination of optical flow and deep learning-based gaze estimation techniques. Traditional optical flow methods, such as the Lucas-Kanade or Farneback algorithms, are suitable for tracking minute displacements of pixels in the eye region, detecting lateral and vertical eye movement trends. More precise gaze estimation relies on deep neural networks (such as GazeNet or RT-GENE models), which extract eye image features through convolutional neural networks and predict three-dimensional gaze vectors (pitch, yaw). Eye movement trajectories are typically recorded as two-dimensional or three-dimensional time series, with a frame rate set at 30fps and a duration consistent with the music segment. Parameters of interest in trajectory analysis include mean gaze shift, fixation duration, and saccade velocity, which reflect the distribution of the user's visual attention and reaction rhythm during music playback. For example, when the mood of music changes drastically, the user's eye movement trajectory often shows more frequent left and right glances, indicating increased emotional fluctuation. Smoothing of the trajectory can be achieved using Kalman filtering or first-order difference denoising to ensure the analyzability and structure of the trajectory data. Pupil constriction frequency and blinking behavior are two physiological signals highly correlated with autonomic nervous system responses, effectively reflecting the level of psychological stress during emotional changes. Using a contrast-optimized facial image sequence, a pupil detection algorithm is employed to locate and estimate the pupil region in each frame. Pupil detection is typically based on infrared-enhanced image analysis (e.g., using pseudo-color enhancement) or binarization extraction using a U-Net-based segmentation neural network. Based on the inter-frame pupil area change, the constriction frequency (in Hz) can be calculated; for example, 4 constrictions within 10 seconds are recorded as 0.4Hz. Simultaneously, blink detection is achieved by analyzing changes in the eye aspect ratio (EAR). When the EAR value is below a set threshold (e.g., 0.2) for more than 3 consecutive frames, it is considered a blink. By statistically analyzing the total number of blinks, average blink interval, and maximum blink density throughout a music playback segment, the user's blinking behavior pattern within the current music segment can be determined. Typically, blinking frequency increases during emotional excitement, while pupil dilation is more pronounced in states of tension or immersion. Pupil and blink parameters are normalized and synchronized with timestamps to provide foundational eye-movement physiological data for subsequent temporal modeling.

[0030] After collecting the aforementioned eye-tracking parameters, these parameters are integrated into a multidimensional temporal eye state vector to reflect the emotional mapping between the user's visual attention and neural modulation processes. This process first constructs a time-aligned eye-tracking data matrix (T × N-dimensional features), where T is the number of time frames and N is the type of eye-tracking feature (e.g., GazeX / Y, PupilSize, BlinkRate, etc.). Then, feature selection and compression are performed on this matrix, typically using Principal Component Analysis (PCA) or Linear Discriminant Analysis (LDA) to reduce dimensionality while retaining key state variables. Further, Hidden Markov Models (HMMs) or LSTM networks are used to model the changes in eye state over time, extracting probability transition matrices between different state transition patterns (e.g., fixation-salivation-eye closure states) to construct a user temporal eye state model. Key indicators such as average fixation entropy and eye response latency after emotion induction can be extracted from this model. These features, serving as auxiliary channels in emotion recognition, help identify potential psychological states such as anxiety, relaxation, and alertness, significantly improving the overall accuracy of emotion decoding. The tension and relaxation of facial muscles are directly related to the strength and intensity of a user's facial emotional expression, and are a core factor in distinguishing between natural micro-expressions and deliberate expressions. In this step, pixel-level motion analysis is performed on key facial muscle regions (such as the frontalis, buccinator, and orbicularis oculi) based on the optimized image sequence. This process often employs deformation tracking methods, such as Active Appearance Model (AAM) or 3D Morphable Model (3DMM), combined with dynamic change data of the coordinates of 68 key facial points, to construct a tension and relaxation change vector field for each facial region. These vector fields are used to calculate the Facial Muscle Tension Index (FMTI), for example, measuring the degree of muscle activity by the average deformation amplitude (in pixels or mm) of the muscle region per unit time. In experiments, if sustained high-amplitude changes (such as micro-movement amplitudes greater than 0.3 mm) are observed in the forehead or corner of the mouth area, it can be determined as a high-tension state. Further analysis of the frequency and symmetry of changes in each muscle group identifies natural tension and relaxation states as well as sudden contractions triggered by emotions. By constructing a temporal atlas of muscle tension and relaxation, a foundation is laid for subsequent micro-expression recognition and dynamic emotion recognition.

[0031] Microexpressions are the most authentic reflection of emotional changes. Due to their extremely short duration (usually less than 500ms), analysis requires high-speed image capture and deep recognition models. Based on the extracted muscle tension / relaxation sequences, multi-channel feature fusion and classification are performed using microexpression recognition networks (such as STSTNet or the 3D-CNN structure proposed in the MEGC competition). The recognition process includes two stages: the first stage detects potential microexpression segments based on the rate and direction of tension / relaxation curve changes; the second stage inputs these segments into a deep network to extract local features (such as the combined features of eyebrow raising and eye corner stretching) and performs emotion label mapping (such as surprise, disgust, fear, etc.). In actual experiments, a sliding window is constructed every 100 frames to identify multiple microexpression events and calculate their start and end times, intensity levels, and durations. Each microexpression instance is labeled with a specific emotion category and corresponding confidence level, adding high-frequency fluctuation markers to the emotion fluctuation curve. This level of data possesses high precision and high temporal resolution, providing crucial information support for fine-grained emotion analysis. To obtain a more intuitive reflection of the user's internal physiological state, heart rate data is synchronously collected through smart wearable devices (such as Apple Watch, Mi Band, Fitbit, etc.). These devices typically use PPG (photovolumetric pulse wave) technology as their core, calculating heart rate by observing changes in the signal reflected from light penetrating the skin. The sampling frequency is often set to 1Hz or higher. A data channel is established with the wearable device via Bluetooth or Wi-Fi protocols to collect indicators such as heart rate (bpm), heart rate variation (HRV), and respiratory rate in real time. A baseline resting heart rate is set before music playback, recording the average resting heart rate for 5 minutes as a reference for subsequent analysis. During music playback, the user's heart rate change trend is recorded second by second, and timestamps are added to correspond to emotionally charged segments. For example, if the heart rate rises from 78 bpm to 96 bpm during a high-energy segment, and the HRV decreases significantly, it is identified as a "highly aroused" state. The data collection process requires ensuring data continuity and stability, using outlier removal and interpolation repair mechanisms to avoid the impact of short-term communication interruptions or signal interference on data integrity.

[0032] The collected heart rate parameters need to be transformed into a heart rate transition curve through time-series trend modeling to reflect the dynamic changes in the user's physiological activation during music playback. First, the heart rate sequence is denoised and smoothed using methods such as Loess smoothing or exponentially weighted moving average (EWMA) to filter out random fluctuations and sensing errors. Next, by setting a heart rate transition threshold (e.g., a sudden increase of 5 bpm or more), drastic heart rate changes are detected, and their duration, direction of change, and response delay are recorded. Segmented fitting of the heart rate curve (e.g., based on Bézier curves or spline interpolation) yields a continuous curve shape, and key transition points are marked. Finally, the heart rate transition curve exhibits a continuous structure including multiple stages such as the activation phase, recovery phase, and plateau phase, reflecting the rhythm of the user's physiological response to music stimulation. This curve can also be correlated with parameters such as music volume and rhythm to further enhance the coupling modeling capability between music content and physiological signals. After all single-channel physiological signals have been extracted, the data from each channel needs to be synchronized with a unified timestamp to ensure that data from different modalities correspond correctly on the same time scale. First, a unified start time (T0) is set as the music playback start time, and all acquisition modules (cameras, wearable devices) synchronize their timestamps upon startup. Different modal data are timestamped based on frame rate or sampling frequency (e.g., 30fps images and 1Hz heart rates need to be sampled and matched), and missing values ​​or desynchronization points are aligned using linear interpolation or Dynamic Time Warping (DTW). After alignment, eye state, micro-expression indicators, and heart rate curves are combined at each time point into a unified multimodal physiological feature vector, constructing a complete multimodal physiological state perception matrix (in the form of T × N, where T is time and N is the multimodal feature dimension). This matrix integrates the dynamic changes of external behavioral signals and internal physiological parameters, providing high-dimensional, high-resolution, multi-source input for emotional state modeling, which helps improve the accuracy and fine-grained perception capability of emotional state recognition in personalized music recommendations.

[0033] In this embodiment, see Figure 3 The diagram below illustrates the detailed implementation steps of step S2. In this embodiment, the detailed implementation steps of step S2 include: Spectral analysis was performed on the initially played music to obtain the audio rhythm and pitch frequency components; Based on the frequency components of audio rhythm and pitch, the characteristics of different musical stages are analyzed to generate rhythmic characteristic curves for different musical stages. Based on the rhythm feature curve, the user's emotional state evolution curve is mined for emotion-music resonance, and multi-time point emotional resonance matching calculation is performed to generate an emotional resonance index curve. Based on the emotion resonance index curve, feature nodes of emotion jump, continuous response, and delayed resonance are captured to generate emotion resonance response data.

[0034] In this embodiment, high-resolution spectral analysis is performed on the current audio segment the moment the user begins playing music, extracting its rhythmic features and pitch frequency components. This process is primarily based on the Short-Time Fourier Transform (STFT) and Mel-Frequency Cepstral Coefficients (MFCC) extraction methods. First, the original audio signal is framed using a sliding window with a window length of 2048 points and 50% overlap. After applying STFT, a two-dimensional time-spectrum graph is obtained. This graph shows the intensity changes of each frequency component over different time periods, facilitating the identification of rhythmic fluctuations and structural changes. Simultaneously, MFCC is used to model the pitch components and auditory characteristics of the music, extracting 13- to 39-dimensional Mel-frequency cepstral features, representing the pitch, brightness, and harmonic structure of the music. In addition, beat detection algorithms (such as Dynamic Time Warping-based Beat Tracking) are applied to extract BPM (Beats Per Minute) values, identifying the strong beat positions and beat cycles. In the experiment, when analyzing multiple music genres, the rhythmic range mostly fell between 60 and 160 BPM, while the frequency components were mainly concentrated between 20 Hz and 4 kHz. These analyses laid the foundation for subsequent music segmentation and emotional alignment. The analysis results were stored as two matrix data structures: a rhythm spectrum and a frequency spectrum, with the dominant frequency and beat intensity corresponding to specific time periods. The entire piece of music was divided into stages, and the rhythmic feature curves of each stage were extracted. The segmentation method employed music segmentation detection algorithms (such as Spectral Clustering + Novelty Curve), which determined whether the music had entered a new stage by comparing the MFCC distance between adjacent frames, changes in rhythmic contours, and abrupt changes in frequency energy density. Typical segments included the intro, verse, chorus, and bridge, each corresponding to a different emotional engagement. Within each stage, its rhythmic feature curve was further constructed to reflect its rhythmic density, beat stability, and tempo changes. For example, rhythmic density was obtained by counting the number of strong beats per unit time, rhythmic stability was measured by the standard deviation of the beat interval, and tempo changes reflected the slope of the BPM curve. In the experiment, for music with complex structures (such as electronic music or rock music), there are often abrupt rhythmic changes, and the curve shape is similar to a sawtooth. In contrast, in light music or classical music, the rhythmic feature curve is smoother and more gradual. The rhythmic curve of each musical stage is matched with a timestamp and output as a basic reference for emotion modeling and resonance detection, enabling the ability to identify "musical emotional rhythm points".

[0035] Given user emotional state evolution curves (such as Valence-Arousal dynamic sequences) and music rhythm feature curves, this task mines "emotion-music resonance," focusing on detecting the temporal impact and coupling strength of music changes on user emotional fluctuations. This task employs Dynamic Time Warping (DTW) and Cross-Correlation Function (CCF) for multi-time-point matching, calculating the matching degree of two time series at different delays. DTW adjusts the alignment of the two time-series curves on the time axis to find the optimal path, synchronizing music rhythm fluctuations with emotional fluctuations; while CCF outputs the resonance coefficient value (typically ranging from -1 to 1) for each time delay window. A minimum alignment window of 5 seconds and a maximum of 30 seconds are set to simulate the user's immediate or delayed response to music. The resonance matching degree at each time point throughout the music is calculated in steps of one second, generating a continuous "Emotion-Resonance Index Curve" (ERIC). A higher resonance index indicates a stronger emotional impact of the music's rhythm on the user. A sustained index above 0.8 may indicate a climax or resonance point in the music. This curve serves as a key indicator of emotion-driven performance, providing a precise measure of music matching in personalized recommendations. In-depth structural analysis of the generated emotional resonance index curve reveals three key feature nodes: Emotional Jump Point, Sustained Response Segment, and Delayed Resonance Node. Jump Point detection is based on a combined analysis of first-derivative abrupt changes and second-derivative extrema. A jump point is marked if the index shows a slope change greater than 0.4 within one second, often corresponding to a shift in emotion from a stable state to a state of high excitement or depression. A sustained response segment is defined as a period where the resonance index remains high (e.g., greater than 0.75) for more than 10 seconds, indicating that a particular section of the music has a long-term emotional impact on the user. The delayed resonance point is calculated by comparing it with the peak delay of the music rhythm. If the emotional response occurs within 5-15 seconds after the musical stimulus, it can be identified as a delayed resonance response, indicating a secondary response to complex musical structures. The timestamps, feature values, and resonance levels of the above three types of nodes will be integrated into a set of "emotional resonance response data," including multiple dimensions such as transition frequency, response duration rate, and mean resonance delay. These factors can be used in personalized modeling to distinguish user emotional profiles such as "emotionally irritable," "emotionally stable," and "delayed-response," and to construct music recommendation strategies with a more emotionally resonant logic.

[0036] In this embodiment, reference Figure 4The above is a detailed implementation flowchart of step S3. In this embodiment, the detailed implementation steps of step S3 include: Identify audio files in the user's music library and perform acoustic feature analysis to extract fundamental frequency, overtones, and timbre envelope feature parameters; Based on the fundamental frequency, overtones, and timbre envelope characteristic parameters, time-frequency domain decomposition is performed to identify the emotional trigger frequency bands and emotional transition points of each piece of music. The semantic features of lyrics in audio files are analyzed to obtain the emotional polarity and emotional intensity. Construct a lyrics emotion vector space based on emotional polarity and emotional intensity; Cross-modal emotional feature fusion is performed on the lyric emotional vector space, emotional trigger frequency band, and emotional transition turning point, and global music fitting is performed to construct a global music feature map of the user's music library.

[0037] In this embodiment, the user's local music library is automatically identified and standardized during the initialization phase. The identification range covers mainstream audio formats (such as MP3, FLAC, WAV, AAC, etc.). Track information and metadata (such as artist, style, release year, etc.) are extracted through embedded tags (ID3 Tag) and audio fingerprinting technology (such as Chromaprint). The audio content extraction module uses Short Time Fourier Transform (STFT) and Mel Frequency Cepstral Coefficients (MFCC) analysis algorithms to complete frame-by-frame decomposition of acoustic parameters. First, the fundamental frequency (F0) is identified to determine the tonic tone and pitch basis. Its calculation range is usually set to 60Hz~1500Hz to match the core frequency band of human hearing. Then, the harmonic spectrum is extracted to analyze the distribution changes of energy at integer multiples of the fundamental frequency to determine the timbre complexity and harmonic structure of the music. Finally, the timbre envelope parameters are extracted, including ADSR model parameters such as attack time, sustain time, and release time. These feature parameters are quantized into a unified high-dimensional timbre vector representation and stored in a track-level audio feature database as the basis for subsequent frequency domain emotion analysis. Based on the extracted core acoustic features, in-depth time-frequency domain analysis is performed to identify the hidden emotional triggering structures in the music. A multi-scale time-frequency decomposition of the audio signal is performed using a combination of Continuous Wavelet Transform (CWT) and Onset Strength Envelope. First, high-resolution frequency domain analysis locates the emotional triggering frequency bands, i.e., the frequency range most likely to evoke emotional resonance in human subjective testing (usually concentrated between 300 and 800 Hz), corresponding to areas such as "soft vocals," "mid-range piano," and "string overtones." Then, the trajectory of frequency energy changes is analyzed in the time dimension to find turning points with abrupt changes, such as suddenly enhanced drum beats, gradual string build-ups, and strong chord transitions. By constructing frequency envelope curves and their derivative functions, key turning points in the gradual emotional structure are extracted, marking the moments of high arousal (arousal spike) and emotional drop points. An "emotional dynamic structure vector" is constructed using the emotional trigger frequency band structure and timestamp index of each song, providing cross-modal anchor points for subsequent integration with lyric semantics and emotional modeling.

[0038] To enhance the depth of emotion understanding in music recommendations, semantic sentiment analysis is performed on the lyrics accompanying the user's music library. First, the lyrics text is obtained synchronously via a lyrics API or embedded tags, supporting multilingual processing, with priority given to word segmentation and part-of-speech tagging for Chinese and English lyrics. Pre-trained language models such as BERT or ERNIE are used to embed sentiment semantic vectors into each line of lyrics, and a sentiment classifier is employed to determine the sentiment polarity (Positive / Neutral / Negative) and intensity score (e.g., continuous values ​​from 0 to 1) for each sentence. Furthermore, to distinguish between subjective emotions and narrative descriptions, a subjectivity detector is introduced, including only sentences with strong subjectivity in the sentiment modeling weight center. In the experimental setup, if the sentiment intensity of a single sentence is >0.7 and three consecutive sentences maintain the same polarity, that segment is identified as an "emotion-dominant segment," and the start time of that segment is used as the trigger point for the lyrics' sentiment. Finally, all lyric sentiment tags are aggregated into a "sentiment tendency time series" for a single track and synchronized to the sentiment temporal modeling module, providing a data foundation for the next step of constructing the sentiment vector space. After completing the semantic analysis of the lyrics, the emotional features of the lyrics for each song are embedded into a unified Lyric Emotional Vector Space (LEVS). This space, based on word-level emotional features, uses Principal Component Analysis (PCA) or t-SNE to reduce the dimensionality of the high-dimensional BERT embeddings, mapping each lyric line to a ternary structure vector based on its emotional polarity (+1 / -1), emotional intensity (0~1), and temporal location information (timestamp). All lyric line vectors constitute the temporal emotional trajectory of the song, forming an "emotional flow path" in the vector space. Simultaneously, emotional center clusters (such as joy, melancholy, anger, and tranquility) are constructed in this space, and the "emotional distance" index for the song is calculated—the Euclidean distance or cosine similarity between the lyric emotional trajectory and each emotional center—serving as an auxiliary reference for emotion classification. In the aggregation layer of multiple songs, K-means clustering analysis can be used to analyze the emotional distribution density in the user's music library. For example, if a user's lyrics are concentrated in the sadness and tranquility emotional regions, their emotion perception recommendation model will prioritize recommending content with an emotionally comforting tendency.

[0039] This approach involves cross-modal joint modeling of the "musical emotion structure" extracted from the audio time-frequency domain and the "emotional trajectory vector" from the lyrics semantic model. First, the data from both modalities are synchronized along the time axis. An "emotional resonance anchor point map" is constructed based on the co-occurrence rate of lyrics timestamps and audio turning points to identify high-confidence emotional co-occurrence segments. Then, multimodal feature fusion techniques (such as Multimodal Compact Bilinear Pooling, MCB) are used to concatenate the lyrics emotion vector and the music frequency trigger vector using tensors, generating a fused emotion vector (Emotion Fusion Tensor). This fusion allows for assessment of whether a particular piece of music exhibits enhanced low-frequency overtones in sad lyrics or synchronized high-intensity rhythmic impacts during emotional climaxes, thus determining its overall emotional expression capability. Finally, a structured global music emotion feature vector is generated for each song (e.g., dominant emotion color = sad, emotional polarity fluctuation = medium, spectral emotional impact level = high, lyrics emotion peak time = 01:47), and all tracks are embedded into a unified music emotion feature map. This graph not only supports rapid querying and sorting based on emotion tags, but also supports various downstream tasks such as user music preferences, emotion history modeling, and intelligent adjustment of recommendation strategies.

[0040] In this embodiment, step S4 includes the following steps: Real-time emotion adaptation analysis is performed on the emotion resonance response data to obtain the music adaptation style; Deep rhythm requirements are calculated based on music adaptation style to generate optimal music rhythm requirements; The similarity of the global music feature map is calculated based on the optimal music rhythm requirement, and threshold filtering is performed based on the preset music rhythm requirement threshold to extract the music set that matches the emotional requirement.

[0041] In this embodiment, after extracting the emotional resonance response data, the personalized adaptation analysis stage is entered to identify the music style type that best matches the user's current emotional state. The core of this step is to map the user's "emotional response factor" vector to a pre-trained "music style emotion adaptation space." This space is built on large-scale emotion-music resonance experimental data, and its dimensions include physiological and emotional characteristic factors such as emotional jump frequency, resonance delay distribution, duration of response zone length, and standard deviation of emotional intensity; corresponding to the label data such as music rhythm intensity, emotional color (valence / arousal), frequency density, and structural change complexity. Support Vector Machine (SVM) or K-Nearest Neighbors (KNN) algorithm is used as a preliminary classification tool to project the current emotional factors into this space, and match the most suitable music style label (such as "bright electronic," "soothing classical," "deep blues," etc.) based on the nearest historical data sample. In addition, a dynamic adjustment mechanism is introduced to consider the trend of resonance performance changes in the currently played music, injecting time weights into the music style prediction (e.g., if the resonance index is greater than 0.85 in the last 10 seconds, it is biased towards the "rhythm enhancement" style). The final output is a music style tag set with confidence weights, providing a basis for setting specific style directions and rhythm intensity for subsequent rhythm matching. After obtaining the music style, the "rhythm requirements" of the user in the current emotional state are further refined—that is, the dynamic quantitative modeling of music rhythm change characteristics and time intensity. This stage first calls the historical rhythm parameter model library based on the music style tags. Each style has a preset rhythm profile template (e.g., electronic dance music corresponds to a BPM range of 120-140 and an average beat density of 0.65). Combined with the emotional intensity gradient and emotional transition rate identified in the user's current resonance response, a personalized weight function is constructed. For example, if the user's current emotional jump frequency is high (>0.4 times / 10s), it is inferred that they tend to have a high-rhythm stimulation rhythm structure. Subsequently, a rhythm-aware augmented network (RNN-TempoNet) is used to fine-tune the regression of the user's rhythm response history, outputting a rhythm requirement vector with multiple dimensions (BPM target value, rhythm density range, beat stability tolerance, etc.). In the experimental scenario, the optimal BPM fluctuation range was set to ±10, with a rhythm density error not exceeding 0.1, ensuring that the matched music could both stimulate emotional resonance and prevent nerve fatigue. Ultimately, this "optimal rhythm requirement" was structured into a dynamic query template, used to subsequently filter candidates with high rhythmic similarity from the music library, and could be adjusted in real time.

[0042] The global music feature atlas is a multi-dimensional database atlas pre-constructed during the training and indexing phases, spanning music style, genre, emotional tags, and rhythmic parameters. Each piece of music in this atlas stores its structural features (such as BPM, mode, pitch energy distribution, rhythm density, beat structure, and emotional color) as a vector, and is labeled with its emotional resonance test records. In this phase, the "optimal music rhythm requirement" vector is used as the query vector. Similarity is calculated for all music vectors using cosine similarity or Mahalanobis distance, and a descending-ordered candidate matching list is constructed. To ensure recommendation accuracy, a rhythm requirement threshold function is set. This function consists of the target BPM center value, the upper limit of density error, and the rhythmic stability tolerance. Only when the dimensional errors of all rhythmic parameters are less than the set threshold (e.g., |ΔBPM| ≤ 8, density difference ≤ 0.15) is a music item considered a "fitting item." Simultaneously, the final matching score is adjusted by combining the weight of emotional tag consistency, excluding items with suitable rhythm but deviating emotional tendencies (such as "upbeat tunes matching depressive states"). Finally, a set of music that satisfies both rhythm and emotion matching is extracted, called the "emotionally appropriate music set," as the candidate recommendation content pool. This set possesses both immediate adaptability to emotional resonance and reflects individual preferences in musical style and structure, serving as the core output for emotional regulation in the entire recommendation process.

[0043] In this embodiment, step S6 includes the following steps: Identify the user's current behavior scenario and real-time behavior parameters, perform user state pattern analysis and real-time scene music demand mining to obtain real-time scene music requirements; Based on real-time scene music needs, the music set is adapted to emotional needs to make adaptive music recommendation decisions for the next song, and a music recommendation strategy is constructed; and real-time recommendation feedback information from users is collected. Based on real-time recommendation feedback information, user ratings and the number of times a user has saved their items are identified to obtain explicit feedback metrics. The playback duration, skip frequency, and replay frequency of recommended music are calculated based on real-time recommendation feedback information to obtain implicit feedback metrics. A comprehensive evaluation of the recommendation effect was conducted based on explicit and implicit feedback indicators, resulting in a music recommendation evaluation report. The music recommendation strategy is intelligently iterated and optimized based on the music recommendation evaluation report in order to implement audio recommendation control.

[0044] In this embodiment, after generating the "real-time scene music demand" vector and obtaining the "emotional demand-matched music set," the dynamic music recommendation decision-making stage begins. The goal of this stage is to select the next song from the candidate music set that best matches the user's current state and to implement strategic control over the entire recommendation process. First, a multi-objective recommendation optimization function is constructed, with core optimization metrics including: maximizing the emotional resonance index, minimizing rhythm fit deviation, maximizing behavioral state style correlation, and merging historical preference weights. For example, the resonance index can account for 40% of the weight in the recommendation function, rhythm matching 20%, historical consistency of style preference 20%, and the remaining 20% ​​reserved for exploratory strategies (such as cold start track placement). Subsequently, reinforcement learning methods (such as context-based Deep Q-Learning) can be selected to execute the recommendation decision, maximizing long-term user satisfaction for the "next song" recommendation. After the recommendation is output, the platform opens a recommendation feedback collection channel in real time to record the user's multi-dimensional reactions to the currently recommended music, including interface interaction behaviors (rating, favorite, skip), playback control behaviors (repeat playback, pause time), and behavioral physiological resonance responses (such as changes in heart rate and facial emotion consistency indicators when continuing to listen). This type of feedback data serves as real-time feedback input, providing a foundation for subsequent strategy optimization. User's proactive behavior after receiving recommended music constitutes the core component of explicit feedback. This feedback typically includes direct evaluation of the recommended content, such as clicking the "like / dislike" button, rating (5 stars), adding to playlists, or sharing. A behavior log is created for each recommended music item, recording whether the user performed these actions, along with the time of the action, the device, the music playback progress, and the corresponding emotional state. For rating behavior, it serves as a direct quantitative indicator, with a 5-star rating defined as high acceptance and below 2 stars as negative feedback. For adding to playlists, a weighted "interest confirmation relationship" between the user and the music item is constructed, which can be used as a "static long-tail interest enhancement factor" in the personalized preference model. In the experiment, to reduce subjective bias in user ratings, a guided rating interaction design was also adopted (e.g., proactively asking "Would you like to receive similar recommendations?" before playback ends). Finally, using daily / weekly statistical periods, an explicit feedback matrix is ​​formed to record the frequency of each user's positive response to the recommendation results and the intensity of their cumulative behavior, which is used to evaluate the acceptance rate, interest consistency, and goal completion of the recommendation algorithm.

[0045] Compared to explicit feedback, implicit feedback is closer to users' natural behavior and can reflect potential interests and true preferences, thus playing an important role in recommendation optimization. Real-time data collection of users' playback duration, skip actions, and replay behaviors during recommended music playback is performed, and then normalized and trend modeled. Specifically, the playback duration percentage (actual playback time / total music length) is a basic indicator of user acceptance; generally, playback exceeding 70% is considered "effective listening." Skip frequency statistics track whether a "skip" action occurs within the first 30 seconds of the recommendation; if three consecutive songs are skipped, a "content mismatch" warning mechanism is triggered. Replay behavior serves as a strong interest indicator, marking the number of times the same user replays a song within 48 hours (e.g., ≥2 times). This data, input into the implicit feedback feature model, generates multiple metrics, such as "average acceptance time," "preferred track replay rate," and "short-term rejection rate," for the evaluation module to score the recommendation effectiveness. In the experiment, the average user skip rate was 15%–25%, while the playback completion rate reached 82% under satisfactory recommendation conditions. These statistical characteristics provided an empirical boundary and target baseline for model parameter tuning. By integrating explicit feedback (such as ratings and favorites) with implicit feedback (such as playback duration and skip rate), a complete recommendation effect evaluation system can be constructed. First, a behavioral weighting mechanism is used to assign rating weights to different indicators, such as 30% for ratings, 30% for playback completion rate, and 20% each for favorites and repeated playback. Then, multidimensional normalization and weighted fusion methods are used to calculate the "Overall Acceptance Score" (OAS) and "Content Matching Score" (CMS) for the current recommendation period. Further, A / B testing is used to compare different recommendation strategies (such as strategies based on resonance factors and strategies based solely on historical preferences), evaluating the effectiveness of the strategies by comparing their feedback performance differences in real time. In addition, an "Emotional Adaptation Consistency Report" is output, analyzing the consistency of the current recommended music's response on the emotional fluctuation curve (such as the average resonance index and resonance delay time). The evaluation report is presented to the model tuning module or operations analysts in the form of a visual panel. The content includes charts such as score distribution histogram, skip rate trend line, and user activity heatmap, which support the data feedback loop of the intelligent recommendation mechanism.

[0046] Based on the phased music recommendation evaluation report, the system enters the intelligent strategy optimization phase, the core of which is to construct a self-circulating intelligent agent mechanism of "recommendation-feedback-adjustment". First, explicit and implicit feedback are used as training samples input into a reinforcement learning framework, such as the Deep Reinforcement Learning with Attention (DRLA) structure, to optimize the policy function in the policy network regarding "when to recommend which style, rhythm, and mood combination". The state space definition includes the user's current emotional state, behavioral scenario, historical preference model, and music feature embedding; the reward function design is based on user feedback data (e.g., high rating +1, skip -1, favorite +0.5, low rating -0.8, etc.). Through continuous interactive training, the agent can form recommendations with adaptive adjustment capabilities, able to perceive the effect of the current recommendation strategy and dynamically transfer strategies. Simultaneously, diverse incentive mechanisms (such as Softmax temperature control) are introduced into the construction of the recommendation content pool to avoid user fatigue caused by over-recommendation of a single style. Ultimately, this intelligent recommendation agent not only maximizes the resonance between emotion and behavior at the personalization level but also achieves a dual improvement in recommendation accuracy and long-term user satisfaction throughout the entire lifecycle.

[0047] In this embodiment, the specific steps of identifying the user's current behavior scenario and real-time behavior parameters, performing user state pattern analysis and real-time scene music demand mining to obtain real-time scene music demand include the following steps: Integrate user geolocation information to perform real-time user scenario analysis in order to obtain the user's current behavioral scenario; Identify real-time user behavior parameters and perform user state pattern analysis based on the current user behavior scenario to generate the current behavior state pattern; Perform emotional evolution analysis on the current behavioral state pattern to generate user emotional evolution patterns; Predict music preference migration trends based on current behavioral patterns and generate music preference migration trend features; Based on the patterns of user emotional evolution and the characteristics of music preference migration, real-time scene music demand is mined to obtain real-time scene music requirements.

[0048] In this embodiment, to achieve more accurate music recommendation responses, real-time scene analysis needs to be performed by integrating user geographic location information to perceive the user's current behavioral scenario (such as "commuting", "resting at home", "exercising outdoors", "social occasions"). First, the GPS module of the user's terminal device (such as a smartphone or smartwatch) is called to obtain the user's current latitude and longitude coordinates. Simultaneously, indoor positioning accuracy can be enhanced through Wi-Fi environmental fingerprinting, with the error controlled within 5-10 meters. Then, the coordinates are reverse geocoded using map APIs (such as Amap or Google Places) to obtain surrounding environment labels (such as "shopping mall", "subway", "office building"), and the time dimension (such as weekday morning rush hour, weekend night) is integrated to model the scene context. Furthermore, the location labels are jointly clustered with historical user behavior data at that location (such as heart rate changes, music listening style, and emotional state records) to train a location-behavior scene classifier (such as a multi-label classification model based on XGBoost) to assign a high-confidence "behavior scene" label to the current geographic location. For example, if the location is a CBD office building at 9 AM, and considering the user's movement speed is less than 1 km / h and their past preference for listening to light music in this area, the current scenario can be inferred to be a "quiet preparation period before work." The final scenario label serves as crucial prior information for subsequent behavior analysis and music preference adjustment. After determining the user's behavior scenario, further data collection and analysis of current behavioral parameters are conducted to construct a complete behavioral state pattern. Behavioral parameter collection is based on smart wearable devices and mobile phone sensors (accelerometers, gyroscopes, pedometers, etc.), with key indicators including user activity intensity (MET value estimation), cadence, standing time, phone holding status, voice activity, and touch frequency. For example, in an "outdoor scenario," if a user's average step frequency is detected to be 120 steps / minute, body vibration frequency is higher than 2.0Hz for more than 10 minutes, and heart rate reaches the aerobic zone (e.g., 120-150 bpm), then the current state can be determined as "moderate to high intensity exercise behavior." Conversely, if the user's heart rate is low, the distance traveled is short, the touch screen frequency is high, and voice conversations are detected, it may be a "static social state." These data features are uniformly input into a pre-trained multimodal behavior recognition model (e.g., a time-series multi-label recognition network based on a Transformer structure) to generate current behavior state labels (e.g., "commuting while sitting," "brisk walking," "light exercise," "relaxing at home," etc.). In addition, state stability correction can be performed based on temporal context (e.g., weekday evening), historical behavior trends, and user behavior habits to ensure that the identified behavior state patterns have semantic stability and behavioral continuity, providing strong background support for subsequent emotion evolution modeling.

[0049] After identifying the user's current behavioral pattern, the model further analyzes the influence of this pattern on the evolution of the user's emotional state, establishing an evolutionary mapping relationship from "behavior → emotion." This step is based on training and modeling using historical behavior-emotion data pairs, utilizing models with memory structures (such as Bi-LSTM or Temporal Convolutional Network) to jointly model the time series of behavioral states and the emotional fluctuation series. The input data includes the emotional change trends of the same behavioral state over multiple historical days, such as a gradual increase in valence and a stable arousal when sitting at home; while valence fluctuates greatly in the short term but shows a clear overall upward trend during exercise. The model outputs the expected emotional evolution trajectory under the current behavioral state, including the directional trend (increasing / decreasing / stable) of valence and arousal over a future period, the maximum fluctuation amplitude, and the reaction lag time. Individual difference factors are also introduced, such as the user's emotional stability index (SEI) and emotional jump sensitivity, to personalize the model's prediction results and ensure that the evolutionary pattern closely reflects the user's own emotional rhythm. For example, if a user frequently experiences emotional depression during multiple commutes, the model will adjust its prediction trajectory to a pattern of decreasing arousal and low valence fluctuations, providing a basis for scene-based emotion-modulating music recommendations. Based on understanding the current behavioral state, the model further predicts whether the user's music preferences will shift in the current state, as well as the direction, degree, and rhythmic structure shift characteristics of this shift. The core of this module lies in building a behavior-music preference transfer model. It calls upon a database of historical user behavior states and music styles to analyze music selection records under similar behavioral states. Through collaborative filtering and trend-enhanced time series prediction (such as Prophet or ARIMA), it models the trajectory of music preference changes during different state transitions. For example, when a user transitions from a "resting" state to a "light exercise" state, their preferred music often gradually shifts from "light music" to "electronic pop" or "rhythm and blues," while under "high-intensity exercise," they may prefer high-BPM, low-key music. The predicted trend features include: target style shift (e.g., from style code 4 → 6), rhythm target value (BPM increase / decrease), and melody density change trends. These features not only reflect the direction of preference changes but also reveal the dynamic nature of how user behavior influences music selection. For example, if the current behavior is "brisk walking," it may predict that future preferences will transition from lyrical music to high-energy, rhythmic music within 3 minutes, and the migration trend will be labeled as "progressive strong stimulus shift." By combining the user's current behavioral emotion evolution trend with the predicted music preference migration features, the real-time music needs of users in specific geographical and behavioral scenarios are mined to form dynamic recommendation inputs.First, the "emotional evolution trajectory" and "music preference transfer features" are merged into a unified multi-dimensional demand vector (called Scene-Adaptive MusicDemand Vector, SMDV), which includes dimensions such as expected BPM range, melody density, emotional tonality range (valence / arousal range), tempo complexity tolerance, and music style confidence vector. This vector represents the music demand form that the user is most likely to resonate with in their current state. Next, a real-time matching algorithm (such as nearest neighbor matching or deep similarity embedding) is used to quickly search the music library to confirm music content that matches this demand feature. Furthermore, "behavioral scenario adjustment weights" are set according to the user's current scenario type. For example, in a "sitting quietly at work" scenario, the rhythm intensity weight is automatically reduced and the melody stability tolerance is increased to ensure that recommended music does not interrupt the user's attention. Finally, this "real-time scene music demand" is output as a set of dynamic preference parameters with strong context adaptability, providing real-time input for the recommendation model, enabling music recommendations to achieve high matching degree and dynamic adaptability under multi-dimensional coupling of time, space, behavior, and emotion.

[0050] In this embodiment, an intelligent audio recommendation control system is provided for executing the intelligent audio recommendation control method described above, including: The emotional state evolution module is used to detect real-time facial images of users during the initial music playback process, perform multi-time period emotional evolution modeling, and construct user emotional state evolution curves. The emotional resonance module is used to identify the initial music played, perform emotional resonance matching based on the user's emotional state evolution curve, and obtain emotional resonance response data. The music feature graph module is used to construct a global music feature graph for the user's music library; The emotional demand adaptation module is used to perform deep rhythmic demand analysis on the emotional resonance response data based on the global music feature map and extract the emotional demand adaptation music set. The adaptive music recommendation module is used to make adaptive music recommendation decisions based on the music set adapted to emotional needs, and to perform recommendation control execution.

[0051] Therefore, the embodiments should be considered as exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of the equivalents of the application are intended to be included within the invention.

[0052] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement it. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein are implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features of the invention herein.

Claims

1. A method for intelligent audio recommendation control, characterized in that, Includes the following steps: Step S1: Detect real-time facial images of the user during the initial music playback process, perform multi-time period emotion evolution modeling, and construct the user's emotional state evolution curve; Step S2: Identify the initial music played, perform emotional resonance matching based on the user's emotional state evolution curve, and obtain emotional resonance response data; Step S3: Construct a global music feature map of the user's music library; Step S4: Perform deep rhythm demand analysis on the emotional resonance response data based on the global music feature map, and extract the music set that matches the emotional demand. Step S5: Make adaptive music recommendation decisions based on the music set adapted to emotional needs, and implement recommendation control.

2. The intelligent audio recommendation control method according to claim 1, characterized in that, The specific steps of step S1 are as follows: Detects real-time facial images of the user during the initial music playback process; Adaptive contrast optimization is performed on the real-time facial image to generate an adaptively optimized facial image; Multimodal physiological state response perception is performed based on adaptively optimized facial images, and a multimodal physiological state perception matrix is ​​constructed. Decode the potential emotional state of the multimodal physiological state perception matrix to generate potential emotional fluctuation features of users. The potential emotional fluctuation characteristics of users are decomposed in a multi-level manner to extract the emotional intensity gradient, emotional transition rate and emotional stability. Based on the gradient of emotional intensity, the rate of emotional transition, and the stability of emotional persistence, a multi-period emotional evolution model is constructed to build a curve of user emotional state evolution.

3. The intelligent audio recommendation control method according to claim 2, characterized in that, The specific steps for constructing a multimodal physiological state perception matrix based on adaptively optimized facial images to perceive multimodal physiological state responses are as follows: Perform dynamic eye optical flow tracking on adaptively optimized facial images to extract the user's eye movement trajectory; The user's pupil constriction frequency and blink count are calculated based on adaptively optimized facial images; Multidimensional temporal state mining of the user's eyeball drift trajectory, pupil contraction frequency and blink count is performed to generate user temporal eyeball state features. The facial muscle tension and relaxation micro-changes are calculated on the adaptively optimized facial image to extract the facial muscle tension and relaxation degree. Based on facial muscle tension and relaxation, deep micro-expression recognition is performed to obtain the user's real-time micro-expression features; Based on collecting users' heart rate parameters using smart wearable devices; The user's heart rate parameters are analyzed for changes over multiple eras to generate a user heart rate fluctuation curve; The system performs time-stamp synchronization calibration on the user's heart rate fluctuation curve, real-time micro-expression features, and temporal eye state features, and performs multimodal physiological state response perception to construct a multimodal physiological state perception matrix.

4. The intelligent audio recommendation control method according to claim 1, characterized in that, The specific steps of step S2 are as follows: Spectral analysis was performed on the initially played music to obtain the audio rhythm and pitch frequency components; Based on the frequency components of audio rhythm and pitch, the characteristics of different musical stages are analyzed to generate rhythmic characteristic curves for different musical stages. Based on the rhythm feature curve, the user's emotional state evolution curve is mined for emotion-music resonance, and multi-time point emotional resonance matching calculation is performed to generate an emotional resonance index curve. Based on the emotion resonance index curve, feature nodes of emotion jump, continuous response, and delayed resonance are captured to generate emotion resonance response data.

5. The intelligent audio recommendation control method according to claim 1, characterized in that, Step S3 is as follows: Identify audio files in the user's music library and perform acoustic feature analysis to extract fundamental frequency, overtones, and timbre envelope feature parameters; Based on the fundamental frequency, overtones, and timbre envelope characteristic parameters, time-frequency domain decomposition is performed to identify the emotional trigger frequency bands and emotional transition points of each piece of music. The semantic features of lyrics in audio files are analyzed to obtain the emotional polarity and emotional intensity. Construct a lyrics emotion vector space based on emotional polarity and emotional intensity; Cross-modal emotional feature fusion is performed on the lyric emotional vector space, emotional trigger frequency band, and emotional transition turning point, and global music fitting is performed to construct a global music feature map of the user's music library.

6. The intelligent audio recommendation control method according to claim 1, characterized in that, The specific steps of step S4 are as follows: Real-time emotion adaptation analysis is performed on the emotion resonance response data to obtain the music adaptation style; Deep rhythm requirements are calculated based on music adaptation style to generate optimal music rhythm requirements; The similarity of the global music feature map is calculated based on the optimal music rhythm requirement, and threshold filtering is performed based on the preset music rhythm requirement threshold to extract the music set that matches the emotional requirement.

7. The intelligent audio recommendation control method according to claim 1, characterized in that, The specific steps of step S5 are as follows: Identify the user's current behavior scenario and real-time behavior parameters, perform user state pattern analysis and real-time scene music demand mining to obtain real-time scene music requirements; Based on real-time scene music needs, the music set is adapted to emotional needs to make adaptive music recommendation decisions for the next song, and a music recommendation strategy is constructed; and real-time recommendation feedback information from users is collected. Based on real-time recommendation feedback information, user ratings and the number of times a user has saved their items are identified to obtain explicit feedback metrics. The playback duration, skip frequency, and replay frequency of recommended music are calculated based on real-time recommendation feedback information to obtain implicit feedback metrics. A comprehensive evaluation of the recommendation effect was conducted based on explicit and implicit feedback indicators, resulting in a music recommendation evaluation report. The music recommendation strategy is intelligently iterated and optimized based on the music recommendation evaluation report in order to implement audio recommendation control.

8. The intelligent audio recommendation control method according to claim 7, characterized in that, The specific steps for identifying the user's current behavior scenario and real-time behavior parameters, performing user state pattern analysis and real-time scene music demand mining to obtain real-time scene music demand are as follows: Integrate user geolocation information to perform real-time user scenario analysis in order to obtain the user's current behavioral scenario; Identify real-time user behavior parameters and perform user state pattern analysis based on the current user behavior scenario to generate the current behavior state pattern; Perform emotional evolution analysis on the current behavioral state pattern to generate user emotional evolution patterns; Predict music preference migration trends based on current behavioral patterns and generate music preference migration trend features; Based on the patterns of user emotional evolution and the characteristics of music preference migration, real-time scene music demand is mined to obtain real-time scene music requirements.

9. An intelligent audio recommendation control system, characterized in that, For performing the intelligent audio recommendation control method as described in claim 1, including: The emotional state evolution module is used to detect real-time facial images of users during the initial music playback process, perform multi-time period emotional evolution modeling, and construct user emotional state evolution curves. The emotional resonance module is used to identify the initial music played, perform emotional resonance matching based on the user's emotional state evolution curve, and obtain emotional resonance response data. The music feature graph module is used to construct a global music feature graph for the user's music library; The emotional demand adaptation module is used to perform deep rhythmic demand analysis on the emotional resonance response data based on the global music feature map and extract the emotional demand adaptation music set. The adaptive music recommendation module is used to make adaptive music recommendation decisions based on the music set adapted to emotional needs, and to perform recommendation control execution.

Images