Music recommendation methods, devices, electronic equipment and storage media

By collecting the user's current signals through heart rate and voice acquisition components, the system determines the user's emotional state and recommends target music, solving the problem that existing music recommendation systems cannot adapt to the user's emotions in real time and improving the effect of emotion regulation.

CN123115456APending Publication Date: 2026-11-17GOERTEK INC
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611168659.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-08-03
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

Existing music recommendation systems rely on historical listening records and struggle to adapt to users' emotional states in real time, resulting in a lack of proactive adjustment of user emotions and exhibiting lag and subjectivity.

Method used

The system collects the user's current heart rate and voice signals using a heart rate acquisition unit and a voice acquisition unit, respectively, determines the current fusion features of the user's emotional state, and recommends target music based on the emotion recognition results.

Benefits of technology

It enables matching music based on the user's current emotional state, reduces the lag in historical playlist recommendations, and improves the effectiveness of proactively regulating the user's emotions.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123115456A_ABST
    Figure CN123115456A_ABST
Patent Text Reader

Abstract

This application discloses a music recommendation method, apparatus, electronic device, and storage medium, relating to the field of audio technology. The method is applied to an electronic device equipped with a heart rate acquisition component and a voice acquisition component, comprising: acquiring the user's current heart rate signal through the heart rate acquisition component and the user's current voice signal through the voice acquisition component; determining a current fusion feature representing the user's emotional state based on the current heart rate signal and the current voice signal; performing emotion recognition on the user based on the current fusion feature to obtain an emotion recognition result; matching the corresponding target music based on the emotion recognition result and recommending the target music to the user for playback. Compared to existing methods that rely on historical listening records for music recommendation, this application can perceive the user's emotional state through heart rate and voice signals, thereby matching and playing appropriate target music, improving the effectiveness of proactively regulating the user's emotions.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of audio technology, and more particularly to a music recommendation method, apparatus, electronic device, and storage medium. Background Technology

[0002] Currently, with the fast pace of life, mental health is receiving increasing attention, and music has been proven to have a significant effect on regulating emotions. In everyday life scenarios (such as commuting, working in the office, or resting at home), users generally regulate their emotions by playing music.

[0003] During music playback, devices typically rely on historical listening records to generate recommended playlists for automatic music playback. However, these recommended playlists often fail to adapt to the user's current emotional state, exhibiting a degree of lag and subjectivity, making it difficult to proactively regulate the user's emotions. For example, a user feeling tired and anxious due to work stress on their way home might want to listen to some soothing music to relax, but if the headphones still recommend sad songs based on their past preferences, it could actually exacerbate their negative emotions. Summary of the Invention

[0004] The main purpose of this application is to provide a music recommendation method, device, electronic device and storage medium, which aims to solve the technical problem that existing music recommendation methods rely on historical listening records and are difficult to adapt to the user's emotional state in real time.

[0005] To achieve the above objectives, this application proposes a music recommendation method, which is applied to an electronic device equipped with a heart rate acquisition component and a voice acquisition component. The method includes: The user's current heart rate signal is acquired through the heart rate acquisition component, and the user's current voice signal is acquired through the voice acquisition component. Based on the current heart rate signal and the current voice signal, determine the current fusion feature representing the user's emotional state; Based on the current fusion features, the user's emotion is identified to obtain the emotion identification result; Based on the emotion recognition results, a corresponding target music is matched, and the target music is recommended to the user for playback.

[0006] In one embodiment, the step of determining the current fusion feature representing the user's emotional state based on the current heart rate signal and the current voice signal includes: Heart rate parameters and heart rate variability parameters are extracted from the current heart rate signal, and the user's current physiological characteristics are determined based on the heart rate parameters and the heart rate variability parameters. Extract the envelope spectrum corresponding to the current speech signal, and analyze the envelope spectrum to obtain speech frequency parameters and speech amplitude parameters; Based on the speech frequency parameters and the speech amplitude parameters, the corresponding initial emotional features are determined; The initial emotional features and the current physiological features are fused to obtain the current fused features that characterize the user's emotional state.

[0007] In one embodiment, the electronic device further includes an inertial acquisition component, and the step of fusing the initial emotional features and the current physiological features to obtain current fused features characterizing the user's emotional state further includes: The inertial acquisition unit acquires the user's acceleration and angular velocity data. The user's motion posture is determined based on the angular velocity data, and the user's motion gait is determined based on the acceleration data; Based on the motion posture and the motion gait, the user's current motion characteristics are determined; The initial emotional features, the current physiological features, and the current motion features are concatenated to obtain the current fusion features that characterize the user's emotional state.

[0008] In one embodiment, the step of matching the corresponding target music based on the emotion recognition result and recommending the target music to the user for playback includes: The emotion recognition results are matched against a preset emotion list to obtain a target emotion tag, and the positive or negative emotion attributes of the target emotion tag are determined. Matching the positive and negative emotion attributes and the target emotion tags in a preset music library yields at least two target music tracks. The preset music library includes a variety of music tracks with various emotion tags. A playlist to be recommended is generated based on the target music, and the playlist is recommended to the user for playback.

[0009] In one embodiment, after the step of generating a playlist to be recommended based on each of the target music tracks and recommending the playlist to the user for playback, the method further includes: Obtain the playback completion rate of each target song in the playlist to be recommended during playback; If the playback completion rate is lower than a preset threshold, the target music corresponding to the lower than the preset threshold will be regarded as music to be optimized. Extract the current emotion tag corresponding to the music to be optimized, and refine and adjust the current emotion tag of the music to be optimized to obtain emotion sub-tags; The preset emotion list is updated based on the emotion sub-tags; Based on the updated preset emotion list, the process returns to the step of collecting the user's current heart rate signal through the heart rate acquisition component, obtains the updated playlist to be recommended, and recommends the updated playlist to the user for playback.

[0010] In one embodiment, prior to the step of acquiring the user's current heart rate signal via the heart rate acquisition component, the following steps are included: Acquire the user's historical heart rate signal, historical voice signal, and corresponding first historical emotion result; Determine the first physiological feature corresponding to the historical heart rate signal and the first emotional feature corresponding to the historical voice signal, and fuse the first physiological feature and the first emotional feature to obtain the first fused feature; The model is trained based on the first fusion feature and the first historical emotion result to obtain a preset emotion recognition model; The step of performing emotion recognition on the user based on the current fusion features to obtain the emotion recognition result includes: The current fused features are used to perform emotion recognition using the preset emotion recognition model to obtain the emotion recognition result.

[0011] In one embodiment, the step of performing emotion recognition on the user based on the current fusion features to obtain an emotion recognition result includes: Obtain at least two second fusion features of the user within a preset time period, and the second historical sentiment result corresponding to the second fusion features; Based on each of the second fusion features and each of the second historical sentiment results, a corresponding temporal feature sequence is constructed; Based on the temporal feature sequence and the current fused features, the user's emotional state is subjected to temporal evolution to obtain an emotional transition probability distribution; The emotional state with the highest probability in the emotional transfer probability distribution is taken as the emotion recognition result.

[0012] Furthermore, to achieve the above objectives, this application also proposes a music recommendation device, the device comprising: The acquisition module is used to acquire the user's current heart rate signal through the heart rate acquisition component and to acquire the user's current voice signal through the voice acquisition component; The fusion module is used to determine the current fusion feature representing the user's emotional state based on the current heart rate signal and the current voice signal; The recognition module is used to perform emotion recognition on the user based on the current fused features, and obtain the emotion recognition result; The recommendation module is used to match the corresponding target music based on the emotion recognition result and recommend the target music to the user for playback.

[0013] In addition, to achieve the above objectives, this application also proposes an electronic device, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the music recommendation method as described above.

[0014] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and which, when executed by a processor, implements the steps of the music recommendation method described above.

[0015] One or more technical solutions proposed in this application have at least the following technical effects: The music recommendation method of this application is applied to an electronic device equipped with a heart rate acquisition component and a voice acquisition component. The method includes: acquiring the user's current heart rate signal through the heart rate acquisition component and acquiring the user's current voice signal through the voice acquisition component; determining a current fusion feature representing the user's emotional state based on the current heart rate signal and the current voice signal; performing emotion recognition on the user based on the current fusion feature to obtain an emotion recognition result; matching a corresponding target music based on the emotion recognition result and recommending the target music to the user for playback.

[0016] This application first collects the user's current heart rate and voice signals using a heart rate acquisition unit and a voice acquisition unit, respectively. Then, it determines the current fusion features representing the user's emotional state based on the current heart rate and voice signals, and identifies the emotion recognition result based on the current fusion features to match corresponding target music for recommendation to the user. Compared to existing methods that rely on historical listening records for music recommendation, this application can perceive the user's emotional state through heart rate and voice signals, thereby matching and playing appropriate target music. This reduces recommendation bias caused by the lag in historical recommended playlists, which makes it difficult to adapt to changes in user emotions, and improves the effectiveness of proactively regulating user emotions. Attached Figure Description

[0017] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart illustrating an embodiment of the music recommendation method of this application. Figure 2 This is an overall flowchart of music recommendation via headphones provided in Embodiment 1 of this application; Figure 3 This is a schematic diagram of the model training process provided in Embodiment 2 of the music recommendation method of this application; Figure 4 This is a schematic diagram of the music matching process provided in Embodiment 3 of the music recommendation method of this application; Figure 5 This is a block diagram of the module structure of the music recommendation device according to an embodiment of this application; Figure 6 This is a schematic diagram of the hardware operating environment involved in the electronic device in the embodiments of this application.

[0020] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0021] To make the above-mentioned objectives, features, and advantages of this application more apparent and understandable, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this application, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0022] The main solution proposed in this application is as follows: Currently, with the fast pace of life, mental health is receiving increasing attention, and music has been proven to have a significant effect on regulating emotions. In daily life scenarios (such as commuting, working in the office, or resting at home), users generally regulate their emotions by playing music.

[0023] During music playback, devices typically rely on historical listening records to generate recommended playlists for automatic music playback. However, these recommended playlists often fail to adapt to the user's current emotional state, exhibiting a degree of lag and subjectivity, making it difficult to proactively regulate the user's emotions. For example, a user feeling tired and anxious due to work stress on their way home might want to listen to some soothing music to relax, but if the headphones still recommend sad songs based on their past preferences, it could actually exacerbate their negative emotions.

[0024] To address the aforementioned issues, this application provides a music recommendation method. First, a heart rate acquisition component and a voice acquisition component collect the user's current heart rate signal and current voice signal, respectively. Then, based on the current heart rate signal and current voice signal, a current fusion feature representing the user's emotional state is determined. The emotion recognition result is then identified based on the current fusion feature to match corresponding target music and recommend it to the user. Compared to existing methods that rely on historical listening records for music recommendation, this application can perceive the user's emotional state through heart rate and voice signals, thereby matching and playing appropriate target music. This reduces recommendation bias caused by the lag in historical recommended playlists, which makes it difficult to adapt to changes in user emotions, and improves the effectiveness of proactively regulating user emotions.

[0025] Based on this, this application proposes a music recommendation method according to the first embodiment, referring to... Figure 1 , Figure 1 This is a flowchart illustrating an embodiment of the music recommendation method of this application. In this embodiment, the music recommendation method is applied to an electronic device equipped with a heart rate acquisition component and a voice acquisition component, and the music recommendation method may include steps S10~S40: Step S10: Collect the user's current heart rate signal through the heart rate acquisition component, and collect the user's current voice signal through the voice acquisition component.

[0026] It should be noted that the executing entity of this application embodiment can be an electronic device with data acquisition, network communication, and music recommendation functions, such as smart headphones, headphones, mobile phones, music players, and other electronic devices capable of executing the music recommendation method of this application. This embodiment does not limit this. The following uses a smart headphone as an example to describe this embodiment and the following embodiments. It should be noted that the music recommendation method of this embodiment can be executed entirely by the headphone itself, or it can work in conjunction with a matching mobile terminal or cloud server. For ease of understanding, this embodiment and the following embodiments are described using the example of the headphone itself executing the music recommendation method entirely.

[0027] It should be understood that the heart rate acquisition component can refer to a sensor integrated within the aforementioned electronic device for acquiring physiological signals related to the user's heartbeat. For example, a photoelectrovascular volumetric sensor can emit light of a specific wavelength to irradiate skin tissue and receive reflected or transmitted light. Utilizing the difference in light absorption caused by the periodic changes in blood volume with heartbeats, it outputs a raw electrical signal reflecting changes in heart rate.

[0028] Understandably, the current heart rate signal can be the raw time series data containing heart rhythm information collected by the heart rate acquisition component of the smart earphone in real-time working state. This raw time series data is usually represented as a pulsating waveform signal that fluctuates periodically with the heartbeat.

[0029] It should be understood that the voice acquisition component can be an electroacoustic conversion device integrated inside the smart earphone, used to acquire the user's voice or ambient sound. It can convert sound wave vibrations into analog electrical signals and output a digitized voice signal after analog-to-digital conversion; examples include microelectromechanical system (MEMS) microphones and electret condenser microphones, etc., and this embodiment does not limit this. The current voice signal can be digitized audio stream data containing the user's voice content, acquired by the voice acquisition component at the current moment during real-time operation of the smart earphone.

[0030] In actual use, after confirming that the smart earphone is being worn by the user, it uses a built-in heart rate acquisition component to continuously irradiate and receive reflected light from the user's ear skin tissue to collect the current heart rate signal reflecting the heart's rhythm in real time. At the same time, it also uses a built-in voice acquisition component to pick up the sound waves generated when the user speaks, converts the sound wave vibrations into analog electrical signals, and then performs analog-to-digital conversion to generate the current voice signal.

[0031] Step S20: Determine the current fusion feature representing the user's emotional state based on the current heart rate signal and the current voice signal.

[0032] It should be noted that the current fusion feature can be a vectorized representation generated by the smart headphones after processing the current heart rate signal and the current voice signal from different sensors, which can comprehensively represent the user's emotional state at the current moment. This current fusion feature can be extracted from the heart rate signal and the voice signal respectively and combined after normalization or splicing fusion operations.

[0033] For example, the current fusion feature can be a one-dimensional feature vector containing multiple elements such as heart rate value, heart rate variability value, speech rate value, speech energy value, and speech spectrum centroid value, or it can be a multi-dimensional tensor formed by splicing multiple feature matrices. Its function is to unify heterogeneous physiological signal features and speech signal features into the same representation space for subsequent emotion recognition.

[0034] To facilitate understanding, the following example illustrates the concept. For instance, in a specific application scenario, after a smart headset acquires a 5-second current heart rate signal and a 5-second current voice signal, it first performs bandpass filtering on the current heart rate signal, calculating the user's current heart rate to be 78 beats per minute, with the root mean square of the difference between adjacent heartbeats in heart rate variability being 32 milliseconds. Simultaneously, it performs frame-by-frame processing on the current voice signal, extracting the short-time energy of 0.25 joules, the fundamental frequency mean of 220 Hz, and the first-order difference mean vector of the Mel frequency cepstral coefficients. These seven feature parameters are then concatenated sequentially into a seven-dimensional feature vector, which is then Z-score standardized to ensure that the mean of each dimension is 0 and the standard deviation is 1. This results in a standardized seven-dimensional current fusion feature vector, which serves as the fusion feature data representing the user's current emotional state.

[0035] In practical use, after acquiring the current heart rate signal and the current voice signal, the smart earphone first filters and denoises the current heart rate signal to eliminate motion artifacts and power frequency interference. Then, it extracts the heart rate value and heart rate variability from the denoised heart rate signal. The heart rate variability can further include sub-parameters such as the root mean square of the difference between adjacent heartbeats and the low-frequency to high-frequency power ratio. Simultaneously, it pre-emphasizes, frames, and windows the current voice signal, extracting the time-domain and frequency-domain features of the voice in each frame, including at least the short-time energy, zero-crossing rate, fundamental frequency trajectory, and Mel-frequency cepstral coefficients. Next, the extracted heart rate parameters and voice parameters are concatenated in sequence to form a current fusion feature that integrates the autonomic nervous activity information reflected by the heart rate signal and the acoustic emotional information reflected by the voice signal.

[0036] Step S30: Perform emotion recognition on the user based on the current fusion features to obtain the emotion recognition result.

[0037] Understandably, the emotion recognition result can be a labeled conclusion used to describe the user's current emotion category. This emotion recognition result can be represented as a discrete emotion category label, such as happy, sad, calm, excited, angry, etc., or it can be represented as a quantitative value of emotion on a continuous dimension, such as two-dimensional coordinate values ​​of valence and arousal. This implementation does not impose any restrictions on this.

[0038] To facilitate understanding, the following example illustrates the concept. For instance, in a specific application scenario, the smart earphones have determined that the current fused feature is a standardized feature vector comprising dimensions such as heart rate value, heart rate variability value, short-time energy of speech, mean fundamental frequency of speech, and Mel-frequency cepstral coefficients. The smart earphones can input this feature vector into an internally stored lightweight neural network emotion recognition model. After calculation, it outputs a four-dimensional probability vector, where the probability value for the "happy" category is 0.72, for the "calm" category is 0.15, for the "sad" category is 0.08, and for the "excited" category is 0.05. Finally, the "happy" category, with the highest probability value, is selected as the current user's emotion recognition result.

[0039] In practical use, after determining the current fused features, the smart earphones can use these features as input data to pass them to the emotion recognition model running inside the earphones, ultimately outputting a probability vector containing probability values ​​for each category. Then, the emotion category with the highest probability value is selected from this probability vector as the emotion recognition result for the current moment.

[0040] Step S40: Match the corresponding target music based on the emotion recognition result, and recommend the target music to the user for playback.

[0041] It should be understood that the target music can be one or more musical pieces selected by the smart headphones from a pre-set music library, either locally or in the cloud, that match the identified user's emotional state, based on emotion recognition results. It should be noted that each piece in this music library is pre-labeled with a corresponding emotion tag, such as "exhilarating," "soothing," "cheerful," or "calm." These tags can be automatically generated through artificial intelligence analysis of audio features (such as rhythm, tonality, and timbre) or manually labeled; this embodiment does not impose any limitations on this.

[0042] To facilitate understanding, the following example illustrates this. For instance, after emotion recognition, the smart headphones determine the emotion as "calm." Then, using "calm" as the query tag, they search a locally stored music library containing 200 tracks. Of these, 25 are tagged "soothing," 30 "energetic," 20 "cheerful," and 18 "serene." Next, they select 43 tracks from the two categories most closely related to "calm"—"soothing" and "serene"—as target music, automatically adding them to a playlist. Finally, they recommend these target tracks to the user for playback.

[0043] In practical use, after obtaining the emotion recognition results, the smart earphones can use the emotion recognition results as query conditions to search the music library for music that matches the emotion recognition results as target music, and recommend the target music to the user for playback.

[0044] It's worth noting that when the smart headphones cannot find enough matching music in the local music library, they can wirelessly connect to a cloud music server, send a music query request containing emotion recognition results to the cloud, and receive a list of matching music returned by the cloud as the target music.

[0045] Furthermore, in order to obtain the aforementioned current fusion features, in this embodiment, the step of determining the current fusion features characterizing the user's emotional state based on the current heart rate signal and the current voice signal includes: Step S21: Extract heart rate parameters and heart rate variability parameters from the current heart rate signal, and determine the user's current physiological characteristics based on the heart rate parameters and the heart rate variability parameters.

[0046] It should be noted that heart rate parameters can be quantitative indicators extracted from the current heart rate signal to describe the basic rhythm of heartbeats, such as the heart rate value corresponding to the number of heartbeats per unit time. Heart rate variability parameters can be quantitative indicators extracted from the current heart rate signal to describe the degree of fluctuation in the time interval between adjacent heartbeats, and may include sub-parameters such as the root mean square of the difference between adjacent heartbeat intervals, the standard deviation of the difference between adjacent heartbeat intervals, and the ratio of low-frequency power to high-frequency power. These parameters are highly correlated with emotional states such as stress and relaxation.

[0047] Understandably, the current physiological characteristics can be a vectorized representation of the user's current autonomic nervous system activity state, formed by combining the aforementioned heart rate parameters and heart rate variability parameters. For example, heart rate values ​​and heart rate variability values ​​can be concatenated into a two-dimensional feature vector.

[0048] Specifically, after acquiring the current heart rate signal, the smart earphones can perform a peak detection algorithm to identify the peak position corresponding to each heartbeat and calculate the time interval between adjacent peaks. Then, based on the detected peak time series, the number of heartbeats per unit time is counted to obtain heart rate parameters. Simultaneously, the standard deviation and root mean square difference of the interval series between adjacent peaks are calculated to obtain heart rate variability parameters. Finally, the obtained heart rate parameters and heart rate variability parameters are combined according to the feature dimensions to form the current physiological characteristics.

[0049] Step S22: Extract the envelope spectrum corresponding to the current speech signal, and analyze the envelope spectrum to obtain speech frequency parameters and speech amplitude parameters.

[0050] It should be understood that the envelope spectrum can be obtained by extracting the envelope of the current speech signal, and is a spectral representation of the outer envelope line reflecting the change of the speech signal amplitude over time. Specifically, smart headphones can use the Hilbert transform method to process the speech signal, transforming the original time-domain speech signal into an analytic signal, calculating the magnitude of the analytic signal to obtain the envelope line of the speech signal, and then performing a Fourier transform on the envelope line to obtain the envelope spectrum.

[0051] Understandably, speech frequency parameters can be quantitative indicators related to the frequency components extracted from the envelope spectrum, such as the main peak frequency or centroid frequency of the envelope spectrum. Speech amplitude parameters can be quantitative indicators related to the amplitude components extracted from the envelope spectrum, such as the peak amplitude or the average amplitude of each frequency component of the envelope spectrum.

[0052] Step S23: Determine the corresponding initial emotional features based on the speech frequency parameters and the speech amplitude parameters.

[0053] It should be noted that the initial emotional features can be based on the speech signal and are relevant feature representations used to infer emotions (such as excitement, sadness, calmness).

[0054] Step S24: Fuse the initial emotional features and the current physiological features to obtain the current fused features that characterize the user's emotional state.

[0055] In practical use, smart headphones preprocess the acquired current speech signal, extract its envelope using the Hilbert transform, and then perform a Fourier transform on the envelope to obtain the envelope spectrum. Next, spectral analysis is performed on the envelope spectrum to extract the main peak frequency and maximum amplitude, which are used as the speech frequency parameter and speech amplitude parameter, respectively. Then, the speech frequency parameter and the aforementioned speech amplitude parameter are mapped to their corresponding emotional dimension values, combining them to form initial emotional features. Finally, the initial emotional features are fused with the current physiological features, for example, by concatenating the two feature vectors end-to-end to form a current fused feature that includes all feature dimensions.

[0056] In this embodiment, the smart earphone first extracts heart rate parameters and heart rate variability parameters from the current heart rate signal to determine the user's current physiological characteristics, and then extracts the envelope spectrum corresponding to the current speech signal to obtain speech frequency parameters and speech amplitude parameters. Next, it determines the corresponding initial emotional features based on the speech frequency parameters and speech amplitude parameters. Finally, it fuses the initial emotional features and the current physiological characteristics to obtain the current fused features representing the emotional state. Since heart rate variability parameters can reflect autonomic nervous activity and thus represent stress and relaxation levels, while envelope spectrum analysis can capture rhythmic and energy changes in speech that are associated with emotions, the fusion of the two results in richer feature dimensions. This avoids the limitations of single-modal features in emotion recognition, achieving a more accurate representation of the user's emotional state and improving the accuracy of subsequent emotion recognition and music recommendation.

[0057] Furthermore, considering that exercise affects heart rate, in this embodiment, the electronic device is also equipped with an inertial acquisition component. The step of fusing the initial emotional features and the current physiological features to obtain the current fused features characterizing the user's emotional state further includes: Step S241: Acquire the user's acceleration and angular velocity data through the inertial acquisition component.

[0058] It should be noted that the inertial acquisition component can be a three-axis motion sensing component used to measure the motion state of the smart earphones themselves, such as a three-axis accelerometer and a three-axis gyroscope. The accelerometer is used to measure the linear acceleration of the smart earphones in three-dimensional space, and the gyroscope is used to measure the angular velocity of the smart earphones rotating about the three coordinate axes.

[0059] Understandably, the acceleration data can be output from the accelerometer in the inertial acquisition unit, reflecting the time-series values ​​of the linear acceleration changes of the smart headphones along each axis in three-dimensional space. The angular velocity data can be output from the gyroscope in the inertial acquisition unit, reflecting the time-series values ​​of the rotational angular velocity of the smart headphones around each axis in three-dimensional space.

[0060] In practical use, refer to Figure 2 , Figure 2 This is an overall flowchart of music recommendation via headphones provided in Embodiment 1 of this application. When the user wears the headphones, the headphone's emotion perception function includes an inertial sensing component in addition to the aforementioned heart rate and voice acquisition components. Specifically, the smart headphones use a built-in three-axis accelerometer to acquire linear acceleration values ​​along the front-back, left-right, and up-down axes at a preset sampling frequency, serving as acceleration data; simultaneously, a built-in three-axis gyroscope acquires the angular velocity values ​​of the headphones around the three axes at the same sampling frequency, serving as angular velocity data.

[0061] Step S242: Determine the user's motion posture based on the angular velocity data, and determine the user's motion gait based on the acceleration data.

[0062] It should be understood that motion posture can be the spatial orientation and rotation state of the smart headphones or the user's head calculated based on angular velocity data, including changes in pitch, yaw, and roll angles, used to determine whether the head has rotated and the direction and magnitude of the rotation. Gait can be the user's body movement pattern obtained from acceleration data analysis, including changes in the user's movement speed and movement type, such as standing still, walking, jogging, or climbing stairs.

[0063] Specifically, after acquiring angular velocity data, smart headphones need to perform data processing and feature fusion on the aforementioned data. At this stage, time integration can be performed on the angular velocity data to calculate the sequence of attitude angle changes of the headphones in space, and the user's current motion posture can be determined based on the amplitude and rate of angle change. Then, after acquiring acceleration data, filtering and gravity component separation processes are performed on the acceleration data to extract the linear acceleration component generated by the user's body movement. The user's current gait is then determined by performing a second integral on the linear acceleration or by combining it with a gait period detection algorithm.

[0064] Step S243: Determine the user's current motion characteristics based on the motion posture and the motion gait.

[0065] It should also be understood that the current motion features can be a vectorized representation of the user's current physical activity and movement state, formed by combining motion posture and gait. For example, a feature vector containing two dimensions: head rotation amplitude and walking speed.

[0066] Step S244: The initial emotional features, the current physiological features, and the current motion features are concatenated to obtain the current fusion features that represent the user's emotional state.

[0067] In practical use, smart headphones can extract and combine key quantitative indicators from movement posture and gait to form current movement features. Then, they acquire previously determined initial emotional features and current physiological features, and combine these features in the same feature space according to the feature concatenation order to form a higher-dimensional fused feature that integrates voice emotion information, physiological autonomic neural activity information, and body movement context information for subsequent emotion recognition.

[0068] In this embodiment, since movement posture and gait can reflect the user's head movement pattern and behavioral context such as walking or standing still, they provide a basis for motion interference discrimination and state calibration for physiological features and initial emotional features. As a result, the fused features contain motion context information, which can effectively distinguish between physiological fluctuations caused by movement and real emotional changes, thus improving the robustness of user emotional state recognition.

[0069] This application provides a music recommendation method. First, a heart rate acquisition component and a voice acquisition component collect the user's current heart rate signal and current voice signal, respectively. Then, based on the current heart rate signal and current voice signal, a current fusion feature representing the user's emotional state is determined. The emotion recognition result is then identified based on the current fusion feature to match corresponding target music and recommend it to the user. Compared to existing methods that rely on historical listening records for music recommendation, this application can perceive the user's emotional state through multiple built-in sensors, thereby selecting and playing music most suitable for the current emotional state from local or cloud music libraries, achieving the goal of proactively regulating the user's emotions and improving the user experience.

[0070] Based on the first embodiment of this application, in the second embodiment of this application, the same or similar content as the above embodiment can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 3 , Figure 3 This is a schematic diagram illustrating the model training process provided in Embodiment 2 of the music recommendation method of this application. To obtain the aforementioned emotion recognition results, such as... Figure 3 As shown, in this embodiment, before the step of acquiring the user's current heart rate signal through the heart rate acquisition component, the following steps are included: Step S01: Obtain the user's historical heart rate signal, historical voice signal, and corresponding first historical emotion result.

[0071] It should be noted that the historical heart rate signal can be the raw photoelectrovascular volumetric signal data related to the user's heart rate at a certain time in the past, which is collected and stored by the headphones. This signal is the same type of physiological signal as the current heart rate signal mentioned above, and the only difference is the time of collection.

[0072] Understandably, historical voice signals can be digital audio signal data related to user voice at a specific historical moment, collected and stored by the headphones during a certain period of time in the past. This signal belongs to the same type of voice signal as the current voice signal, with the only difference being the time of collection.

[0073] It should be understood that the first historical emotion result can be known emotion category labeled data used as training labels, which corresponds in time to the aforementioned historical heart rate signal and the aforementioned historical speech signal. This emotion category labeled data can be obtained through manual labeling.

[0074] Step S02: Determine the first physiological feature corresponding to the historical heart rate signal and the first emotional feature corresponding to the historical voice signal, and fuse the first physiological feature and the first emotional feature to obtain the first fused feature.

[0075] It should be noted that the first physiological characteristic and the aforementioned current physiological characteristic belong to the same type of characteristic, differing only in time. Similarly, the first emotional characteristic and the aforementioned initial emotional characteristic belong to the same type of characteristic, differing only in time. Likewise, the first fusion characteristic and the aforementioned current fusion characteristic belong to the same type of characteristic, differing only in time.

[0076] Step S03: Train the model based on the first fusion feature and the first historical emotion result to obtain a preset emotion recognition model.

[0077] The step of performing emotion recognition on the user based on the current fusion features to obtain the emotion recognition result includes: Step S31': Perform emotion recognition on the current fused features using the preset emotion recognition model to obtain the emotion recognition result.

[0078] Understandably, the preset emotion recognition model can be a machine learning model, such as a vector machine, random forest, lightweight neural network, or deep neural network, which is obtained by the headphones through an offline or online training process and is used to map the fused features of the input to the emotion category output.

[0079] In practical use, the current fused features are input into a preset emotion recognition model, which can output emotion labels of one or more dimensions (such as happy, sad, calm, excited, etc.). It should be noted that this preset emotion recognition model can run on the embedded processor of the headset, or on a mobile phone or in the cloud; this embodiment does not impose any limitations on this. Since the preset emotion recognition model is trained based on the user's own historical physiological characteristics, historical emotional characteristics, and corresponding historical emotional results, the model learns the personalized emotion mapping relationship corresponding to the user under specific physiological and vocal expression patterns. Through this personalized training method, a more accurate recognition of the user's current emotional state is achieved, improving the matching degree between the emotion recognition results and the user's actual feelings.

[0080] Furthermore, in order to output recognition results that better conform to the continuous change pattern of emotions, in this embodiment, the step of performing emotion recognition on the user based on the current fusion features to obtain the emotion recognition result includes: Step S31: Obtain at least two second fusion features of the user within a preset time period and the second historical sentiment result corresponding to the second fusion features.

[0081] It should be noted that the preset time period can be a historical data time window selected during temporal sentiment analysis, such as the past 30 seconds, the past 1 minute, or the past 5 minutes. The second fusion feature can be multiple historical fusion features collected and stored by the headphones within the preset time period. Each second fusion feature has the same feature dimension as the current fusion feature, the only difference being that its collection time is earlier than the current time. The second historical sentiment result can be the historical sentiment recognition result that corresponds one-to-one with each of the above second fusion features in time.

[0082] Step S32: Based on each of the second fusion features and each of the second historical sentiment results, construct the corresponding temporal feature sequence.

[0083] Understandably, a temporal feature sequence can be an ordered data set formed by arranging all the second fusion features and their corresponding second historical sentiment results within a preset time period in chronological order, used to reflect the trajectory of changes in user emotional state and fusion features over time.

[0084] In practical use, when performing emotion recognition, smart headphones, in addition to relying on the fusion features of the current moment for single-point judgment, can also incorporate historical information from the time dimension for comprehensive analysis of temporal evolution. Specifically, at least two second fusion features within a preset time period, as well as the second historical emotion result corresponding to each second fusion feature, can be read from the internally stored historical data cache. Then, the acquired at least two second fusion features and the second historical emotion result are arranged in chronological order of acquisition time to construct a temporal feature sequence. Each element of this sequence contains a fusion feature vector at a certain historical moment and the corresponding emotion label.

[0085] Step S33: Perform a temporal evolution analysis of the user's emotional state based on the temporal feature sequence and the current fusion feature to obtain an emotional transition probability distribution.

[0086] It should be understood that the emotion transfer probability distribution can be calculated by the headphones based on the temporal feature sequence and the current fused features. It is used to describe the probability distribution of the user's emotion transferring from a historical state to various possible emotional states at the current moment. Each probability value in the emotion transfer probability distribution corresponds to the probability of an emotion category appearing at the current moment.

[0087] Step S34: Take the emotional state with the highest probability in the emotional transfer probability distribution as the emotion recognition result.

[0088] Specifically, after constructing the temporal feature sequence, the current fused features can be used as the observation data at the current moment. Combined with the emotional state change trend reflected in the temporal feature sequence, a pre-defined temporal model (such as a Hidden Markov Model, Recurrent Neural Network, or a statistical model based on the state transition matrix) is used to extrapolate the temporal evolution of emotional states. The conditional probability of transitioning from each historical emotional state to each emotional category at the current moment is calculated, thus obtaining the emotional transition probability distribution. This distribution contains the transition probability values ​​corresponding to all pre-defined emotional categories at the current moment. Finally, the emotional category with the highest probability value is selected from the emotional transition probability distribution, and this category is output as the final emotion recognition result. This result integrates information from both the historical emotional evolution trend and the current fused features.

[0089] In this embodiment, by introducing a temporal evolution mechanism for emotional states over time, the state transition relationship between historical emotional results and current fused features is used to predict the most likely current emotional state. This avoids the instantaneous fluctuation error introduced by relying solely on single-frame features at the current moment for isolated judgment, making the emotion recognition results follow the continuity and smoothness of natural emotional changes, and achieving a more consistent and stable recognition of the user's emotional state.

[0090] Based on the first and / or second embodiments of this application, in the third embodiment of this application, the content that is the same as or similar to that in embodiments one and two above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 4 , Figure 4 This is a schematic diagram of the music matching process provided in Embodiment 3 of the music recommendation method of this application. To obtain the aforementioned target music, such as... Figure 4 As shown, in this embodiment, the step of matching the target music based on the emotion recognition result and recommending the target music to the user for playback includes: Step S41: Match the emotion recognition results in the preset emotion list to obtain the target emotion tag, and determine the positive or negative emotion attributes of the target emotion tag.

[0091] It should be noted that the preset emotion list can be a structured data set pre-stored inside the headphones, containing various preset emotion tags and their attribute information. Each emotion tag in the preset emotion list corresponds to a recognizable user emotion category, such as happy, sad, calm, excited, angry, etc.

[0092] For example, regarding physiological characteristics, heart rate can be categorized into high, medium, and low levels, and heart rate variability can be categorized into high and low levels, resulting in six possible combinations of these two parameters. For emotional characteristics, initial emotional states of anger, excitement, sadness, and calmness can be defined through combinations of frequency and amplitude. For motor characteristics, information such as rest, walking direction, walking speed, and type of movement can be derived from combinations of posture and gait, and written into the model's traversal table as a preset emotion list. During matching, the preset emotion list is compared to find the most suitable or closest target emotion label.

[0093] Understandably, the target emotion label can be the specific emotion category name that the headphones find in a preset emotion list and that matches the emotion recognition result. Positive and negative emotion attributes can be polarity markers carried by the target emotion label, used to distinguish whether the emotion belongs to a positive or negative category.

[0094] Step S42: Match the positive and negative emotion attributes and the target emotion tag in a preset music library to obtain at least two target music tracks. The preset music library includes a variety of music tracks with various emotion tags.

[0095] It should be understood that the preset music library can refer to a digital music collection that is stored inside the headphones or accessible via the network, containing multiple musical works and their corresponding mood tag indexes. Each piece of music in the music library is pre-labeled with a corresponding mood tag. For example, a piece of music with a brisk rhythm and a major key can be labeled as "happy", and a piece of music with a slow rhythm and a minor key can be labeled as "calm".

[0096] Specifically, based on positive and negative emotional attributes and target emotional labels, different music recommendation strategies can be employed to match corresponding target music. For example, when a user is identified as "happy," equally cheerful and upbeat music can be played to reinforce the emotion. When a user is identified as "stressed" or "sad," soothing and calming music can be proactively played to help the user relax and alleviate negative emotions, thus regulating their mood. Furthermore, the aforementioned acceleration and angular velocity data can be combined; for instance, if fatigue is detected during exercise, more motivating music can be played for context-adaptive mood regulation.

[0097] Step S43: Generate a playlist to be recommended based on each of the target music tracks, and recommend the playlist to the user for playback.

[0098] It should also be understood that the recommended playlist can be an ordered music playlist generated by the headphones based on at least two matched target music tracks, with the music in the playlist arranged according to a preset sorting rule.

[0099] In practical use, such as Figure 2As shown, after obtaining the emotion recognition result, the result can be used as a query key to match and search within a preset emotion list. The emotion tag matching the result is taken as the target emotion tag, and the corresponding positive and negative emotion attributes are read from the preset emotion list. Next, using the target emotion tag and positive / negative emotion attributes as matching conditions, a music search is performed in a preset music library. Music with tags matching the target emotion tag or with a similarity exceeding a preset threshold is selected as candidate target music. Simultaneously, positive and negative emotion attributes are considered during matching to filter candidates, ensuring that the emotional direction of the selected music matches the identified user's emotional direction. Then, based on the matching results, at least two target music tracks are selected from the preset music library. These tracks are then arranged and combined in an orderly manner according to their emotion matching score (high to low) or musical style to obtain a playlist to be recommended. Finally, this playlist is recommended to the user for playback. The recommendation method can be automatic playback or a recommendation prompt generated on the headphones and announced to the user via voice, with playback starting after user confirmation.

[0100] This embodiment first maps the emotion recognition results into target emotion tags in a unified format by using a preset emotion list and determines their positive and negative attributes. Then, based on the positive and negative attributes and the target emotion tags, it performs a two-dimensional matching in a music library with preset emotion tags. This makes the music selection consider both the consistency of emotion type and the clarity of emotion regulation direction, achieving a dual adaptation of recommended content and the user's current emotional state in terms of type and regulation direction.

[0101] Furthermore, to further optimize the playlist, in this embodiment, after the step of generating a playlist to be recommended based on each of the target music tracks and recommending the playlist to the user for playback, the method further includes: Step S51: Obtain the playback completion rate of each target song in the playlist to be recommended during playback.

[0102] It should be noted that the playback completion rate can be the percentage of time that each target song in the recommended playlist is actually played by the user out of the total duration of that target song. For example, if a song with a total duration of 4 minutes is played for 3 minutes, its playback completion rate is 75%. The preset threshold can be a critical value for determining whether the music recommendation is accurate. This threshold can be a fixed value or dynamically adjusted according to the user's overall listening habits, for example, set to 80% as the judgment standard.

[0103] Step S52: If the playback completion rate is lower than a preset threshold, the target music corresponding to the lower than the preset threshold is taken as music to be optimized.

[0104] Understandably, music to be optimized can be target music in the recommended playlist with a completion rate below a preset threshold. This type of music is considered to have failed to effectively match the user's current emotional state and needs to be tagged for optimization.

[0105] Step S53: Extract the current emotion tag corresponding to the music to be optimized, and refine and adjust the current emotion tag of the music to be optimized to obtain emotion sub-tags.

[0106] It should be understood that the current emotion tag can be the name of the emotion category currently labeled in the preset music library for the music to be optimized. The emotion sub-tag can be a more granular description of the emotion category used to replace or supplement the original emotion tag. This sub-tag has a higher degree of matching with the user's actual listening reaction and can improve the accuracy of music matching in subsequent recommendations. For example, "exhilarating" can be refined into subcategories such as "high-intensity exhilaration" and "medium-intensity exhilaration".

[0107] Step S54: Update the preset emotion list according to the emotion sub-tags.

[0108] Step S55: Based on the updated preset emotion list, return to the step of collecting the user's current heart rate signal through the heart rate acquisition component to obtain the updated playlist to be recommended, and recommend the updated playlist to be recommended to the user for playback.

[0109] For example, during user interaction, the accuracy of music recommendations can be assessed by observing the completion rate or song skipping behavior of recommended music. For instance, if the completion rate of music for a particular emotion tag is below 80%, the emotion tag definition can be deemed inaccurate. In this case, data analysis across three dimensions—heart rate changes, voice recognition, and movement status—can be used to refine and add more emotion tags to the preset emotion list, while inaccurate emotion tags are removed to achieve more precise matching. This process is continuously updated and optimized.

[0110] In practical use, such as Figure 2As shown, after the recommended playlist is completed, feedback data can be continuously collected during its playback. For example, the actual playback duration of each target song can be monitored and recorded, and the playback completion rate of each song can be calculated by dividing its playback duration by its total duration. When the playback completion rate of a target song is lower than a preset threshold, the target song is marked as a song to be optimized, indicating that the recommendation effect of the song under the current emotion recognition result is not ideal. Then, the current emotion tag corresponding to the song to be optimized is read from the preset music library and refined. Based on the changing trends of physiological signals, voice signals, and motion signals collected simultaneously by the user during the playback of the song, it is determined whether the current emotion tag is too general, and the tag is split into two or more more specific emotion sub-tags. After the refinement is completed, the newly generated emotion sub-tags are written into the preset emotion list for update and optimization. After the update is completed, based on the updated preset emotion list, a new round of complete processing is started to obtain the updated recommended playlist, which is then recommended to the user for playback.

[0111] In this embodiment, since the playback completion rate objectively reflects the user's acceptance of recommended music, the emotion tags corresponding to music with a playback completion rate below a threshold are further refined and segmented to generate more granular emotion sub-tags. These sub-tags are then used to update the preset emotion list, enabling personalized learning and allowing for the differentiation of more granular emotional states during subsequent recognition. This achieves adaptive learning based on the user's personalized emotional preferences, improving the accuracy of emotion recognition and music recommendation over long-term use.

[0112] It should be noted that the above examples are for understanding this application only and do not constitute a limitation on the music recommendation method of this application. Any simple modifications based on this technical concept are within the scope of protection of this application. Furthermore, all actions involving the acquisition of signals, information, or data in this application are performed in accordance with the relevant data protection regulations of the country where the application is located and with authorization from the owner of the corresponding device.

[0113] This application also provides a music recommendation device; please refer to... Figure 5 , Figure 5 This is a block diagram of the module structure of a music recommendation device according to an embodiment of this application; in this embodiment, the music recommendation device includes: The acquisition module 101 is used to acquire the user's current heart rate signal through the heart rate acquisition component and to acquire the user's current voice signal through the voice acquisition component; The fusion module 102 is used to determine the current fusion feature representing the user's emotional state based on the current heart rate signal and the current voice signal; The recognition module 103 is used to perform emotion recognition on the user based on the current fusion features to obtain an emotion recognition result; The recommendation module 104 is used to match the corresponding target music based on the emotion recognition result and recommend the target music to the user for playback.

[0114] This embodiment first collects the user's current heart rate signal and current voice signal using a heart rate acquisition component and a voice acquisition component, respectively. Then, it determines the current fusion feature representing the user's emotional state based on the current heart rate signal and current voice signal, and identifies the emotion recognition result based on the current fusion feature to match corresponding target music for recommendation to the user. Compared with existing methods that rely on historical listening records to recommend music, this embodiment can perceive the user's emotional state through heart rate and voice signals, thereby matching and playing appropriate target music. This reduces recommendation bias caused by the lag in historical recommended playlists, which makes it difficult to adapt to changes in user emotions, and improves the effectiveness of proactively regulating user emotions.

[0115] In one implementation, the fusion module 102 is further configured to extract heart rate parameters and heart rate variability parameters from the current heart rate signal, and determine the user's current physiological characteristics based on the heart rate parameters and heart rate variability parameters; extract the envelope spectrum corresponding to the current speech signal, and analyze the envelope spectrum to obtain speech frequency parameters and speech amplitude parameters; determine the corresponding initial emotional features based on the speech frequency parameters and speech amplitude parameters; and fuse the initial emotional features and the current physiological characteristics to obtain a current fused feature representing the user's emotional state.

[0116] In one implementation, the fusion module 102 is further configured to acquire the user's acceleration data and angular velocity data through the inertial acquisition component; determine the user's motion posture based on the angular velocity data, and determine the user's motion gait based on the acceleration data; determine the user's current motion characteristics based on the motion posture and the motion gait; and concatenate the initial emotional characteristics, the current physiological characteristics, and the current motion characteristics to obtain the current fusion characteristics representing the user's emotional state.

[0117] In one implementation, the recommendation module 104 is further configured to match the emotion recognition results in a preset emotion list to obtain a target emotion tag and determine the positive or negative emotion attributes of the target emotion tag; match the positive or negative emotion attributes and the target emotion tag in a preset music library to obtain at least two target music tracks, the preset music library including various types of music with different emotion tags; generate a playlist to be recommended based on each of the target music tracks, and recommend the playlist to the user for playback.

[0118] In one implementation, the recommendation module 104 is further configured to: obtain the playback completion rate of each target music in the playlist to be recommended during playback; if the playback completion rate is lower than a preset threshold, designate the target music with a completion rate lower than the preset threshold as music to be optimized; extract the current emotion tag corresponding to the music to be optimized, and refine and adjust the current emotion tag of the music to be optimized to obtain an emotion sub-tag; update the preset emotion list according to the emotion sub-tag; based on the updated preset emotion list, return to the step of collecting the user's current heart rate signal through the heart rate acquisition component to obtain the updated playlist to be recommended, and recommend the updated playlist to the user for playback.

[0119] In one implementation, the recognition module 103 is further configured to acquire the user's historical heart rate signal, historical voice signal, and corresponding first historical emotion result; determine the first physiological feature corresponding to the historical heart rate signal and the first emotional feature corresponding to the historical voice signal, and fuse the first physiological feature and the first emotional feature to obtain a first fused feature; and perform model training based on the first fused feature and the first historical emotion result to obtain a preset emotion recognition model. The recognition module 103 is also used to perform emotion recognition on the current fused features through the preset emotion recognition model to obtain the emotion recognition result.

[0120] In one implementation, the recognition module 103 is further configured to acquire at least two second fusion features of the user within a preset time period and the second historical emotion results corresponding to the second fusion features; construct a corresponding temporal feature sequence based on each second fusion feature and each second historical emotion result; perform temporal evolution of the user's emotional state according to the temporal feature sequence and the current fusion feature to obtain an emotion transition probability distribution; and take the emotional state with the highest probability in the emotion transition probability distribution as the emotion recognition result.

[0121] Other embodiments or specific implementations of the music recommendation device of this application can be found in the above-described method embodiments, and will not be repeated here.

[0122] The music recommendation device provided in this application, employing the music recommendation method described in the above embodiments, can solve the technical problem that existing music recommendations rely on historical listening records and are difficult to adapt to the user's emotional state in real time. Compared with the prior art, the beneficial effects of the music recommendation device provided in this application are the same as those of the music recommendation method provided in the above embodiments, and other technical features in the music recommendation device are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0123] This application provides an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the music recommendation methods described in the above embodiments.

[0124] The following is for reference. Figure 6 , Figure 6 This is a schematic diagram of the hardware operating environment of the electronic device involved in the embodiments of this application, illustrating a structural schematic diagram of an electronic device suitable for implementing the embodiments of this application. The electronic device in the embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, smart headphones, and headsets. Figure 6 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0125] like Figure 6 As shown, the electronic device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory 1002 or a program loaded from a storage device 1003 into a random access memory 1004. The random access memory 1004 also stores various programs and data required for the operation of the electronic device. The processing unit 1001, the read-only memory 1002, and the random access memory 1004 are interconnected via a bus 1005. An input / output interface 1006 is also connected to the bus. Typically, the following systems can be connected to the input / output interface 1006: an input device 1007 including, for example, a touchscreen, touchpad, image sensor, microphone, accelerometer, gyroscope, etc.; an output device 1008 including, for example, a speaker, vibrator, etc.; a storage device 1003 including, for example, magnetic tape, hard disk, etc.; and a communication device 1009. The communication device 1009 allows the electronic device to communicate wirelessly or wiredly with other devices to exchange data. Although the diagrams show electronic devices with various systems, it should be understood that it is not required to implement or have all of the systems shown. More or fewer systems may be implemented alternatively.

[0126] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from read-only memory 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0127] The electronic device provided in this application, employing the music recommendation method described in the above embodiments, can solve the technical problem that existing music recommendations rely on historical listening records and are difficult to adapt to the user's emotional state in real time. Compared with the prior art, the beneficial effects of the electronic device provided in this application are the same as those of the music recommendation method provided in the above embodiments, and other technical features of this electronic device are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0128] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0129] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0130] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the music recommendation method described in the above embodiments.

[0131] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, radio frequency (RF), etc., or any suitable combination thereof.

[0132] The aforementioned computer-readable storage medium may be included in an electronic device or may exist independently without being assembled into an electronic device.

[0133] The aforementioned computer-readable storage medium carries one or more programs that, when executed by an electronic device, cause the electronic device to: acquire the user's current heart rate signal through the heart rate acquisition component and acquire the user's current voice signal through the voice acquisition component; determine a current fusion feature representing the user's emotional state based on the current heart rate signal and the current voice signal; perform emotion recognition on the user based on the current fusion feature to obtain an emotion recognition result; match a corresponding target music based on the emotion recognition result, and recommend the target music to the user for playback.

[0134] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0135] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation that may be implemented in systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0136] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0137] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described music recommendation method. This solves the technical problem that existing music recommendations rely on historical listening records and struggle to adapt to users' emotional states in real time. Compared to the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the music recommendation method provided in the above embodiments, and will not be repeated here.

[0138] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the music recommendation method described above.

[0139] The computer program product provided in this application can solve the technical problem that existing music recommendation methods rely on historical listening records and are difficult to adapt to the user's emotional state in real time. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the music recommendation method provided in the above embodiments, and will not be repeated here.

[0140] The above description is only a part of the embodiments of this application and does not limit the scope of protection of this application. All equivalent structural transformations made under the technical concept of this application and using the content of this application specification and drawings, or direct / indirect applications in other related technical fields, are included in the scope of protection of this application.

Claims

1. A music recommendation method, characterized in that, The music recommendation method is applied to an electronic device equipped with a heart rate acquisition component and a voice acquisition component, and the method includes: The user's current heart rate signal is acquired through the heart rate acquisition component, and the user's current voice signal is acquired through the voice acquisition component. Based on the current heart rate signal and the current voice signal, determine the current fusion feature representing the user's emotional state; Based on the current fusion features, the user's emotion is identified to obtain the emotion identification result; Based on the emotion recognition results, a corresponding target music is matched, and the target music is recommended to the user for playback.

2. The method as described in claim 1, characterized in that, The step of determining the current fusion feature representing the user's emotional state based on the current heart rate signal and the current voice signal includes: Heart rate parameters and heart rate variability parameters are extracted from the current heart rate signal, and the user's current physiological characteristics are determined based on the heart rate parameters and the heart rate variability parameters. Extract the envelope spectrum corresponding to the current speech signal, and analyze the envelope spectrum to obtain speech frequency parameters and speech amplitude parameters; Based on the speech frequency parameters and the speech amplitude parameters, the corresponding initial emotional features are determined; The initial emotional features and the current physiological features are fused to obtain the current fused features that characterize the user's emotional state.

3. The method as described in claim 2, characterized in that, The electronic device is further equipped with an inertial acquisition component. The step of fusing the initial emotional features and the current physiological features to obtain the current fused features characterizing the user's emotional state further includes: The inertial acquisition unit acquires the user's acceleration and angular velocity data. The user's motion posture is determined based on the angular velocity data, and the user's motion gait is determined based on the acceleration data; Based on the motion posture and the motion gait, the user's current motion characteristics are determined; The initial emotional features, the current physiological features, and the current motion features are concatenated to obtain the current fusion features that characterize the user's emotional state.

4. The method as described in claim 1, characterized in that, The step of matching the target music based on the emotion recognition result and recommending the target music to the user for playback includes: The emotion recognition results are matched against a preset emotion list to obtain a target emotion tag, and the positive or negative emotion attributes of the target emotion tag are determined. Matching the positive and negative emotion attributes and the target emotion tags in a preset music library yields at least two target music tracks. The preset music library includes a variety of music tracks with various emotion tags. A playlist to be recommended is generated based on the target music, and the playlist is recommended to the user for playback.

5. The method as described in claim 4, characterized in that, After the step of generating a playlist to be recommended based on each of the target music tracks and recommending the playlist to the user for playback, the method further includes: Obtain the playback completion rate of each target song in the playlist to be recommended during playback; If the playback completion rate is lower than a preset threshold, the target music corresponding to the lower than the preset threshold will be regarded as music to be optimized. Extract the current emotion tag corresponding to the music to be optimized, and refine and adjust the current emotion tag of the music to be optimized to obtain emotion sub-tags; The preset emotion list is updated based on the emotion sub-tags; Based on the updated preset emotion list, the process returns to the step of collecting the user's current heart rate signal through the heart rate acquisition component, obtains the updated playlist to be recommended, and recommends the updated playlist to the user for playback.

6. The method as described in claim 1, characterized in that, Before the step of acquiring the user's current heart rate signal through the heart rate acquisition component, the following steps are included: Acquire the user's historical heart rate signal, historical voice signal, and corresponding first historical emotion result; Determine the first physiological feature corresponding to the historical heart rate signal and the first emotional feature corresponding to the historical voice signal, and fuse the first physiological feature and the first emotional feature to obtain the first fused feature; The model is trained based on the first fusion feature and the first historical emotion result to obtain a preset emotion recognition model; The step of performing emotion recognition on the user based on the current fusion features to obtain the emotion recognition result includes: The current fused features are used to perform emotion recognition using the preset emotion recognition model to obtain the emotion recognition result.

7. The method according to any one of claims 1 to 6, characterized in that, The step of performing emotion recognition on the user based on the current fusion features to obtain the emotion recognition result includes: Obtain at least two second fusion features of the user within a preset time period, and the second historical sentiment result corresponding to the second fusion features; Based on each of the second fusion features and each of the second historical sentiment results, a corresponding temporal feature sequence is constructed; Based on the temporal feature sequence and the current fused features, the user's emotional state is subjected to temporal evolution to obtain an emotional transition probability distribution; The emotional state with the highest probability in the emotional transfer probability distribution is taken as the emotion recognition result.

8. A music recommendation device, characterized in that, The device includes: The acquisition module is used to acquire the user's current heart rate signal through the heart rate acquisition component and to acquire the user's current voice signal through the voice acquisition component; The fusion module is used to determine the current fusion feature representing the user's emotional state based on the current heart rate signal and the current voice signal; The recognition module is used to perform emotion recognition on the user based on the current fused features, and obtain the emotion recognition result; The recommendation module is used to match the corresponding target music based on the emotion recognition result and recommend the target music to the user for playback.

9. An electronic device, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the music recommendation method as described in any one of claims 1 to 7.

10. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the music recommendation method as described in any one of claims 1 to 7.