Music recommendation method and device, music emotion determination method and device, equipment and medium

Abstract

The invention relates to the technical field of smart televisions, in particular to a music recommendation and music emotion determination method and device, equipment and a medium, and the method comprises the steps: analyzing behavior characteristics, constructing a vector representation of a current emotion, and matching a music file corresponding to a user emotion vector in a preset resource library; the preset resource library comprises a mapping relation between the music files and the music emotion characteristics corresponding to the music files, and the music files are sequentially displayed on a display interface of the television equipment. User body data is collected in real time through television equipment, behavior features are extracted and analyzed to obtain emotion features and current emotions, user emotion vectors are constructed, music files are matched in a preset resource library containing the mapping relation between the music files and the music emotion features according to the vectors, and a music recommendation list is generated according to the sequence of the current emotions. And finally, sequentially displaying on a television display interface. Therefore, the body data collected by the television equipment is analyzed to determine the emotion of the user, so as to accurately match the music file.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent television, and in particular to a music recommendation and music emotion determination method, device, equipment and medium. BACKGROUND

[0002] With the popularity of intelligent television and streaming media services, users' demand for personalized music recommendation is increasing. Traditional music recommendation systems are mainly based on users' historical behavior and preferences, lacking the ability to perceive users' real-time emotional state, resulting in a mismatch between recommended content and users' current emotional needs.

[0003] Currently, most music recommendation systems analyze the music that users have played in the past, and recommend similar music to users based on information such as music type, singer, and style. This method only focuses on users' past music preferences, ignoring users' state and emotional changes. Users' music preferences change with time, scene, and emotion, and relying solely on historical data cannot meet users' diverse needs at different times, making it difficult to provide accurate services to users.

[0004] Therefore, how to analyze the body data collected by the television device to determine the user's emotion, so as to accurately match the music file has become a problem to be solved. SUMMARY

[0005] The embodiments of the present application provide a music recommendation method, device, computer equipment and storage medium to solve the problem of how to analyze the body data collected by the television device to determine the user's emotion, so as to accurately match the music file.

[0006] In a first aspect, the embodiments of the present application provide a music recommendation method applied to a television terminal, the music recommendation method comprising: obtaining body data of a user collected by a television device in real time, and extracting behavior features of the body data; analyzing the behavior features to obtain emotion features, obtaining the current emotion of the user according to the emotion features, constructing a vector representation of the current emotion, and obtaining a user emotion vector; matching a music file corresponding to the user emotion vector in a preset resource library according to the user emotion vector, the preset resource library comprising a mapping relationship between music files and music emotion features corresponding to the music files; sorting the music files according to the current emotion to obtain a music recommendation list, and sequentially displaying the music files on a display interface of the television device according to the music recommendation list.

[0007] In a second aspect, the embodiments of the present application provide a music emotion determination method, comprising: Before the user's body data collected by the television device in real time is acquired in the music recommendation method, a music file to be analyzed is acquired, and element features in the music file are extracted; The element features are spliced to obtain a feature vector, and music emotion features of the music file are determined according to the feature vector; The music file and the music emotion features corresponding to the music file are stored in a preset resource library.

[0008] In a third aspect, an embodiment of the present application provides a music recommendation device, and the music recommendation method is applied to a television terminal, and the music recommendation device comprises: A feature extraction module is configured to acquire body data of a user collected by a television device in real time, and extract behavior features of the body data; A vector construction module is configured to analyze the behavior features to obtain emotion features, obtain a current emotion of the user according to the emotion features, construct a vector representation of the current emotion, and obtain a user emotion vector; A resource matching module is configured to match, according to the user emotion vector, a music file corresponding to the user emotion vector in a preset resource library, and the preset resource library comprises a mapping relationship between music files and music emotion features corresponding to the music files; A music file recommendation module is configured to sort the music files according to the current emotion to obtain a music recommendation list, and sequentially display the music files on a display interface of the television device according to the music recommendation list.

[0009] In a fourth aspect, an embodiment of the present application provides a music emotion determination device, comprising: An element feature extraction module is configured to acquire, before the user's body data collected by the television device in real time is acquired in the music recommendation method, a music file to be analyzed, and extract element features in the music file; A feature determination module is configured to splice the element features to obtain a feature vector, and determine music emotion features of the music file according to the feature vector; A storage module is configured to store the music file and the music emotion features corresponding to the music file in a preset resource library.

[0010] In a fifth aspect, an embodiment of the present application provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, and the processor implements the music recommendation method or the music emotion determination method when the computer program is executed.

[0011] In a sixth aspect, an embodiment of the present application provides a computer readable storage medium, which stores a computer program. The computer program is executed by a processor to implement the music recommendation method or the music emotion determination method.

[0012] Compared with the prior art, the present application has the beneficial effects that: the user's body data collected in real time by the television device is obtained, the behavior features of the body data are extracted, the emotion features are obtained by analyzing the behavior features, the current emotion of the user is obtained according to the emotion features, the vector representation of the current emotion is constructed, the user emotion vector is obtained, the music files corresponding to the user emotion vector are matched in a preset resource library according to the user emotion vector, the preset resource library includes the mapping relationship between the music files and the music emotion features of the corresponding music files, the music files are sorted according to the current emotion, and a music recommendation list is obtained, and the music files are sequentially displayed on the display interface of the television device according to the music recommendation list. The user's body data is collected in real time by the television device, the behavior features are extracted and analyzed to obtain the emotion features and the current emotion, the user emotion vector is constructed, the music files are matched in the preset resource library containing the mapping relationship between the music files and the music emotion features according to the vector, the music recommendation list is generated by sorting according to the current emotion, and finally the music files are sequentially displayed on the television display interface. Thus, the user's emotion is determined by analyzing the body data collected by the television device, so that the music files are accurately matched. BRIEF DESCRIPTION OF DRAWINGS

[0013] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the description of the embodiments of the present application. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0014] Figure 1 is an application environment schematic diagram of a music recommendation method provided by an embodiment of the present application; Figure 2 is a flowchart of a music recommendation method provided by an embodiment of the present application; Figure 3 is a flowchart of a music recommendation method provided by an embodiment of the present application; Figure 4 is a flowchart of a music recommendation method provided by an embodiment of the present application; Figure 5 is a flowchart of a music emotion determination method provided by an embodiment of the present application; Figure 6 is a structural schematic diagram of a music recommendation device provided by an embodiment of the present application; Figure 7 is a structural schematic diagram of a music emotion determination device provided by an embodiment of the present application; Figure 8 is a structural schematic diagram of a computer device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0015] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0016] As shown in Figure 1 , it is an application environment schematic diagram of a music recommendation method provided by an embodiment of the present application, wherein the music recommendation method is applied to a television terminal, wherein a client and a server are connected to communicate, and a user can provide conditions, requirements and operation instructions for music recommendation by operating the client. The client includes but is not limited to various personal computers, notebook computers, smart phones, tablet computers and portable wearable devices and other computer devices. The computer device corresponding to the server can be realized by an independent server or a server cluster composed of multiple servers.

[0017] As shown in Figure 2 , it is a flow schematic diagram of a music recommendation method provided by an embodiment of the present application, which can include the following steps: Step S201, acquiring user's body data collected by a television device in real time, and extracting behavior features of the body data; Among them, the face data is various information of the user's face captured by the built-in camera of the television. For example, the user's facial expression (smile, frown, surprise, etc.), the movement of facial muscles (wink, open mouth, etc.). The camera will continuously record these pictures to provide raw materials for subsequent analysis. The voice data is collected by the microphone of the television. It includes information such as the content, tone, speed, volume of the voice. Thus reflecting the emotional state and intention of the user when speaking. The physiological data of the user can be obtained through the wearable device (such as smart bracelet, smart watch, etc.) connected with the television, reflecting the physiological state of the user's body, which has important reference value for judging the user's emotion.

[0018] The collected facial data is processed to identify the facial actions of the user and calculate the proportion of different facial actions. For example, the number of times a smiling action occurs in a period of time accounts for the proportion of the total number of facial actions. These facial action proportions can reflect the emotional tendencies of the user, such as frequent smiling indicating that the user is in a happy mood.

[0019] The changes in the voice data are analyzed to extract data fluctuation values. For example, by monitoring changes in the volume, tone, and speed of speech, the fluctuations of these parameters over a period of time are calculated. According to the data fluctuation values, the user's volume intensity changes and tone fluctuations can be determined. These features can reflect the emotional state of the user when speaking, such as a sudden increase in volume and an increase in speech speed, which may indicate that the user is in an excited or agitated mood. According to the preset physiological indicators, the changes in the corresponding physiological data are analyzed. For example, the trend of heart rate changes over a period of time is analyzed, whether it is gradually increasing or decreasing, and the magnitude of the change. The change characteristics of physiological data can reflect the stress state of the user's body, thereby inferring the user's emotional state, such as an increased heart rate indicating that the user is in a tense or excited mood.

[0020] In step S202, the behavior characteristics are analyzed to obtain emotional characteristics, the current emotion of the user is obtained according to the emotional characteristics, a vector representation of the current emotion is constructed, and a user emotion vector is obtained.

[0021] Optionally, the body data includes facial data, voice data, and physiological data. According to the facial data, the facial actions of the user are determined, and the proportion of facial actions is calculated to obtain a facial action proportion. The data changes of the voice data are analyzed to obtain data fluctuation values, and the volume intensity changes of the user are determined according to the data fluctuation values. According to the preset physiological indicators, the changes in the physiological data corresponding to the physiological indicators are analyzed to obtain the changes in the indicator data. According to the facial action proportion, the volume intensity changes, and the changes in the indicator data, the current emotion of the user is determined.

[0022] Among them, through image recognition and analysis technology on the collected facial data, the specific facial actions of the user are identified, such as frowning, smiling, blinking, and opening the mouth. In a period of time, the number of times each facial action occurs is counted, and then the proportion of the total number of facial actions is calculated. For example, in 1 minute, the frowning action occurs 10 times, and the total number of facial actions is 50 times, so the frowning action proportion is 10-50=20%. Different facial action proportions can reflect different emotional tendencies, such as a high proportion of smiling actions indicating that the user is in a happy mood.

[0023] The collected voice data is processed to analyze its changes in volume, tone, speed, etc. For example, it is detected whether the voice volume gradually increases or decreases over time, whether the tone is smooth or fluctuates greatly, whether the speed is fast or slow, etc. By quantifying these changes, a data fluctuation value is obtained. For example, the difference between the maximum and minimum volume of the volume in 1 minute is calculated, or the standard deviation of the speed over a period of time is calculated, etc. According to the data fluctuation value, the user's volume intensity change is determined. If the data fluctuation value is large and the overall volume is increasing, it means that the user's volume intensity is increasing, which may reflect that the user's emotion is more excited. Some physiological indicators related to emotion are set in advance, such as heart rate, blood pressure, and respiratory rate. The collected physiological data corresponding to the physiological indicators is analyzed to observe its change trend over time. For example, it is analyzed whether the heart rate is gradually increasing or decreasing, whether the blood pressure has a significant fluctuation, etc. The changes of these physiological data are quantified to obtain the corresponding indicator data change. For example, the average change rate of heart rate in 5 minutes is calculated, etc.

[0024] The information of facial action proportion, volume intensity change, and indicator data change, etc. is comprehensively considered to determine the current emotion of the user. For example, the facial action proportion shows that the smiling action proportion is high, the voice data shows that the volume is moderate and the tone is smooth, and the physiological data shows that the heart rate is normal, so it can be inferred that the user is in a calm and happy emotional state. Frequent frowning facial actions, sudden increase in voice volume and speed, and significant increase in heart rate may indicate that the user is in an angry or excited emotional state.

[0025] The determined current emotion of the user is represented by a vector to obtain a user emotion vector. The user emotion vector is a mathematical representation method that can convert complex emotional information into numerical form that can be processed by a computer. For example, an emotion can be represented by a three-dimensional vector, and the three dimensions correspond to different emotional characteristics, such as happiness, excitement, and tension. Assuming that the user is in a very happy, somewhat excited but not tense state, the corresponding emotion vector may be (0.8, 0.6, 0.2), where each value represents the degree in the corresponding dimension.

[0026] In step S203, according to the user emotion vector, a music file corresponding to the user emotion vector is matched in a preset resource library, and the preset resource library includes a mapping relationship between music files and music emotion characteristics corresponding to the music files.

[0027] The preset resource library is a database storing a large number of music files and music emotion features corresponding to each music file. The mapping relationship is like giving each music an emotional label, such as happy, sad, calm, and excited. The establishment of the resource library is usually completed by manual annotation, music feature analysis algorithm, or a combination of both. For example, a music with a fast rhythm and lively melody will be labeled as a happy emotion feature, and a music with a slow rhythm and low melody will be labeled as a sad emotion feature.

[0028] To achieve matching of the user emotion vector and the music file, the emotion features of the music need to be quantified first, so that they can be compared with the user emotion vector in the same dimension. For example, the emotion features of the music, such as happy, sad, calm, and excited, are converted into vector representation. It can be assumed that a three-dimensional vector (pleasure, activity, and calmness) is used to represent the emotion features of the music, so that a happy music may correspond to a vector of (0.8, 0.7, 0.1), indicating that it has high pleasure and activity, and low calmness. After quantifying the emotion features of the music into vectors, the system calculates the similarity between the user emotion vector and the emotion feature vector corresponding to each music file in the resource library. Common similarity calculation methods include Euclidean distance and cosine similarity.

[0029] The distance between two vectors in space is calculated, and the closer the distance, the higher the similarity. For example, the user emotion vector is (0.7, 0.6, 0.2), and the emotion feature vector of a certain music is (0.8, 0.7, 0.1). The distance between the two is calculated by the Euclidean distance formula. If the distance is small, it means that the emotion features of this music are similar to the current emotion of the user. The cosine value of the angle between the two vectors is used to measure the similarity, and the closer the cosine value to 1, the more similar the two vectors. The system selects the music files with high similarity to the user emotion vector according to the calculated similarity. For example, a similarity value is set, and when the similarity between the emotion feature vector of a music file and the user emotion vector exceeds the threshold value, this music file is considered as a matched music.

[0030] In step S204, the music files are sorted according to the current emotion, and a music recommendation list is obtained. According to the music recommendation list, the music files are displayed in sequence on the display interface of the television device.

[0031] The music files matched are sorted in detail according to the current emotion of the user. For example, if the current emotion of the user is extremely happy, the music with strong happy emotion features should be ranked first, and the music with less happy emotion features should be ranked second. The matching degree can be accurately measured by the quantified similarity value, and the higher the similarity, the higher the ranking.

[0032] In addition to the emotional fit, other factors can be combined to refine the ranking. For example, refer to the popularity and play count of the music and other data. Popular music is often more popular, and its priority can be appropriately increased in the ranking, but it cannot deviate too much from the main line of emotional matching. For example, a song that meets both the happy mood and is very popular may be ranked in a more prominent position. If the system records the user's past music playback history and preference information, for the type of music that the user often plays or collects in the past, a certain weight can be given in this ranking. For example, the user often likes the happy style music of a certain singer, so the songs of the singer that meet the current happy mood can be ranked in a more prominent position. After comprehensive ranking, the system ranks the matched music files in order of priority from high to low to form an ordered music recommendation list. This list is a customized music menu for the user, and each piece of music has been carefully sorted according to the user's current mood and other related factors.

[0033] The system will present the generated music recommendation list on the display interface of the television device. The design of the display interface usually takes into account the user's operation convenience and visual experience, and may display in the form of a list, with each row displaying the basic information of a piece of music, such as the song name, singer name, etc.

[0034] The music files will be displayed in order according to the recommendation list, and the user can intuitively see the recommended music. Some display interfaces of television devices may also provide more interactive functions, such as the user can scroll up and down the list to view more music, click on a piece of music to start playing, or perform operations such as collecting, sharing, etc.

[0035] In the embodiments of the present application, the user's body data collected in real time by the television device is obtained, the behavior features of the body data are extracted, the behavior features are analyzed to obtain the emotional features, the current emotion of the user is obtained according to the emotional features, a vector representation of the current emotion is constructed, a user emotion vector is obtained, music files corresponding to the user emotion vector are matched in a preset resource library according to the user emotion vector, the preset resource library includes a mapping relationship between the music files and the music emotional features corresponding to the music files, the music files are sorted according to the current emotion to obtain a music recommendation list, and the music files are sequentially displayed on the display interface of the television device according to the music recommendation list. By collecting the user's body data in real time by the television device, extracting the behavior features and analyzing the emotional features and the current emotion to construct a user emotion vector, matching the music files in the preset resource library containing the mapping relationship between the music files and the music emotional features according to the vector, generating a music recommendation list by sorting according to the current emotion, and finally displaying sequentially on the television display interface, the user's emotion is determined by analyzing the body data collected by the television device, so as to accurately match the music files.

[0036] As Figure 3 shown, a flowchart of a music recommendation method provided by Embodiment Three of the present application is shown. In step S203, the real-time audio recorded by the recording device is analyzed by the playing device to obtain a real-time audio analysis result, which can include the following steps: Step S301: classifying the current emotion according to the emotional features to obtain an emotional type.

[0037] Step S302: matching the emotional type in a preset category vector library to obtain an emotional vector corresponding to the emotional type.

[0038] In the foregoing analysis process, features such as pitch, timbre, volume change, speech rate, and intonation have been extracted from the real-time audio, which reflect the emotional state of the speaker in the audio. According to a preset classification rule, these emotional features are mapped to different emotional types. The classification rule can be based on psychological research, statistical analysis of a large amount of audio data, or expert experience. The emotional types include but are not limited to joy, sadness, anger, fear, calmness, etc. For example, if it is detected that there are frequent happy intonations, higher pitch, and faster speech rate in the audio, and the volume is moderate, the current emotion can be classified as joy.

[0039] The preset category vector library is a pre-constructed database that stores vector representations corresponding to various emotional types. Each vector is usually a multidimensional array, and different dimensions represent different emotional features or attributes. For example, a three-dimensional vector can represent happiness, activity, and tension, and different emotional types have different values in these three dimensions. According to the emotional type obtained in step S301, the corresponding vector is found in the category vector library. This process is like looking up a specific word in a dictionary and obtaining its corresponding explanation. For example, if the emotional type determined in step S301 is joy, the vector representing the joy emotion is found in the category vector library, and it is assumed that this vector is (0.8, 0.7, 0.1), indicating that the joy emotion has a high degree of happiness, a certain degree of activity, and a low degree of tension.

[0040] In the embodiments of the present application, the emotional type is obtained by classifying the emotional features, and the emotional type is mapped to the emotional vector, which provides a more accurate and operable data basis for subsequent audio processing, music recommendation, and other applications.

[0041] As Figure 4As shown, a flowchart of a music recommendation method provided by Embodiment Four of the present application is shown. After the music files are sequentially displayed on the display interface of the television device according to the music recommendation list in step S204, the following steps can also be included: In step S401, the feedback behavior of the user to the television device is acquired, and the feedback behavior is analyzed to determine the preference level of the user to the music file.

[0042] In step S402, an optimization strategy of the music recommendation list is generated according to the preference level.

[0043] Among them, the direct operation behavior refers to the explicit operation of the user on the television device for the music file, such as clicking to play, pause, skip, collect, share, etc. For example, the user clicks to play a certain music and listens to it completely, or collects a certain music to his own playlist, which are positive feedback behaviors; while quickly skipping a certain music may indicate that it is not interesting. Indirect behavior data refers to some indirect behavior data in addition to direct operation, such as the duration of the user staying on a certain music, the viewing progress when watching a music video, etc. If the user stays on a certain music for a long time, it means that he may have some interest in it.

[0044] According to different feedback behaviors, corresponding rules are formulated to evaluate the preference degree of the user. For example, clicking to play and listening to a song completely can be counted as a higher score, while quickly skipping can be counted as a lower score; collecting and sharing can be given a higher score, because it indicates that the user has a higher degree of recognition for the music. Considering the combination and weight of multiple feedback behaviors, the preference degree of the user is comprehensively evaluated. For example, if the user collects a certain music and shares it with others, it can be judged that the user has a very high preference level for this music. The preference degree of the user is divided into different levels, such as very like, like, general, dislike, and very dislike. The score range obtained from the previous analysis can be used to correspond to different levels, for example, a score of 80-100 can be defined as very like, a score of 60-79 can be defined as like, and so on.

[0045] Among them, for the music with high user preference level, its position in the recommendation list is advanced, so that the user can easily discover and play again. For example, the music that the user likes very much is moved to the top of the list. For the music with low user preference level, its position is adjusted backward or directly removed from the recommendation list. For example, the music that the user dislikes very much is deleted from the list to avoid being recommended to the user again. The music that the user likes is analyzed for features, including melody, rhythm, lyrics style, singer, etc. For example, if the user likes a popular rock style song, the melody characteristics and chord progressions can be analyzed. According to the analysis results, music with similar characteristics is selected from the music resource library and added to the recommendation list. In this way, the user's preference for a specific music style can be met, and the relevance of the recommendation can be improved. The user's preference level information is fed back to the music recommendation algorithm to adjust the parameters of the algorithm. For example, if it is found that the user has a high preference for a certain style of music, the weight of music of that style in subsequent recommendations can be increased. With the continuous accumulation of user feedback data, the recommendation algorithm is continuously optimized to make the recommendation list more in line with the user's individual needs.

[0046] In the embodiments of the present application, the music recommendation list is continuously optimized according to user feedback to provide more accurate and user-preferred music recommendation services.

[0047] As shown in Figure 5 The music emotion determination method provided by the fifth embodiment of the present application is shown in the flowchart. Before the music recommendation method acquires the user's body data collected by the television device in real time, the music emotion determination method can include the following steps: Step S501, acquire a music file to be analyzed, and extract element features in the music file; Step S502, splice the element features to obtain a feature vector, and determine the music emotion features of the music file according to the feature vector; Optionally, the element features include low-level features and high-level features, and step S502 can include the following steps: Extract the spectral features in the low-level features and the timing features in the high-level features; Splice the spectral features and the timing features to obtain the feature vector, and determine the music emotion features of the music file according to the feature vector.

[0048] Step S503, store the music file and the music emotion features corresponding to the music file in a preset resource library.

[0049] Wherein, the music files that need to be analyzed for emotion are obtained from channels such as local storage, network music platforms, etc. The music files can be various common audio formats such as MP3, WAV, etc. The element features of music refer to various parameters that can reflect the characteristics of music, which can be generally divided into low-level features and high-level features. Low-level features refer to basic features directly extracted from audio signals, such as spectral features (including spectral energy distribution, spectral center frequency, etc.), time domain features (such as amplitude, tone length, etc.) reflecting the basic acoustic characteristics of music.

[0050] High-level features are features obtained by analyzing certain algorithms and models, such as rhythm patterns, melody trends, and harmonic structures. High-level features can better reflect the overall style and emotional expression of music. The low-level features and high-level features are spliced to form a multi-dimensional feature vector. For example, the spectral features and the time sequence features are spliced, the spectral features can reflect the characteristics of music in the frequency domain, and the time sequence features reflect the change rule of music over time, and the combination of the two can more comprehensively describe the features of music.

[0051] A pre-trained machine learning model or deep learning model is used to analyze and classify the feature vector, so as to determine the music emotion features of the music file. The model learns a large number of corresponding relationships between music features and emotions during the training process, and can predict the emotions expressed by the music according to the input feature vector, such as happy, sad, calm, and excited. A record is established for each music file in the resource library, and the corresponding music emotion features are associated. A database table can be used, wherein one column stores the identification information of the music file (such as file name, file path, etc.), and the other column stores the corresponding music emotion features.

[0052] The established mapping relationship is stored in a preset resource library, which can be a local database or a cloud storage system. After storage is completed, in the subsequent music recommendation process, music files that meet the requirements can be quickly screened from the resource library according to the emotional state of the user.

[0053] In the embodiments of the present application, a large number of music files are analyzed and classified for emotion, and the results are stored in the resource library, providing a rich data source for subsequent music recommendation based on user emotions.

[0054] It should be understood that the size of the serial number of each step in the above embodiments does not mean the order of execution, and the execution order of each process should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0055] For example, Figure 6As shown, it is a schematic view of a music recommendation device provided in Embodiment Six of the present application, which corresponds to the music recommendation method in the above-mentioned embodiments.

[0056] The music recommendation device comprises: a feature extraction module 61 configured to acquire body data of a user collected by a television device in real time, and extract behavior features of the body data; a vector construction module 62 configured to analyze the behavior features to obtain emotion features, obtain a current emotion of the user according to the emotion features, construct a vector representation of the current emotion, and obtain a user emotion vector; a resource matching module 63 configured to match, according to the user emotion vector, a music file corresponding to the user emotion vector in a preset resource library, wherein the preset resource library comprises a mapping relationship between music files and music emotion features corresponding to the music files; a music file recommendation module 64 configured to sort the music files according to the current emotion to obtain a music recommendation list, and sequentially display the music files on a display interface of the television device according to the music recommendation list.

[0057] Optionally, the vector construction module 62 comprises: a motion proportion calculation unit configured to determine a facial motion of the user according to the facial data, and calculate a proportion of the facial motion to obtain a facial motion proportion; a volume intensity determination unit configured to analyze data changes of the voice data to obtain a data fluctuation value, and determine a volume intensity change of the user according to the data fluctuation value; a data change determination unit configured to analyze changes of the physiological data corresponding to a preset physiological index according to the preset physiological index to obtain an index data change corresponding to the physiological index; an emotion determination unit configured to determine a current emotion of the user according to the facial motion proportion, the volume intensity change, and the index data change.

[0058] Optionally, the vector construction module 62 comprises: an emotion classification unit configured to classify the current emotion according to the emotion features to obtain an emotion type; an emotion vector determination unit configured to match in a preset category vector library according to the emotion type to obtain an emotion vector corresponding to the emotion type.

[0059] Optionally, the music recommendation device further comprises: The behavior analysis module is configured to obtain feedback behavior of the user on the television device after the music files are sequentially displayed on the display interface of the television device according to the music recommendation list, analyze the feedback behavior, and determine a preference level of the user on the music files. The strategy generation module is configured to generate an optimization strategy of the music recommendation list according to the preference level.

[0060] As shown in Figure 7 FIG. 7 is a schematic diagram of a music emotion determination device according to an embodiment of the present application.

[0061] The element feature extraction module 71 is configured to obtain a music file to be analyzed before the music recommendation method obtains the body data of the user collected by the television device in real time, and extract element features in the music file. The feature determination module 72 is configured to splice the element features to obtain a feature vector, and determine music emotion features of the music file according to the feature vector. The storage module 73 is configured to store the music file and the music emotion features corresponding to the music file in a preset resource library.

[0062] Optionally, the feature determination module 72 includes: The time sequence feature extraction unit is configured to extract frequency spectrum features in the low-level features and extract time sequence features in the high-level features. The emotion feature determination unit is configured to splice the frequency spectrum features and the time sequence features to obtain the feature vector, and determine the music emotion features of the music file according to the feature vector.

[0063] The specific limitations of the music recommendation device can refer to the limitations of the music recommendation method described above, and the specific limitations of the music emotion determination device can refer to the limitations of the music emotion determination method described above, which will not be repeated here. The music recommendation device and each module in the music recommendation device can be realized by software, hardware, or a combination thereof. The music emotion determination device and each module in the music emotion determination device can be realized by software, hardware, or a combination thereof. Each module can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory in the computer device in software form, so as to be called and executed by the processor to perform the operations corresponding to each module.

[0064] As shown in Figure 8As shown in the figure, it is a structural schematic diagram of a computer device provided in Embodiment Eight of the present application. The computer device comprises a processor, a memory, a network interface and a database connected through a system bus. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is configured to communicate with an external terminal through a network connection. The computer program is executed by the processor to implement a music recommendation method or a music emotion determination method.

[0065] In an embodiment, a computer device is provided, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, and the processor implements the music recommendation method as shown in any of the above embodiments or the music emotion determination method as shown in any of the above embodiments when executing the computer program. Details are not described herein again to avoid repetition. Figures 2 to 4 In an embodiment, a computer device is provided, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, and the processor implements the music recommendation method as shown in any of the above embodiments or the music emotion determination method as shown in any of the above embodiments when executing the computer program. Details are not described herein again to avoid repetition. Figures 5 to 6 In an embodiment, a computer device is provided, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, and the processor implements the music recommendation method as shown in any of the above embodiments or the music emotion determination method as shown in any of the above embodiments when executing the computer program. Details are not described herein again to avoid repetition. Figure 6 In an embodiment, a computer device is provided, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, and the processor implements the music recommendation method as shown in any of the above embodiments or the music emotion determination method as shown in any of the above embodiments when executing the computer program. Details are not described herein again to avoid repetition. Figure 7 In an embodiment, a computer device is provided, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, and the processor implements the music recommendation method as shown in any of the above embodiments or the music emotion determination method as shown in any of the above embodiments when executing the computer program. Details are not described herein again to avoid repetition.

[0066] In an embodiment, a computer readable storage medium is provided, and the computer readable storage medium stores a computer program. The computer program is executed by a processor to implement the music recommendation method as shown in any of the above embodiments. Details are not described herein again to avoid repetition. Alternatively, the computer program is executed by the processor to implement the functions of the modules / units in the music recommendation device as shown in any of the above embodiments, such as the functions of the feature extraction module 61, the vector construction module 62, the resource matching module 63 and the music file recommendation module 64 as shown in the figure, or the functions of the element feature extraction module 71, the feature determination module 72 and the storage module 73 as shown in the figure. Details are not described herein again to avoid repetition. Figures 2 to 5 In an embodiment, a computer readable storage medium is provided, and the computer readable storage medium stores a computer program. The computer program is executed by a processor to implement the music recommendation method as shown in any of the above embodiments. Details are not described herein again to avoid repetition. Alternatively, the computer program is executed by the processor to implement the functions of the modules / units in the music recommendation device as shown in any of the above embodiments, such as the functions of the feature extraction module 61, the vector construction module 62, the resource matching module 63 and the music file recommendation module 64 as shown in the figure, or the functions of the element feature extraction module 71, the feature determination module 72 and the storage module 73 as shown in the figure. Details are not described herein again to avoid repetition. Figure 6 In an embodiment, a computer readable storage medium is provided, and the computer readable storage medium stores a computer program. The computer program is executed by a processor to implement the music recommendation method as shown in any of the above embodiments. Details are not described herein again to avoid repetition. Alternatively, the computer program is executed by the processor to implement the functions of the modules / units in the music recommendation device as shown in any of the above embodiments, such as the functions of the feature extraction module 61, the vector construction module 62, the resource matching module 63 and the music file recommendation module 64 as shown in the figure, or the functions of the element feature extraction module 71, the feature determination module 72 and the storage module 73 as shown in the figure. Details are not described herein again to avoid repetition. Figure 7 In an embodiment, a computer readable storage medium is provided, and the computer readable storage medium stores a computer program. The computer program is executed by a processor to implement the music recommendation method as shown in any of the above embodiments. Details are not described herein again to avoid repetition. Alternatively, the computer program is executed by the processor to implement the functions of the modules / units in the music recommendation device as shown in any of the above embodiments, such as the functions of the feature extraction module 61, the vector construction module 62, the resource matching module 63 and the music file recommendation module 64 as shown in the figure, or the functions of the element feature extraction module 71, the feature determination module 72 and the storage module 73 as shown in the figure. Details are not described herein again to avoid repetition.

[0067] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiments of each method. Any reference to memory, storage, database or other medium used in each embodiment provided by the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0068] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above-mentioned division of each functional unit and module is exemplified, and in actual application, the above-mentioned functions can be completed by different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above.

[0069] The above-mentioned embodiments are only used to illustrate the technical solutions of the present application, but not to limit it; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can be modified, or some technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A music recommendation method, characterized in that, The music recommendation method is applied to a television terminal, and the music recommendation method includes: Acquire real-time body data of users collected by television devices, and extract behavioral features from the body data; The behavioral characteristics are analyzed to obtain emotional characteristics. Based on the emotional characteristics, the user's current emotion is obtained, and a vector representation of the current emotion is constructed to obtain the user's emotion vector. Based on the user's emotion vector, a music file corresponding to the user's emotion vector is matched in a preset resource library. The preset resource library includes a mapping relationship between music files and the corresponding music emotional features. The music files are sorted according to the current mood to obtain a music recommendation list, and the music files are displayed sequentially on the display interface of the TV device according to the music recommendation list.

2. The music recommendation method according to claim 1, characterized in that, The body data includes facial data, voice data, and physiological data; The process of analyzing the behavioral characteristics to obtain emotional characteristics, and obtaining the user's current emotion based on the emotional characteristics, includes: Based on the facial data, the user's facial movements are determined, and the proportion of the facial movements is calculated to obtain the proportion of facial movements. Analyze the data changes in the voice data to obtain data fluctuation values, and determine the user's volume intensity changes based on the data fluctuation values; Based on preset physiological indicators, analyze the changes in the physiological data corresponding to the physiological indicators to obtain the changes in the corresponding indicator data; The user's current emotion is determined based on the proportion of facial movements, the changes in volume intensity, and the changes in the indicator data.

3. The music recommendation method according to claim 1, characterized in that, The process of constructing a vector representation of the current emotion to obtain a user emotion vector includes: Based on the emotional characteristics, the current emotion is classified to obtain the emotion type; Based on the emotion type, a matching operation is performed in a preset category vector library to obtain the emotion vector corresponding to the emotion type.

4. The music recommendation method according to claim 1, characterized in that, After the music files are sequentially displayed on the display interface of the television device according to the music recommendation list, the method further includes: Obtain user feedback behavior on the TV device, analyze the feedback behavior, and determine the user's preference level for the music file; An optimization strategy for generating the music recommendation list based on the stated preference level.

5. A method for determining musical emotion, characterized in that, include: In the music recommendation method according to any one of claims 1 to 4, before acquiring the user's physical data collected in real time by the television device, the music file to be analyzed is acquired, and the element features of the music file are extracted; The element features are concatenated to obtain a feature vector, and the musical emotional features of the music file are determined based on the feature vector. The music file and the corresponding musical emotional features are stored in a preset resource library.

6. The method for determining musical emotion according to claim 5, characterized in that, The element features include low-level features and high-level features; The process of concatenating the element features to obtain a feature vector, and determining the musical emotional features of the music file based on the feature vector, includes: Extract spectral features from the low-level features and extract temporal features from the high-level features; The spectral features and the temporal features are concatenated to obtain the feature vector. Based on the feature vector, the musical emotional features of the music file are determined.

7. A music recommendation device, characterized in that, The music recommendation method is applied to a television terminal, and the music recommendation device includes: The feature extraction module is used to acquire the user's body data collected in real time by the television device and extract the behavioral features of the body data; The vector construction module analyzes the behavioral features to obtain emotional features, obtains the user's current emotion based on the emotional features, constructs a vector representation of the current emotion, and obtains the user's emotion vector. The resource matching module is used to match music files corresponding to the user's emotion vector in a preset resource library, wherein the preset resource library includes a mapping relationship between music files and the corresponding music emotional features. The music file recommendation module is used to sort the music files according to the current mood to obtain a music recommendation list, and then display the music files sequentially on the display interface of the TV device according to the music recommendation list.

8. A device for determining musical emotion, characterized in that, include: An element feature extraction module is used to obtain a music file to be analyzed and extract element features from the music file before obtaining the user's body data collected in real time by the television device in the music recommendation method according to any one of claims 1 to 4. The feature determination module is used to concatenate the element features to obtain a feature vector, and determine the musical emotional features of the music file based on the feature vector. The storage module is used to store the music file and the corresponding musical emotional features of the music file into a preset resource library.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the music recommendation method according to any one of claims 1 to 4 or the music emotion determination method according to claims 5 to 6.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the music recommendation method according to any one of claims 1 to 4 or the music emotion determination method according to claims 5 to 6.

Images