Song recommendation method and device, equipment and storage medium

Abstract

The invention relates to a song recommendation method and device, equipment and a storage medium. The method comprises the steps of obtaining a current emotional state of a driver, current driving environment information and a song emotion score of each song; calculating an emotion comprehensive score of the driver according to the current emotion state and the current driving environment information; according to the song emotion score of each song and the emotion comprehensive score, calculating the matching degree of each song and the driver emotion demand; and recommending the songs according to the matching degree. The recommended songs can effectively adjust the emotion of the driver.

Description

Technical Field

[0001] The present application relates to the field of vehicles, and in particular to a song recommendation method, device, equipment and storage medium. Background Art

[0002] With the development of technology, people are increasingly pursuing comfort while driving. In order to improve the comfort during driving, the driver can play music through the car music system to adjust the atmosphere in the car, thereby achieving the purpose of improving comfort.

[0003] In the related art, the in-car music system obtains the popularity of each song and recommends songs to users based on the popularity of the songs.

[0004] However, the above song recommendation method only recommends songs to users based on the popularity of the songs, and the recommended songs cannot effectively regulate the emotions of the drivers. Therefore, a song recommendation method is urgently needed. Summary of the invention

[0005] The present application provides a song recommendation method, device, equipment and storage medium, and the songs recommended by the method can effectively regulate the emotions of drivers.

[0006] In a first aspect, the present application provides a song recommendation method, the method comprising:

[0007] Obtain the driver's current emotional state, current driving environment information, and the song emotion score of each song;

[0008] Calculating a comprehensive emotional score of the driver according to the current emotional state and the current driving environment information;

[0009] Calculating the matching degree of each song with the driver's emotional needs according to the song emotion score of each song and the emotional comprehensive score;

[0010] According to the matching degree, songs are recommended.

[0011] Optionally, the current driving environment information includes a current time point and a current driving condition, and the calculating the driver's comprehensive emotional score according to the current emotional state and the current driving environment information includes:

[0012] Determining a first score corresponding to the current emotional state according to a correspondence between the emotional state and the score;

[0013] Determine the score corresponding to the time period at the current time point according to the correspondence between the time period and the score, and determine the score as the second score corresponding to the current time point;

[0014] Determining a third score corresponding to the current driving condition according to a correspondence between the driving condition and the score;

[0015] A comprehensive emotion score of the driver is calculated based on the first score, the second score, and the third score.

[0016] Optionally, the method further includes:

[0017] According to the correspondence between the time period and the emotional state, and the correspondence between the emotional state and the score, the correspondence between the time period and the score is obtained;

[0018] According to the correspondence between the driving condition and the emotional state, and the correspondence between the emotional state and the score, the correspondence between the driving condition and the score is obtained.

[0019] Optionally, when the comprehensive emotion score is proportional to the intensity of the emotion, and the song emotion score is inversely proportional to the intensity of the emotion, calculating the matching degree of each song with the driver's emotional needs based on the song emotion score and the comprehensive emotion score of each song includes:

[0020] For the song emotion score of each song, the difference between the song emotion score and the comprehensive emotion score is calculated, and the matching degree between the song and the driver's emotional needs is determined based on the difference.

[0021] Optionally, the step of generating the song emotion includes:

[0022] Get the audio data of the song;

[0023] Determine the melody and rhythm features of the song according to the audio data;

[0024] Merging the melody feature and the rhythm feature to obtain a fusion feature;

[0025] The fusion feature is input into the trained emotion model to obtain the emotion of the song.

[0026] Optionally, determining the melody feature of the song according to the audio data includes:

[0027] Extracting pitch sequences and pitch contours from audio data;

[0028] According to the preset rules, the audio data is divided to obtain at least one melody:

[0029] According to the pitch sequence corresponding to each melody, determine the pitch span corresponding to each melody and the pitch difference between adjacent notes;

[0030] In each melody, determine the melody corresponding to the pitch sequence whose similarity exceeds a threshold, and obtain a repeated melody;

[0031] The pitch contour, repeated melody, the pitch span corresponding to each melody segment and the pitch difference between adjacent notes are identified as melodic features.

[0032] Optionally, determining the melody feature of the song according to the audio data includes:

[0033] Using a beat detection algorithm, identifying the audio data to obtain a strong beat position;

[0034] Dividing the song according to the strong beat position to obtain at least one rhythm segment;

[0035] Analyzing the changes in the strength of the beats in the rhythm segment to determine the rhythm pattern corresponding to the rhythm segment;

[0036] A melodic feature of the song is determined based on the audio data, at least one rhythm section, and the rhythm pattern.

[0037] In a second aspect, the present application provides a song recommendation device, the device comprising:

[0038] An acquisition unit, used to acquire the driver's current emotional state, current driving environment information, and a song emotion score of each song;

[0039] A first calculation unit, configured to calculate a comprehensive emotion score of the driver according to the current emotional state and the current driving environment information;

[0040] A second calculation unit is used to calculate the matching degree of each song with the driver's emotional needs according to the song emotion score of each song and the emotional comprehensive score;

[0041] The recommendation unit is used to recommend songs according to the matching degree.

[0042] Optionally, the current driving environment information includes a current time point and a current driving condition, and the first calculation unit includes:

[0043] Determining a first score corresponding to the current emotional state according to a correspondence between the emotional state and the score;

[0044] Determine the score corresponding to the time period at the current time point according to the correspondence between the time period and the score, and determine the score as the second score corresponding to the current time point;

[0045] Determining a third score corresponding to the current driving condition according to a correspondence between the driving condition and the score;

[0046] A comprehensive emotion score of the driver is calculated based on the first score, the second score, and the third score.

[0047] Optionally, the device further includes a relationship determination unit, including:

[0048] According to the correspondence between the time period and the emotional state, and the correspondence between the emotional state and the score, the correspondence between the time period and the score is obtained;

[0049] According to the correspondence between the driving condition and the emotional state, and the correspondence between the emotional state and the score, the correspondence between the driving condition and the score is obtained.

[0050] Optionally, when the comprehensive emotion score is proportional to the intensity of the emotion, and the song emotion score is inversely proportional to the intensity of the emotion, the second calculation unit includes:

[0051] For the song emotion score of each song, the difference between the song emotion score and the comprehensive emotion score is calculated, and the matching degree between the song and the driver's emotional needs is determined based on the difference.

[0052] Optionally, the device further includes an emotion determination unit, including:

[0053] Get the audio data of the song;

[0054] Determine the melody and rhythm features of the song according to the audio data;

[0055] Merging the melody feature and the rhythm feature to obtain a fusion feature;

[0056] The fusion feature is input into the trained emotion model to obtain the emotion of the song.

[0057] Optionally, the emotion determination unit includes:

[0058] Extracting pitch sequences and pitch contours from audio data;

[0059] According to the preset rules, the audio data is divided to obtain at least one melody:

[0060] According to the pitch sequence corresponding to each melody, determine the pitch span corresponding to each melody and the pitch difference between adjacent notes;

[0061] In each melody, determine the melody corresponding to the pitch sequence whose similarity exceeds a threshold, and obtain a repeated melody;

[0062] The pitch contour, repeated melody, the pitch span corresponding to each melody segment and the pitch difference between adjacent notes are identified as melodic features.

[0063] Optionally, the emotion determination unit includes:

[0064] Using a beat detection algorithm, identifying the audio data to obtain a strong beat position;

[0065] Dividing the song according to the strong beat position to obtain at least one rhythm segment;

[0066] Analyzing the changes in the strength of the beats in the rhythm segment to determine the rhythm pattern corresponding to the rhythm segment;

[0067] A melodic feature of the song is determined based on the audio data, at least one rhythm section, and the rhythm pattern.

[0068] In a third aspect, the present application provides a song recommendation device, comprising: at least one communication interface; at least one bus connected to the at least one communication interface; at least one processor connected to the at least one bus; at least one memory connected to the at least one bus, wherein the processor is configured to:

[0069] Obtain the driver's current emotional state, current driving environment information, and the song emotion score of each song;

[0070] Calculating a comprehensive emotional score of the driver according to the current emotional state and the current driving environment information;

[0071] Calculating the matching degree of each song with the driver's emotional needs according to the song emotion score of each song and the emotional comprehensive score;

[0072] According to the matching degree, songs are recommended.

[0073] In a fourth aspect, the present application also provides a computer storage medium storing computer executable instructions, wherein the computer executable instructions are used to execute the song recommendation method described in any one of the above items of the present application.

[0074] The above technical solution provided by the embodiment of the present application has the following advantages compared with the prior art: the present application will calculate the matching degree of each song with the driver's emotional needs according to the song emotion score of each song and the driver's emotional comprehensive score, and then recommend songs based on the matching degree. This is essentially to find songs that meet the driver's emotional needs through the driver's current emotions, and then recommend the song. Since the song emotions of the recommended songs match the driver's current emotional needs, the recommended songs are more in line with the driver's emotional needs, so that the recommended songs can effectively regulate the driver's emotions. At the same time, since the current equipment cannot accurately know the driver's emotions, and the driving environment in which the driver is located can affect the driver's emotions to a certain extent, therefore, the emotional comprehensive score calculated according to the current emotional state and the current driving environment information can more accurately reflect the driver's emotions, and then the recommended songs are more in line with the driver's emotional needs, so that the recommended songs can more effectively regulate the driver's emotions. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0076] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0077] One or more embodiments are exemplarily described by pictures in the corresponding drawings, and these exemplified descriptions do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings represent similar elements, and unless otherwise stated, the figures in the drawings do not constitute proportional limitations.

[0078] Figure 1 A flowchart of a song recommendation method provided in an embodiment of the present application;

[0079] Figure 2 A flowchart of a method for determining a comprehensive emotion score provided in an embodiment of the present application;

[0080] Figure 3 A schematic diagram of a flow chart of a method for determining the emotion of a song provided in an embodiment of the present application;

[0081] Figure 4 A schematic diagram of a flow chart of a method for determining melody features provided in an embodiment of the present application;

[0082] Figure 5A schematic diagram of a flow chart of a method for determining rhythm characteristics provided in an embodiment of the present application;

[0083] Figure 6 A schematic diagram of a flow chart of a song recommendation device provided in an embodiment of the present application;

[0084] Figure 7 A schematic diagram of a song recommendation device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0085] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0086] The disclosure below provides many different embodiments or examples to implement different structures of the present invention. In order to simplify the disclosure of the present invention, the parts and settings of specific examples are described below. Of course, they are only examples, and the purpose is not to limit the present invention. In addition, the present invention can repeat reference numbers and / or letters in different examples. This repetition is for the purpose of simplification and clarity, and does not itself indicate the relationship between the various embodiments and / or settings discussed.

[0087] In real life, drivers can adjust the atmosphere in the car by playing music through the in-car music system, thereby achieving the purpose of improving comfort. In related technologies, the in-car music system obtains the popularity of each song and recommends songs to users based on the popularity of the songs. However, the above method only recommends songs to users based on the popularity of the songs, and the recommended songs cannot effectively adjust the emotions of the driver.

[0088] To solve the above problems, an embodiment of the present application provides a song recommendation method, which calculates the matching degree of each song with the driver's emotional needs based on the song emotion score of each song and the driver's emotional comprehensive score, and then recommends songs based on the matching degree. The above method is essentially to find songs that meet the driver's emotional needs through the driver's current emotions, and then recommend the song. Since the song emotions of the recommended songs match the driver's current emotional needs, the recommended songs are more in line with the driver's emotional needs, so that the recommended songs can effectively regulate the driver's emotions. At the same time, since the current equipment cannot accurately know the driver's emotions, and the driving environment in which the driver is located can affect the driver's emotions to a certain extent, therefore, the emotional comprehensive score calculated based on the current emotional state and the current driving environment information can more accurately reflect the driver's emotions, and then the recommended songs are more in line with the driver's emotional needs, so that the recommended songs can more effectively regulate the driver's emotions.

[0089] In summary, this application provides a song recommendation method, such as Figure 1 As shown, the specific steps include:

[0090] Step 101, obtaining the driver's current emotional state, current driving environment information, and the song emotion score of each song.

[0091] Among them, emotional states include calmness, happiness, anxiety, fatigue, etc.

[0092] In this step, the in-vehicle music system can determine the driver's emotional state through emotion detection technology based on EEG data. Alternatively, the in-vehicle music system can also determine the driver's emotional state through facial expression recognition technology.

[0093] In this step, the driving environment information may include the current driving condition and the current time point. The current driving condition includes the current vehicle speed, the current road congestion and weather conditions. In this step, the in-vehicle music system may determine the current driving condition through data collected by the in-vehicle sensor. For example, the in-vehicle music system obtains the current speed of the vehicle by collecting data from the speed sensor. The in-vehicle music system obtains the acceleration of the current vehicle by collecting data from the acceleration sensor, and when the acceleration is greater than a preset value, it determines that the current road is congested, and when the acceleration is less than a preset value, it determines that the current road is not congested. The in-vehicle music system may obtain the current time point by collecting data from a clock device.

[0094] In this step, the song emotion score is generated based on the song emotion. The song emotion mainly indicates the main emotion of the song. In practice, the song emotion of the song can be determined by manual labeling, or by a trained machine learning model, or by other methods of determining the song emotion, which will not be described here one by one. In this step, after the song emotion of the song is obtained, the song emotion score corresponding to the song can be determined based on the correspondence between the preset song emotion and the score. This step can also use other methods to determine the song emotion score, which will not be described here one by one.

[0095] In practice, the in-car music system can obtain the song emotion score of each song in advance, and obtain the driver's current emotional state and current driving environment information through other sensors.

[0096] Step 102, calculating the driver's comprehensive emotion score according to the current emotional state and the current driving environment information.

[0097] In this step, the in-car music system can input the emotional state and the current driving environment information into a pre-trained neural network model to obtain an emotional comprehensive score, wherein the pre-trained neural network model is trained based on the emotional state and the driving environment information, and the training method is the same as the training method in the prior art, which will not be repeated here. The in-car music system can also pre-set the correspondence between the emotional state and the score, as well as the correspondence between the driving environment information and the score, and determine the score corresponding to the current emotional state and the score corresponding to the current driving environment information based on the above correspondence, and then calculate the emotional comprehensive score based on these two scores. Other calculation methods can also be used in this step, which will not be repeated here.

[0098] Step 103, calculating the matching degree of each song with the driver's emotional needs according to the song emotion score and the emotional comprehensive score of each song.

[0099] Among them, songs that match the driver's emotional needs can adjust the driver's emotions so that the driver can drive the vehicle safely under this emotion. Since the recommended songs can adjust the driver's emotions, the song emotions that meet the driver's emotional needs are more likely to be opposite to the driver's emotions. For example, the driver's emotions in the driving scenario are anxious, and his emotional needs are to play cheerful songs.

[0100] In this step, the in-vehicle music system can calculate the matching degree of each song with the driver's emotional needs based on the song emotion score and the comprehensive emotion score of each song.

[0101] Step 104, recommending songs based on the matching degree.

[0102] In this step, the in-car music system can determine the matching degree of each song with the driver's emotional needs as the matching degree of each song, and sort the songs according to the matching degree of each song. Afterwards, the in-car music system can make recommendations based on the sorted songs. For example, the in-car music system recommends the first preset song to the user, or recommends songs with a matching degree exceeding a preset value to the user. This step does not limit the recommendation method, and will not be described one by one here.

[0103] After obtaining the songs to be recommended based on the matching degree, the popularity of each song to be recommended can also be obtained, and the songs to be recommended whose popularity exceeds a preset value can be recommended to the user. In this way, the songs recommended to the user are more likely to meet the user's needs, thereby improving the user experience.

[0104] Among them, popularity is mainly reflected in the number of song plays, the number of song likes, and the number of song comments. The music recommendation system can count the number of song plays, the number of song likes, and the number of song comments to determine the popularity of the song.

[0105] Alternatively, after obtaining the songs to be recommended based on the matching degree, the user's historical preferences can also be obtained, and based on the user's historical preferences, recommended songs can be selected from the songs to be recommended and pushed to the user. In this way, the songs recommended to the user are more likely to meet the user's needs, thereby improving the user experience.

[0106] In an embodiment of the present application, the matching degree of each song with the driver's emotional needs can be calculated based on the song emotion score of each song and the driver's emotional comprehensive score, and then the song recommendation can be made based on the matching degree. This is essentially to find songs that meet the driver's emotional needs through the driver's current emotions, and then recommend the song. Since the song emotions of the recommended songs match the driver's current emotional needs, the recommended songs are more in line with the driver's emotional needs, so that the recommended songs can effectively regulate the driver's emotions. At the same time, since the current equipment cannot accurately know the driver's emotions, and the driving environment in which the driver is located can affect the driver's emotions to a certain extent, therefore, the emotional comprehensive score calculated based on the current emotional state and the current driving environment information can more accurately reflect the driver's emotions, and then the recommended songs are more in line with the driver's emotional needs, so that the recommended songs can more effectively regulate the driver's emotions.

[0107] In the embodiments of the present application, Figure 2 The step 102 of "calculating the driver's comprehensive emotional score according to the current emotional state and the current driving environment information" is further limited as follows:

[0108] Step 21, determining a first score corresponding to the current emotional state according to the correspondence between the emotional state and the score.

[0109] Among them, the correspondence between emotional state and score is set by the technician as needed. Specifically, the technician sets the score according to the intensity of the emotion, and specifically sets it to the point that the calmer the emotion, the higher the corresponding score, and the more intense the emotion, the lower the corresponding score. For example, the score corresponding to relaxed emotion is set to 100, the score corresponding to happy emotion is set to 90, the score corresponding to mild anxiety is set to 70, the score corresponding to high anxiety is set to 50, and the score corresponding to extreme discomfort is set to 30. The technician can also set the more intense the emotion, the higher the corresponding score, and the calmer the emotion, the lower the corresponding score. The specific example is similar to the above example, and will not be repeated here.

[0110] In this step, the in-vehicle music system may pre-store the correspondence between the emotional state and the score, and determine the score corresponding to the current emotional state based on the correspondence, and determine the score as the first score.

[0111] Step 22, according to the correspondence between the time period and the score, determine the score corresponding to the time period at the current time point, and determine the score as the second score corresponding to the current time point.

[0112] In this step, the music recommendation system may pre-store the correspondence between time periods and scores, and determine the score corresponding to the time period at the current time point based on the correspondence, and determine the score as the second score corresponding to the current time point.

[0113] Step 23: Determine a third score corresponding to the current driving condition according to the correspondence between the driving condition and the score.

[0114] In this step, the in-vehicle music system may pre-store the correspondence between driving conditions and scores, and determine the score of the current driving condition based on the correspondence, and determine the score as the third score.

[0115] Step 24, calculating a comprehensive score of the scene according to the first score, the second score and the third score.

[0116] In this step, among the three parameters of emotional state, time point and driving condition, since the emotional state is the most important to the driver's driving process, followed by driving condition and finally time point, the corresponding weight can be set in advance for each parameter. For example, the weight corresponding to the first score is set to the first weight, the weight corresponding to the second score is set to the second weight, and the weight corresponding to the third score is set to the third weight, wherein the first weight is greater than the second weight, the second weight is greater than the third weight, and the sum of the first weight, the second weight and the third weight is equal to 1. Afterwards, the first score is multiplied by the first weight, the second score is multiplied by the second weight, the third score is multiplied by the third weight, and the three multiplication results are added together to obtain the scene comprehensive score.

[0117] Specifically, the first weight is set to 0.5, the second weight is set to 0.4, and the third weight is set to 0.1.

[0118] In an embodiment of the present application, before obtaining the current emotional state of the driver, a method for determining the correspondence between the time period and the score is also provided, and the method is specifically as follows:

[0119] According to the correspondence between the time period and the emotional state, and the correspondence between the emotional state and the score, the correspondence between the time period and the score is obtained.

[0120] In this step, different time periods may lead to different emotional states. For example, during the morning rush hour, traffic jams are likely to cause drivers to lose their temper. Therefore, the emotional states that may be caused by different time periods can be determined first, and the corresponding relationship between time periods and emotional states can be obtained. Then, the corresponding relationship between time periods and scores can be obtained based on the corresponding relationship and the corresponding relationship between emotional states and scores.

[0121] Alternatively, the corresponding relationship between emotional needs and scores is determined according to the corresponding relationship between emotional states and scores, and the corresponding relationship between time periods and scores is determined according to the emotional needs of each time period and the corresponding relationship between emotional needs and scores. Since the purpose of obtaining the emotional state of the driver is essentially to determine the emotional needs of the driver in the emotional state, the corresponding relationship involved in step 21 is essentially the corresponding relationship between the driver's emotional needs and scores. In this step, the driver's emotional needs are different for different time periods. For example, cheerful emotions are often needed in the morning period to keep the driver awake, and relaxed emotions are often needed in the afternoon period to keep the driver relaxed. In summary, the purpose of determining the driver's driving time period is also to determine the driver's emotional needs in the current time period, so the corresponding relationship between time periods and scores is essentially the corresponding relationship between the driver's emotional needs and scores. In summary, the corresponding relationships involved in step 21 and this step are both the corresponding relationships between the driver's emotional needs and scores, and they should be consistent in the corresponding relationship between emotional needs and scores. Therefore, after the technicians set the correspondence between emotional states and scores, they also need to determine the correspondence between emotional needs and scores based on the correspondence between emotional states and scores, and determine the score corresponding to each time period based on the emotional needs in each time period and the correspondence between emotional needs and scores.

[0122] In an embodiment of the present application, before obtaining the current emotional state of the driver, a method for determining the corresponding relationship between the driving condition and the score is also provided, and the method is specifically as follows:

[0123] According to the correspondence between the driving condition and the emotional state, and the correspondence between the emotional state and the score, the correspondence between the driving condition and the score is obtained.

[0124] In this step, different driving conditions may lead to different emotional states. For example, traffic jams can easily cause drivers to lose their temper. Therefore, the emotional states caused by different driving conditions can be determined first, and the corresponding relationship between driving conditions and emotional states can be obtained. Then, the corresponding relationship between driving conditions and scores can be obtained based on the corresponding relationship and the corresponding relationship between emotional states and scores.

[0125] Specifically, if the correspondence between the emotional state and the score is that the more intense the emotion, the higher the corresponding score, and the calmer the emotion, the lower the corresponding score, then the correspondence between the driving condition and the score can be set to that the more congested the driving condition, the higher the corresponding score, and the less congested the driving condition, the lower the corresponding score. If the correspondence between the emotional state and the score is that the calmer the emotion, the higher the corresponding score, and the more intense the emotion, the lower the corresponding score, then the correspondence between the driving condition and the score can be set to that the less congested the driving condition, the higher the corresponding score, and the more congested the driving condition, the lower the corresponding score. For example, if the score corresponding to the tense emotional state is set to 100 and the score corresponding to the relaxed emotional state is set to 0, then the score corresponding to the congested condition is set to 100 and the score corresponding to the unblocked condition is set to 0.

[0126] Alternatively, the correspondence between the emotional needs and the scores is determined based on the correspondence between the emotional states and the scores, and the correspondence between the driving conditions and the scores is determined based on the emotional needs of each driving condition and the correspondence between the emotional needs and the scores.

[0127] This is similar to the method of determining the correspondence between time periods and scores based on emotional needs, and will not be described in detail here.

[0128] In the embodiment of the present application, the step 103 of "calculating the matching degree between each song and the driver's emotional needs according to the song emotion score and the emotional comprehensive score of each song" is further defined as follows:

[0129] For each song's emotion score, the difference between the song's emotion score and the comprehensive emotion score is calculated, and based on the difference, the degree of match between the song and the driver's emotional needs is determined.

[0130] In this step, since the driver's emotions are adjusted according to the driver's emotional needs, the driver's emotional needs often have a certain gap with the driver's emotions, that is, the song emotions of the recommended songs often have a certain gap with the driver's emotions. Then, when the emotional comprehensive score is proportional to the intensity of the emotions, and the song emotional score is proportional to the intensity of the emotions, the in-vehicle music system can calculate the difference between the emotional comprehensive score and the song emotional score. The smaller the difference, the closer the driver's emotions are to the song emotions, and the less the driver's emotions match the song emotions. The larger the difference, the greater the difference between the driver's emotions and the song emotions, and the more the driver's emotions match the song emotions. Therefore, in this step, the reciprocal of the difference can be used as the matching degree of the driving emotion and the song emotion.

[0131] When the comprehensive emotion score is inversely proportional to the intensity of the emotion, and the song emotion score is proportional to the intensity of the emotion, the in-vehicle music system can calculate the difference between the comprehensive emotion score and the song emotion score. When the difference is smaller, the greater the difference between the driver's emotion and the song emotion, the more the driver's emotion and the song emotion match. When the difference is larger, the closer the driver's emotion and the song emotion are, the less the driver's emotion and the song emotion match. Therefore, in this step, the difference can be used as the matching degree of the driving emotion and the song emotion.

[0132] Similarly, when the comprehensive emotion score is proportional to the intensity of the emotion, and the song emotion score is inversely proportional to the intensity of the emotion, the in-car music system can calculate the difference between the comprehensive emotion score and the song emotion score, and use the difference as the matching degree between the driving emotion and the song emotion. In an embodiment of the present application, in order to achieve real-time song recommendation, the song emotion score of each song can be pre-calculated on the back-end server of the in-car music system, so that when the front-end of the in-car music system needs to make song recommendations, the song emotion score of each song can be directly obtained from the back-end server. Based on the above, the back-end server needs to determine the song emotion of each song before it can continue to determine the song emotion score of each song. Therefore, an embodiment of the present application provides a specific implementation method for determining the emotion of a song, such as Figure 3 As shown, the specific steps are as follows:

[0133] 31. Get the audio data of the song.

[0134] The audio data is the digitized sound data of the song.

[0135] In this step, the database stores the audio data of each song. When the audio data of a certain song needs to be obtained, the back-end server can obtain the audio data of the song in advance in the database.

[0136] 32. Determine the melody and rhythm features of the song based on the audio data.

[0137] Among them, the melody feature is a feature representing the melody of the song, including pitch sequence, melody range, repeated melody, etc. The rhythm feature is a feature representing the rhythm of the song, including rhythm data, rhythm pattern and other data.

[0138] In this step, the back-end server analyzes the audio data and extracts the melody and rhythm features of the song.

[0139] 33, the melody feature and the rhythm feature are integrated to obtain the fusion feature.

[0140] In this step, the melody features and rhythm features are input into the fully connected layer or certain specific layers, and then the spliced ​​features are output. Afterwards, in order to ensure that all features are on the same scale, the spliced ​​features can be normalized to obtain the normalized features. Then, the covariance matrix is ​​calculated by the normalized features, and the eigenvalues ​​and corresponding eigenvectors of the covariance matrix are found. The eigenvalues ​​represent the variance along that direction. Sort by eigenvalue size, select the eigenvectors corresponding to the first k largest eigenvalues, and make them constitute a new feature space. Project the melody features into the new feature space to obtain the reduced-dimensional features.

[0141] After obtaining the features after dimensionality reduction, a deep learning model can be established to automatically learn the complex relationships between different features, and then achieve feature fusion by stacking and connecting in the neural network layer in the deep learning model. The deep learning model involved above includes a multi-input module, a self-attention module, a residual connection module and a fusion module. The multi-input module is used to receive input features. The self-attention module is used to weight the importance of different features, so that the deep learning model learns which features are more important. The residual connection module allows information to be directly transmitted across several layers. The fusion module can be a multiplication module, an addition module, etc., which is used to achieve the fusion of different features.

[0142] 34. Input the fused features into the trained emotion model to obtain the emotion of the song.

[0143] The emotion model is a neural network model, which can be a TensorFlow model.

[0144] In this step, it is necessary to first obtain a trained emotion model. The method for obtaining the trained emotion model is as follows: first set the emotion model and the training set to be used, the training set includes multiple samples, each sample includes the fusion features of the song and the benchmark song emotion. Input the fusion features in the sample into the emotion model to obtain the predicted song emotion. Then, based on the predicted song emotion and the benchmark song emotion in the sample, adjust the emotion model to obtain the adjusted emotion model. Then, based on other samples in the training set, continue to adjust the adjusted emotion model until the adjusted emotion model meets the preset conditions to obtain the trained emotion model.

[0145] In the embodiment of the present application, the audio data of the song is obtained, and then the melody features and rhythm features that can represent the characteristics of the song are obtained based on the audio data, and then the melody features and rhythm features are merged and input into the trained emotion model to obtain the song emotion of the song. Since the present application obtains the song emotion based on the trained emotion model, the present application does not need to obtain the song emotion of the song by manual labeling, which can improve the efficiency of obtaining the song emotion.

[0146] In the embodiment of the present application, the step 32 of "determining the melody characteristics of the song according to the audio data" is further defined as follows: Figure 4 As shown, specifically including:

[0147] Step 41: extract the pitch sequence and pitch contour from the audio data.

[0148] The pitch is based on the frequency of the song and is used to indicate the high or low pitch of the sound. The pitch contour describes the tendency of the melody line to rise, fall, or remain stable.

[0149] In this step, tools such as PRAAT, YIN or CREPE are used. These algorithms can estimate a continuous pitch curve from an audio signal to obtain a pitch contour, and then perform data sampling on the pitch curve to obtain a pitch sequence.

[0150] Step 42: Divide the audio data according to a preset rule to obtain at least one melody.

[0151] The melody is a short segment of audio data in a song.

[0152] In this step, a song often includes a prelude, a verse and a chorus. Therefore, the audio data can be divided into at least one section according to the prelude, the verse and the chorus in the song, and each section of the audio data is used as a melody.

[0153] Step 43, according to the pitch sequence corresponding to each melody section, determine the pitch span corresponding to each melody section and the pitch difference between adjacent notes.

[0154] In this step, for the pitch sequence corresponding to each melody, the maximum pitch value and the minimum pitch value in the pitch sequence are found, and the difference between the two values ​​is calculated to obtain the pitch span corresponding to the melody.

[0155] In this step, a note is a basic unit in a song, and one pitch value corresponds to one note. For the pitch sequence corresponding to each melody, the pitches of adjacent notes in the melody sequence are compared pair by pair to obtain the pitch difference between the adjacent notes.

[0156] Step 44, in each melody, determine the melody corresponding to the pitch sequence whose similarity exceeds the threshold, and obtain a repeated melody.

[0157] Among them, repeated melodies are often related to the theme of the song and can more accurately express the mood of the song.

[0158] In this step, for each melody, dynamic time warping or edit distance algorithm can be used to identify the pitch sequence in each melody whose similarity exceeds a threshold, and the audio data corresponding to the pitch sequence is determined as a repeated melody.

[0159] Step 45, determining the pitch contour, repeated melody, the pitch span corresponding to each melody segment and the pitch difference between adjacent notes as melody features.

[0160] In the embodiment of the present application, the step 42 of "determining the rhythm characteristics of the song according to the audio data" is further defined as follows: Figure 5 As shown, specifically including:

[0161] Step 51, using a beat detection algorithm to identify the audio data and obtain strong beat positions.

[0162] Among them, the beat is often the basic pulse in a song.

[0163] In this step, strong beat positions are detected using a beat detection algorithm from a music information retrieval library like Madmom, Essentia or Librosa.

[0164] Step 52, dividing the song according to the strong beat positions to obtain at least one rhythm segment.

[0165] In this step, a rhythm segment is often composed of three beats, wherein the first beat is often a strong beat. In this way, after determining the strong beat position, the beat corresponding to the position and the two beats after the position can be used as a rhythm segment.

[0166] Step 53: Analyze the changes in the strength of the beats in the rhythm segment to determine the rhythm pattern corresponding to the rhythm segment.

[0167] The corresponding relationship between the rhythm pattern and the beat strength change is pre-stored, so that the beat strength change in the rhythm segment can be analyzed based on the corresponding relationship to obtain the rhythm pattern corresponding to the rhythm segment.

[0168] Step 54, determining the rhythmic features of the song based on the audio data, at least one rhythmic segment and the rhythmic pattern.

[0169] In this step, the entropy of the pitch sequence corresponding to each rhythm segment can also be calculated. When the entropy is higher, the corresponding rhythm segment is more unpredictable, and when the entropy is lower, the corresponding rhythm segment is simpler. At the same time, the average number of notes per unit time in each rhythm segment can also be calculated, which is used as the rhythm density. When the rhythm density is higher, the corresponding rhythm segment is more complex. When the rhythm density is lower, the corresponding rhythm segment is simpler. After determining the rhythm pattern corresponding to each rhythm segment, the type of rhythm pattern and the frequency of occurrence of each rhythm pattern can also be counted to obtain rhythm diversity. Afterwards, at least one rhythm segment, rhythm diversity, and the rhythm pattern, entropy, and rhythm density corresponding to each rhythm segment are determined as the rhythm characteristics of the song.

[0170] like Figure 6 As shown, an embodiment of the present application provides a song recommendation device, the device comprising:

[0171] An acquisition unit 601 is used to acquire the current emotional state of the driver, current driving environment information, and the song emotion score of each song;

[0172] A first calculation unit 602, configured to calculate a comprehensive emotion score of the driver according to the current emotional state and the current driving environment information;

[0173] A second calculation unit 603 is used to calculate the matching degree between each song and the driver's emotional needs according to the song emotion score of each song and the emotional comprehensive score;

[0174] The recommendation unit 604 is used to recommend songs according to the matching degree.

[0175] Optionally, the current driving environment information includes a current time point and a current driving condition, and the first calculation unit 602 includes:

[0176] Determining a first score corresponding to the current emotional state according to a correspondence between the emotional state and the score;

[0177] Determine the score corresponding to the time period at the current time point according to the correspondence between the time period and the score, and determine the score as the second score corresponding to the current time point;

[0178] Determining a third score corresponding to the current driving condition according to a correspondence between the driving condition and the score;

[0179] A comprehensive emotion score of the driver is calculated based on the first score, the second score, and the third score.

[0180] Optionally, the device further includes a relationship determination unit 605, including:

[0181] According to the correspondence between the time period and the emotional state, and the correspondence between the emotional state and the score, the correspondence between the time period and the score is obtained;

[0182] According to the correspondence between the driving condition and the emotional state, and the correspondence between the emotional state and the score, the correspondence between the driving condition and the score is obtained.

[0183] Optionally, when the comprehensive emotion score is proportional to the emotion intensity, and the song emotion score is inversely proportional to the emotion intensity, the second calculation unit 603 includes:

[0184] For the song emotion score of each song, the difference between the song emotion score and the comprehensive emotion score is calculated, and the matching degree between the song and the driver's emotional needs is determined based on the difference.

[0185] Optionally, the device further includes an emotion determination unit 606, including:

[0186] Get the audio data of the song;

[0187] Determine the melody and rhythm features of the song according to the audio data;

[0188] Merging the melody feature and the rhythm feature to obtain a fusion feature;

[0189] The fusion feature is input into the trained emotion model to obtain the emotion of the song.

[0190] Optionally, the emotion determination unit 606 includes:

[0191] Extracting pitch sequences and pitch contours from audio data;

[0192] According to the preset rules, the audio data is divided to obtain at least one melody:

[0193] According to the pitch sequence corresponding to each melody, determine the pitch span corresponding to each melody and the pitch difference between adjacent notes;

[0194] In each melody, determine the melody corresponding to the pitch sequence whose similarity exceeds a threshold, and obtain a repeated melody;

[0195] The pitch contour, repeated melody, the pitch span corresponding to each melody segment, and the pitch difference between adjacent notes are identified as melodic features.

[0196] Optionally, the emotion determination unit 606 includes:

[0197] Using a beat detection algorithm, identifying the audio data to obtain a strong beat position;

[0198] Dividing the song according to the strong beat position to obtain at least one rhythm segment;

[0199] Analyzing the changes in the strength of the beats in the rhythm segment to determine the rhythm pattern corresponding to the rhythm segment;

[0200] A melodic feature of the song is determined based on the audio data, at least one rhythm section, and the rhythm pattern.

[0201] like Figure 7 As shown, the embodiment of the present application provides a song recommendation device, including a processor 701, a communication interface 702, a memory 703 and a communication bus 704, wherein the processor 701, the communication interface 702, and the memory 703 communicate with each other through the communication bus 704.

[0202] Memory 703, used for storing computer programs;

[0203] In one embodiment of the present application, the processor 701 is used to execute the program stored in the memory 703 to implement the song recommendation method provided by any of the above method embodiments, including:

[0204] An embodiment of the present application also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the song recommendation method provided in any of the aforementioned method embodiments are implemented.

[0205] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0206] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a general hardware platform, and of course, by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0207] It should be understood that the terms used herein are only for the purpose of describing specific example embodiments and are not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms "one", "an" and "said" as used herein may also be meant to include plural forms. The terms "include", "comprise", "contain", and "have" are inclusive, and therefore specify the existence of stated features, steps, operations, elements and / or parts, but do not exclude the existence or addition of one or more other features, steps, operations, elements, parts, and / or combinations thereof. The method steps, processes, and operations described herein are not interpreted as necessarily requiring them to be performed in the specific order described or illustrated, unless the execution order is clearly indicated. It should also be understood that additional or alternative steps may be used.

[0208] The foregoing is merely a specific embodiment of the present invention, which enables those skilled in the art to understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A song recommendation method, characterized in that: The method comprises: Obtain the driver's current emotional state, current driving environment information, and the song emotion score of each song; Calculating a comprehensive emotional score of the driver according to the current emotional state and the current driving environment information; Calculating the matching degree of each song with the driver's emotional needs according to the song emotion score of each song and the emotional comprehensive score; According to the matching degree, songs are recommended.

2. The method according to claim 1, characterized in that: The current driving environment information includes the current time point and the current driving condition, and the calculating the driver's comprehensive emotion score according to the current emotional state and the current driving environment information includes: Determining a first score corresponding to the current emotional state according to a correspondence between the emotional state and the score; Determine the score corresponding to the time period at the current time point according to the correspondence between the time period and the score, and determine the score as the second score corresponding to the current time point; Determining a third score corresponding to the current driving condition according to a correspondence between the driving condition and the score; A comprehensive emotion score of the driver is calculated based on the first score, the second score, and the third score.

3. The method according to claim 2, characterized in that: The method further comprises: According to the correspondence between the time period and the emotional state, and the correspondence between the emotional state and the score, the correspondence between the time period and the score is obtained; According to the correspondence between the driving condition and the emotional state, and the correspondence between the emotional state and the score, the correspondence between the driving condition and the score is obtained.

4. The method according to claim 1, characterized in that: When the emotional comprehensive score is proportional to the emotional intensity, and the song emotional score is inversely proportional to the emotional intensity, the matching degree between each song and the driver's emotional needs is calculated based on the song emotional score and the emotional comprehensive score of each song, including: For the song emotion score of each song, the difference between the song emotion score and the comprehensive emotion score is calculated, and the matching degree between the song and the driver's emotional needs is determined based on the difference.

5. The method according to claim 1, characterized in that: The steps of generating the song emotion include: Get the audio data of the song; Determine the melody and rhythm features of the song according to the audio data; Merging the melody feature and the rhythm feature to obtain a fusion feature; The fusion feature is input into the trained emotion model to obtain the emotion of the song.

6. The method according to claim 5, characterized in that: Determining the melody feature of the song according to the audio data includes: Extracting pitch sequences and pitch contours from audio data; According to the preset rules, the audio data is divided to obtain at least one melody: According to the pitch sequence corresponding to each melody, determine the pitch span corresponding to each melody and the pitch difference between adjacent notes; In each melody, determine the melody corresponding to the pitch sequence whose similarity exceeds a threshold, and obtain a repeated melody; The pitch contour, repeated melody, the pitch span corresponding to each melody segment and the pitch difference between adjacent notes are identified as melodic features.

7. The method according to claim 5, characterized in that: Determining the melody feature of the song according to the audio data includes: Using a beat detection algorithm, identifying the audio data to obtain a strong beat position; Dividing the song according to the strong beat position to obtain at least one rhythm segment; Analyzing the changes in the strength of the beats in the rhythm segment to determine the rhythm pattern corresponding to the rhythm segment; A melodic feature of the song is determined based on the audio data, at least one rhythm section, and the rhythm pattern.

8. A song recommendation device, characterized in that: The device comprises: An acquisition unit, used to acquire the driver's current emotional state, current driving environment information, and a song emotion score of each song; A first calculation unit, configured to calculate a comprehensive emotion score of the driver according to the current emotional state and the current driving environment information; A second calculation unit is used to calculate the matching degree of each song with the driver's emotional needs according to the song emotion score of each song and the emotional comprehensive score; The recommendation unit is used to recommend songs according to the matching degree.

9. A song recommendation device, characterized in that: include: at least one communication interface; at least one bus connected to the at least one communication interface; at least one processor connected to the at least one bus; at least one memory connected to the at least one bus, wherein the processor is configured to: Obtain the driver's current emotional state, current driving environment information, and the song emotion score of each song; Calculating a comprehensive emotional score of the driver according to the current emotional state and the current driving environment information; Calculating the matching degree of each song with the driver's emotional needs according to the song emotion score of each song and the emotional comprehensive score; According to the matching degree, songs are recommended.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the song recommendation method according to any one of claims 1 to 7 is implemented.

Images