Music recommendation method and device

By acquiring and fusion of multimodal features, generating target audio sequences, and determining music content based on audio features, the problem of poor matching of music content in the on-board music recommendation system is solved, and more accurate and comprehensive music recommendation is achieved.

CN118916512AActive Publication Date: 2024-11-08AVATR CO LTD

Patent Information

Application Number
CN202410999685.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-24
Publication Date
2024-11-08
Estimated Expiration
2044-07-24

AI Technical Summary

Technical Problem

The existing car music recommendation system is difficult to achieve accurate matching and recall of music content, resulting in poor recommendation results, mainly due to the objective differences in the expression of text and music.

Method used

By acquiring multimodal features, performing feature fusion, generating target audio sequences, determining music content based on audio features, forming a recommendation list, and making up for the differences in expressions between multimodal features and music content.

Benefits of technology

It improves the accuracy and comprehensiveness of music recommendations, achieves accurate matching and recall of music content, and improves recommendation effect.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118916512A_ABST
    Figure CN118916512A_ABST
Patent Text Reader

Abstract

The invention discloses a music recommendation method and device. The method comprises the steps of obtaining at least one type of modal features used for performing music recommendation on a target user; each type of modal features in the at least one type of modal features is obtained by performing feature extraction on a type of modal data; performing feature fusion on the at least one type of modal features to obtain fused modal features; generating a target audio sequence for representing a music recommendation demand of the target user based on the fusion modal feature; determining at least one target music content at least based on target audio features of the target audio sequence to form a music recommendation list; and outputting the music recommendation list. According to the scheme, the difference between the multi-modal features and the music content in expression modes can be effectively made up, the accuracy and comprehensiveness of music recommendation are improved, and then the music recommendation effect is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of data processing, and relates to but is not limited to a music recommendation method and device. Background Art

[0002] As the level of automobile intelligence continues to improve, most cars are equipped with in-car music recommendation systems. In-car music recommendation systems are designed to provide users with personalized music content to meet their music listening needs during travel. At present, in-car music recommendation systems are still in an immature stage and it is difficult to provide users with accurate and satisfactory music content.

[0003] In the related art, when making music recommendations, music recall strategies based on text features such as keywords, tags, and themes are usually used to match and recall music content. However, due to the objective differences in the expression of text and music, the above schemes cannot achieve accurate matching and recall of music content, resulting in poor music recommendation results. Summary of the invention

[0004] The present application provides a music recommendation method, apparatus, device, storage medium and computer program product, which can effectively make up for the difference in expression between multimodal features and music content, improve the accuracy and comprehensiveness of music recommendation, and thus improve the effect of music recommendation.

[0005] The technical solution of this application is implemented as follows:

[0006] In a first aspect, the present application provides a music recommendation method, the music recommendation method comprising:

[0007] Acquire at least one type of modal features for recommending music to a target user; each type of modal features in the at least one type of modal features is obtained by extracting features from a type of modal data;

[0008] Performing feature fusion on the at least one type of modal features to obtain fused modal features;

[0009] Based on the fused modal features, generating a target audio sequence for representing the music recommendation needs of the target user;

[0010] determining at least one target music content based at least on the target audio feature of the target audio sequence to form a music recommendation list;

[0011] Output the music recommendation list.

[0012] In a second aspect, the present application provides a music recommendation device, the device comprising:

[0013] An acquisition unit, used for acquiring at least one type of modal features for making music recommendations to a target user; each type of modal features in the at least one type of modal features is obtained by extracting features from a type of modal data;

[0014] A fusion unit, used for fusing the at least one type of modal features to obtain fused modal features;

[0015] A generating unit, configured to generate a target audio sequence for representing the music recommendation demand of the target user based on the fused modal features;

[0016] a determination unit, configured to determine at least one target music content based at least on the target audio feature of the target audio sequence, so as to form a music recommendation list;

[0017] An output unit is used to output the music recommendation list.

[0018] In a third aspect, the present application further provides an electronic device comprising at least a memory and a processor, wherein the memory stores a computer program executable on the processor, and the processor implements the above-mentioned music recommendation method when executing the program.

[0019] In a fourth aspect, the present application further provides a storage medium having a computer program stored thereon, and when the computer program on the storage medium is executed, the above-mentioned music recommendation method is implemented.

[0020] In a fifth aspect, the present application also provides a computer program product, including a computer program or instructions, which, when executed by a processor, implements the above-mentioned music recommendation method.

[0021] The present application provides a music recommendation method, apparatus, device, storage medium and computer program product, wherein the music recommendation method comprises: obtaining at least one type of modal feature for making music recommendations to a target user; each type of the modal feature in the at least one type of modal feature is obtained after feature extraction of a type of modal data; performing feature fusion on the at least one type of modal feature to obtain fused modal features; based on the fused modal features, generating a target audio sequence for characterizing the music recommendation needs of the target user; determining at least one target music content based at least on the target audio features of the target audio sequence to form a music recommendation list; and outputting the music recommendation list.

[0022] In the scheme of the present application, by generating a target audio sequence for characterizing the user's music recommendation needs, it is possible to determine at least one target music content based on the target audio features of the target audio sequence, and then form and output a music recommendation list based on at least one target music content. It can be seen that the scheme of the present application fully takes into account that audio features pay more attention to the expression of rhythm, melody, harmony, timbre, etc., and can accurately correspond to music content with the help of audio features, and music content can also be fully and accurately characterized by audio features. Therefore, by generating a target audio sequence for characterizing the user's music recommendation needs, and determining at least one target music content based on the target audio features of the target audio sequence, accurate matching and recall of music content can be achieved. In this way, the difference in expression between multimodal features and music content can be effectively compensated, the accuracy and comprehensiveness of music recommendations can be improved, and the effect of music recommendations can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 A first optional flow chart of the music recommendation method provided in the embodiment of the present application;

[0024] Figure 2 A second optional flow chart of the music recommendation method provided in the embodiment of the present application;

[0025] Figure 3 A third optional flow chart of the music recommendation method provided in the embodiment of the present application;

[0026] Figure 4 A fourth optional flow chart of the music recommendation method provided in the embodiment of the present application;

[0027] Figure 5 A fifth optional flow chart of the music recommendation method provided in the embodiment of the present application;

[0028] Figure 6 A sixth optional flow chart of the music recommendation method provided in the embodiment of the present application;

[0029] Figure 7 A seventh optional flow chart of the music recommendation method provided in the embodiment of the present application;

[0030] Figure 8 This is a schematic diagram of an eighth optional flow chart of the music recommendation method provided in the embodiment of the present application;

[0031] Fig. 9 A ninth optional flow chart of the music recommendation method provided in the embodiment of the present application;

[0032] Fig.10An optional schematic diagram of the overall business module of the music recommendation method provided in the embodiment of the present application;

[0033] Fig.11 An optional schematic diagram of the system architecture of the music recommendation method provided in the embodiment of the present application;

[0034] Fig.12 An optional schematic diagram of a recall content balancing solution for the music recommendation method provided in an embodiment of the present application;

[0035] Fig.13 An optional schematic diagram of a time penalty function curve of the music recommendation method provided in an embodiment of the present application;

[0036] Fig.14 An optional schematic diagram of a large model algorithm flow chart of the music recommendation method provided in an embodiment of the present application;

[0037] Fig.15 A schematic diagram of an optional structure of a music recommendation device provided in an embodiment of the present application;

[0038] Fig.16 An optional structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0039] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the specific technical solution of the application will be further described in detail below in conjunction with the drawings in the embodiments of the present application. The following embodiments are used to illustrate the present application, but are not used to limit the scope of the present application.

[0040] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0041] In the following description, the terms "first\second\third" are used only as examples to distinguish different objects, and do not represent a specific order for the objects, nor do they have a limitation on the order of precedence. It is understandable that "first\second\third" can be interchanged with a specific order or order of precedence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0042] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0043] The embodiments of the present application provide a music recommendation method, device, equipment, storage medium and computer program product. In practical applications, the music recommendation method can be implemented by a music recommendation device, and each functional entity in the music recommendation device can be collaboratively implemented by hardware resources of the electronic device, such as computing resources such as a processor, and communication resources (such as for supporting various communication methods such as optical cables and cellular).

[0044] Below, various embodiments of the music recommendation method, device, equipment, storage medium and computer program product provided in the embodiments of the present application are described.

[0045] In the first aspect, the embodiment of the present application provides a music recommendation method. The function implemented by the method can be implemented by calling a program code by a processor in an electronic device (such as a terminal device or a controller), and of course the program code can be stored in a computer storage medium. It can be seen that the electronic device at least includes a processor and a storage medium.

[0046] The following uses an electronic device as an example of an execution subject to illustrate the music recommendation method provided in the embodiment of the present application.

[0047] Figure 1 Schematic diagram of the music recommendation method of the present application embodiment. Figure 1 As shown, the process may include but is not limited to the following S101 to S105.

[0048] S101. The electronic device obtains at least one type of modal features for making music recommendations to a target user.

[0049] Each of the at least one type of modal features is obtained after feature extraction of a type of modal data.

[0050] The target user may be any user who has a music recommendation demand. The embodiment of the present application does not limit the type and number of target users, and the target users may be set according to actual conditions.

[0051] Modal data refers to data used to recommend music to a target user, and can be any data related to the target user. The embodiment of the present application does not limit the type and quantity of modal data, and can be set according to actual conditions. Exemplarily, the modal data can be text type, audio type, image type, video type, etc.

[0052] In a possible implementation, the modal data includes at least user data, environment data, and vehicle data. Among them: user data can be any parameters related to the target user himself, for example, the user's basic data (such as age, gender, education, occupation, driving style, ride duration, etc.), the user's behavioral data (such as movements, expressions, voice, activity, interaction methods, etc.), the user's music preference data (such as listening habits, music preferences, playlists, collection preferences, etc.), the user's historical music data (such as historical listening records, historical music evaluation feedback records, historical music sharing records, historical music search records, historical music browsing records, etc.); environmental data can be any data related to the target user's current environment, such as time, weather, temperature, humidity, light, air quality, ambient sound, location, road type, number of pedestrians, congestion, other vehicle behavior, etc.; vehicle data can be any data related to the vehicle the target user is riding in, for example, the vehicle's driving data (such as speed, location, battery level, mileage, gear, driving mode, etc.), the vehicle's status data (such as the status of each component, ambient light, whether it enters a geographic fence, etc.), and the vehicle's configuration data (such as vehicle model, vehicle layout, drive mode, etc.).

[0053] In practice, modal data may also include other data related to the target user, and these data may be used as a basis for making music recommendations to the target user.

[0054] Modal features refer to features obtained after feature extraction of modal data. The present application embodiment does not limit the type and quantity of modal features, and can be set according to actual conditions. It can be understood that modal features correspond to modal data.

[0055] Exemplarily, at least one type of modal data for music recommendation to target users can be collected or collected in advance through specific sensors and specific application program interfaces and stored in the electronic device. Correspondingly, S101 can be implemented as follows: the electronic device obtains at least one type of modal data from the local or cloud; traverses at least one type of modal data; for each type of modal data, a feature extraction algorithm or a feature extraction model is used to extract features from the modal data to obtain a type of modal feature; after the traversal is completed, at least one type of modal feature is obtained.

[0056] The embodiments of the present application do not limit the specific content or type of the feature extraction algorithm or feature extraction model, which can be set according to actual conditions.

[0057] S102: The electronic device performs feature fusion on the at least one type of modal features to obtain fused modal features.

[0058] The fused modal feature refers to a feature obtained by fusion of at least one type of modal feature. The embodiment of the present application does not limit the specific content of the fused modal feature, and it can be set according to actual conditions.

[0059] Exemplarily, S102 may be implemented as follows: the electronic device uses a feature fusion algorithm or a feature fusion model to perform feature fusion on at least one type of modal features; after the feature fusion is completed, a fused modal feature is obtained.

[0060] The embodiments of the present application do not limit the specific content or type of the feature fusion algorithm or feature fusion model, and can be set according to actual conditions. For example, the feature fusion model can be a Transformer model, which is a deep learning model based on the attention mechanism, has powerful language understanding and generation capabilities, can focus on and associate different parts of the input features, and effectively integrate multiple different types of features, thereby achieving more comprehensive and accurate information expression.

[0061] By fusing at least one type of modal features, we can make up for the shortcomings and limitations of a single modal feature, effectively integrate modal features from different sources and types, and explore the potential correlations and synergies hidden between different modal features, thereby obtaining a more comprehensive and integrated information expression.

[0062] S103: The electronic device generates a target audio sequence for representing the music recommendation needs of the target user based on the fused modal features.

[0063] An audio sequence refers to a set of discrete elements or an audio information stream composed of multiple audio data units arranged in a specific order. The audio data unit can be an audio sampling point, an audio frame, an audio segment, or a complete audio file.

[0064] The specific type and content of the audio sequence are not limited in the present application embodiment and can be set according to the actual situation. For example, the audio sequence can be a discrete digital sequence formed by digitally sampling and encoding the audio signal, such as an MP3 file, or it can be an analog audio sequence with continuity and logic containing the original sound wave signal.

[0065] The target audio sequence refers to an audio sequence used to characterize the music recommendation needs of the target user. The specific content of the target audio sequence is not limited in the present embodiment of the application, and can be set according to actual conditions.

[0066] Exemplarily, S103 may be implemented as follows: the electronic device predicts the music recommendation needs of the target user based on the fused modal features; and based on the music recommendation needs of the target user, generates a target audio sequence that meets the music recommendation needs of the target user.

[0067] The embodiment of the present application does not limit the specific manner in which the electronic device predicts the target user's music recommendation needs based on the fusion modal features, and can be set according to actual conditions.

[0068] In a possible implementation, the electronic device may use a rule-based demand prediction method to determine the target user's music recommendation needs. For example, in the case where Rule 1 includes the user's preference for light music, if the fused modal feature indicates that the target user likes light music, then the electronic device determines that the fused modal feature satisfies Rule 1. At this time, the target user's music recommendation needs may be music recommendation needs with beautiful melody, soothing rhythm, and ethereal timbre; in the case where Rule 2 includes the user's fatigue driving, if the fused modal feature indicates that the target user is in fatigue driving, then the electronic device determines that the fused modal feature satisfies Rule 2. At this time, the target user's music recommendation needs may be music recommendation needs with exciting melody, compact rhythm, and wild timbre.

[0069] In another possible implementation, the electronic device may use a model-based demand prediction method to determine the music recommendation needs of the target user. For example, in the case where the rules include the user's preference for light music and the user's fatigue driving, if the fused modal features indicate that the target user has no specific music preferences, the target user is in a non-fatigue driving state, and the target user opens a music application, then the electronic device determines that the fused modal features do not meet the rules. At this time, the model may be used to process the fused modal features, and the music recommendation needs output by the model are used as the music recommendation needs of the target user.

[0070] Simply put, the rule-based demand prediction method is applicable to the situation where the fusion modal features meet the pre-defined clear rules. In this case, the target user has a clear demand for music recommendation. At this time, the rule-based demand prediction method is used to determine the target user's music recommendation needs, and the implementation plan is simple, convenient and not prone to errors; the model-based demand prediction method is applicable to the situation where the fusion modal features do not meet the pre-defined clear rules. In this case, the target user does not have a clear demand for music recommendation. At this time, the model-based demand prediction method is used to determine the target user's music recommendation needs, and the implementation plan is accurate and efficient with a wide range of applications.

[0071] S104: The electronic device determines at least one target music content based at least on the target audio features of the target audio sequence to form a music recommendation list.

[0072] Audio features refer to quantitative indicators or attributes used to describe the essential characteristics of audio signals.

[0073] The target audio feature refers to the audio feature of the target audio sequence. The specific type and quantity of the target audio feature are not limited in the embodiment of the present application, and can be set according to the actual situation. For example, the target audio feature may include rhythm features, melody features, timbre features, pitch features, harmony features, music emotion features, music style features, time domain features, frequency domain features, energy features, etc.

[0074] The target music content refers to the music content recommended to the target user. The specific type and quantity of the target music content are not limited in the present embodiment of the application and can be set according to the actual situation.

[0075] The music recommendation list refers to a list formed by all target music contents. The specific form of the music recommendation list is not limited in the embodiment of the present application, and can be set according to actual conditions. For example, the music recommendation list can be in a table form, including music name, music type, music introduction, author introduction, music duration, detailed content, etc.

[0076] In a possible implementation, S104 can be implemented as follows: the electronic device matches the target audio sequence with each music content included in the music library based on the target audio features of the target audio sequence; determines at least one target music content based on the degree of matching between the target audio sequence and each music content included in the music library; and sorts the at least one target music content to form a music recommendation list.

[0077] In another possible implementation, S104 may be implemented as follows: the electronic device matches the target audio sequence with each music content included in the music library based on the target audio feature and the target audio change feature of the target audio sequence; determines at least one target music content based on the matching degree between the target audio sequence and each music content included in the music library; and sorts the at least one target music content to form a music recommendation list. The target audio change feature refers to the change trend or direction of the target audio sequence over time.

[0078] S105: The electronic device outputs the music recommendation list.

[0079] Exemplarily, S105 can be implemented as follows: the electronic device displays the music recommendation list in the form of text, tables, charts, etc. on the display interface of the display screen, or the electronic device calls a voice assistant to display and broadcast the music recommendation list.

[0080] The present application provides a music recommendation method, which includes: obtaining at least one type of modal feature for making music recommendations to a target user; each type of the modal feature in the at least one type of modal feature is obtained after feature extraction of a type of modal data; performing feature fusion on the at least one type of modal feature to obtain fused modal features; based on the fused modal features, generating a target audio sequence for characterizing the music recommendation needs of the target user; determining at least one target music content based at least on the target audio features of the target audio sequence to form a music recommendation list; and outputting the music recommendation list.

[0081] In the scheme of the present application, by generating a target audio sequence for characterizing the user's music recommendation needs, it is possible to determine at least one target music content based on the target audio features of the target audio sequence, and then form and output a music recommendation list based on at least one target music content. It can be seen that the scheme of the present application fully takes into account that audio features pay more attention to the expression of rhythm, melody, harmony, timbre, etc., and can accurately correspond to music content with the help of audio features, and music content can also be fully and accurately characterized by audio features. Therefore, by generating a target audio sequence for characterizing the user's music recommendation needs, and determining at least one target music content based on the target audio features of the target audio sequence, accurate matching and recall of music content can be achieved. In this way, the difference in expression between multimodal features and music content can be effectively compensated, the accuracy and comprehensiveness of music recommendations can be improved, and the effect of music recommendations can be improved.

[0082] Next, the process of generating a target audio sequence for representing the music recommendation demand of the target user based on the fused modal features by the electronic device in S103 is described.

[0083] refer to Figure 2 As shown in the content, the process may include but is not limited to the following S201 and S202.

[0084] S201. The electronic device processes the fused modal features using a first audio model to predict a music recommendation type for the target user.

[0085] The music recommendation type includes at least one of the following: melody type, rhythm type, emotion type and music style type.

[0086] The first audio model is an inference-based large model.

[0087] The music recommendation type refers to the music type used to characterize the music recommendation needs of the target users, including at least melody type, rhythm type, emotion type and music style type.

[0088] The embodiments of the present application do not limit the specific contents of melody type, rhythm type, emotion type and music style type, which can be set according to actual conditions. For example, melody types may include: beautiful and smooth, ups and downs, unique and novel, etc.; rhythm types may include: bright and active, calm and powerful, changeable and interesting, etc.; emotion types may include: happy and joyful, warm and touching, exciting and passionate, calm and soothing, etc.; music style types may include: classical elegance, popular fashion, rock passion, jazz freedom, folk simplicity, electronic dynamics, world diversity, etc.

[0089] The first audio model is used to determine and output the music recommendation type based on the fusion modal features. The embodiment of the present application does not limit the specific type of the first audio model, and it can be set according to actual conditions.

[0090] It is understandable that the first audio model can be any pre-trained large model with reasoning capabilities. The first audio model has reasoning capabilities, which means that the first audio model can analyze and understand input information, refer to deep learning, intelligent reasoning, knowledge fusion and other technologies, combine the language rules, logical rules and massive knowledge learned in the pre-training stage, analyze and deduce through semantic analysis and pattern recognition, and finally obtain and output conclusions.

[0091] In advance, multimodal data related to different users can be collected or acquired, such as user data of different groups of people, environmental data of different environments, and vehicle data of different vehicles; feature extraction is performed on the multimodal data to obtain multimodal features; feature fusion is performed on the multimodal features to obtain fused modal features; the initial model is trained based on the fused modal features and related prompt words, and the first audio model is obtained after the training is completed.

[0092] The embodiment of the present application does not limit the specific training process of the first audio model, which can be set according to actual conditions.

[0093] Exemplarily, S201 can be implemented as follows: the electronic device inputs the fusion modal features into a pre-trained first audio model; the first audio model uses its reasoning ability to calculate and process the input fusion modal features, predicts and outputs the music recommendation type corresponding to the fusion modal features; the electronic device determines the music recommendation type as the music recommendation type for the target user.

[0094] S202: The electronic device processes the music recommendation type using a second audio model to generate a target audio sequence that matches the music recommendation type.

[0095] The second audio model is a generative large model.

[0096] The second audio model is used to generate and output an audio sequence that matches the music recommendation type based on the music recommendation type. The embodiment of the present application does not limit the specific type of the second audio model, and it can be set according to actual conditions.

[0097] It is understandable that the second audio model can be any pre-trained large model with generation capability. The second audio model has generation capability, which means that the second audio model can independently create and generate content with certain logic and rationality by analyzing and understanding input information, combining the language rules, logical rules and massive knowledge learned in the pre-training stage.

[0098] In advance, multiple groups of music recommendation types can be obtained; the initial model is trained based on the multiple groups of music recommendation types and related prompt words, and the second audio model is obtained after the training is completed.

[0099] The embodiment of the present application does not limit the specific training process of the second audio model, which can be set according to actual conditions.

[0100] Exemplarily, S202 can be implemented as follows: the electronic device inputs the music recommendation type into a pre-trained second audio model; the second audio model uses its generation capability to calculate and process the input music recommendation type, generates and outputs an audio sequence that is consistent with the music recommendation type; and the electronic device determines the audio sequence as a target audio sequence.

[0101] It should be noted that the first audio model and the second audio model may be the same or different.

[0102] In the case where the first audio model is the same as the second audio model, the first audio model and the second audio model can be collectively referred to as an audio model, and the audio model has both reasoning and generation capabilities. Correspondingly, S201 and S202 can be implemented together as follows: the electronic device inputs the fusion modal features into a pre-trained audio model; the audio model uses its reasoning and generation capabilities to calculate and process the input fusion modal features, generates and outputs an audio sequence for characterizing the music recommendation needs of the target user; the electronic device determines the audio sequence as a target audio sequence.

[0103] Simply put, by introducing a pre-trained first audio model with reasoning capabilities to calculate and process the fused modal features and then predict the music recommendation type for the target user, accurate reasoning of the music recommendation type can be achieved, ensuring that the predicted music recommendation type is more in line with the potential interests of the target user; by introducing a pre-trained second audio model with generation capabilities to calculate and process the music recommendation type and then generate a target audio sequence that is consistent with the music recommendation type, the accuracy and comprehensiveness of the target audio sequence can be improved, making the generated target audio sequence more consistent with the music recommendation type for the target user. The implementation scheme is simple, efficient and highly scalable.

[0104] Next, the process in S104 in which the electronic device determines at least one target music content based on at least the target audio feature of the target audio sequence to form a music recommendation list is described.

[0105] refer to Figure 3 As shown in the content, the process may include but is not limited to the following S301 to S304.

[0106] S301. The electronic device determines at least one candidate music content in a music library based on the target audio feature; or determines at least one candidate music content in the music library based on the target audio feature and the target audio variation feature of the target audio sequence.

[0107] Candidate music content refers to music content that has not been screened and sorted and is initially determined to be recommended to the target user. The embodiment of the present application does not limit the specific type and quantity of the candidate music content, which can be set according to actual conditions.

[0108] Audio change characteristics refer to the temporal change trend or direction of the audio signal.

[0109] The target audio change feature refers to the audio change feature of the target audio sequence. The embodiment of the present application does not limit the specific type and quantity of the target audio change feature, which can be set according to the actual situation. Exemplarily, the target audio change feature may include rhythm change feature, melody change feature, timbre change feature, pitch change feature, harmony change feature, music emotion change feature, music style change feature, time domain change feature, frequency domain change feature, energy change feature, etc.

[0110] In a possible implementation, S301 can be implemented as follows: the electronic device uses an audio feature extraction tool to extract audio features from a target audio sequence to obtain target audio features; matches the target audio features with the audio features of each music content included in the music library; and determines at least one candidate music content based on the degree of match between the target audio features and the audio features of each music content included in the music library.

[0111] In another possible implementation, S301 can be implemented as follows: the electronic device uses an audio feature extraction tool to extract audio features from a target audio sequence to obtain target audio features, and uses an audio change feature analysis tool to process and analyze the target audio sequence to obtain target audio change features; matches the target audio features with the audio features of each music content included in the music library, and matches the target audio change features with the audio change features of each music content included in the music library; and determines at least one candidate music content based on the degree of matching between the target audio features and the audio features of each music content included in the music library, and the degree of matching between the target audio change features and the audio change features of each music content included in the music library.

[0112] The embodiments of the present application do not limit the specific types of audio feature extraction tools and audio change feature analysis tools, which can be set according to actual conditions. For example, audio feature extraction tools can include software such as Praat, Audacity, and MATLAB; audio change feature analysis tools can include software such as Sonic Visualiser and FL Studio, and can also include pre-trained artificial intelligence models with audio change feature analysis capabilities.

[0113] S302: The electronic device determines the at least one target music content from the at least one candidate music content.

[0114] Exemplarily, S302 may be implemented as follows: the electronic device screens at least one candidate music content based on a pre-set screening rule to obtain at least one target music content.

[0115] The embodiments of the present application do not limit the specific content and number of the screening rules, which can be set according to actual conditions. Exemplarily, the screening rules may include at least one of the following: representative content screening, similar content deduplication, listened content filtering, sensitive word filtering, label filtering, and sentiment score filtering. Among them: representative content screening refers to screening out representative high-value music content to improve the quality of music recommendation; similar content deduplication refers to retaining one of the multiple music contents with a similarity score greater than a threshold and deleting other music contents to avoid redundancy; listened content filtering refers to deleting music content that the user has already listened to to avoid repeated recommendations; sensitive word filtering refers to deleting music content containing specific sensitive words to avoid the spread of negative music content; label filtering refers to deleting music content containing specific labels to avoid the spread of negative music content; sentiment score filtering refers to deleting music content with a negative sentiment score to avoid the spread of negative music content.

[0116] S303: The electronic device sorts the at least one target music content to obtain the music recommendation list.

[0117] Exemplarily, S303 may be implemented as follows: the electronic device sorts at least one target music content based on a pre-set sorting rule to obtain a music recommendation list.

[0118] The embodiments of the present application do not limit the specific content and number of the sorting rules, which can be set according to actual conditions. For example, the sorting rules may include at least one of the following: matching degree sorting, timeliness sorting, and popularity sorting. Among them: matching degree sorting refers to a method of sorting according to the degree of matching; timeliness sorting refers to a method of sorting according to the degree of closeness to the current hot events; popularity sorting refers to a method of sorting according to the degree of popularity.

[0119] In simple terms, by determining the target audio features and target audio change features of the target audio sequence, the target audio sequence can be accurately matched with the music content included in the music library, thereby improving the accuracy and comprehensiveness of the candidate music content. In addition, by screening the candidate music content and sorting the target music content, more accurate and comprehensive high-value music content can be provided to users, thereby improving the accuracy and rationality of music recommendations.

[0120] Next, the process of determining the at least one candidate music content in the music library based on the target audio feature and the target audio change feature of the target audio sequence by the electronic device in S301 is described.

[0121] refer to Figure 4 The method may include but is not limited to the following S401 and S402.

[0122] For each of the at least one first music content included in the music library, the following processing is performed to obtain the at least one candidate music content:

[0123] S401. The electronic device determines an audio feature matching degree between the target audio feature and a first audio feature of the first music content, and an audio change feature matching degree between the target audio change feature and a first audio change feature of the first music content.

[0124] The first music content refers to the music content included in the music library. The embodiment of the present application does not limit the type and quantity of the first music content, which can be determined according to actual conditions.

[0125] The first audio feature refers to the audio feature of the first music content. For example, similar to the target audio feature, the first audio feature may also include rhythm feature, melody feature, timbre feature, pitch feature, harmony feature, music emotion feature, music style feature, time domain feature, frequency domain feature, energy feature, etc.

[0126] The audio feature matching degree refers to the matching degree between the target audio feature and the first audio feature. The embodiment of the present application does not limit the specific form of the audio feature matching degree, which can be determined according to actual conditions. For example, the audio feature matching degree can be expressed in the form of a score or a percentage.

[0127] The first audio change feature refers to the audio change feature of the first music content. Exemplarily, similar to the target audio change feature, the first audio change feature may also include rhythm change feature, melody change feature, timbre change feature, pitch change feature, harmony change feature, music emotion change feature, music style change feature, time domain change feature, frequency domain change feature, energy change feature, etc.

[0128] The audio change feature matching degree refers to the matching degree between the target audio change feature and the first audio change feature. The embodiment of the present application does not limit the specific form of the audio change feature matching degree, which can be determined according to actual conditions. For example, the audio change feature matching degree can be expressed in the form of a score or a percentage.

[0129] Exemplarily, S401 can be implemented as follows: the electronic device obtains the audio feature matching degree between the target audio feature and the first audio feature by calculating the distance value or cosine similarity value between the target audio feature and the first audio feature; and obtains the audio change feature matching degree between the target audio change feature and the first audio change feature by calculating the distance value or cosine similarity value between the target audio change feature and the first audio change feature.

[0130] S402: When the audio feature matching degree is greater than a first threshold and the audio change feature matching degree is greater than a second threshold, the electronic device determines the first music content as one of the candidate music contents.

[0131] The first threshold and the second threshold are used to assist in determining the candidate music content. The specific values ​​of the first threshold and the second threshold are not limited in the present embodiment of the application and can be determined according to actual conditions.

[0132] Exemplarily, S402 can be implemented as: the electronic device determines the size relationship between the audio feature matching degree and the first threshold, and the size relationship between the audio change feature matching degree and the second threshold; when the audio feature matching degree is greater than the first threshold, and the audio change feature matching degree is greater than the second threshold, the first music content is determined as a candidate music content.

[0133] Simply put, by determining the audio feature matching degree between the target audio feature and the first audio feature of each first music content in the music library, as well as the audio change feature matching degree between the target audio change feature and the first audio change feature of each first music content in the music library, it is possible to achieve precise matching of the target audio feature with the first audio feature, as well as the target audio change feature with the first audio change feature, thereby achieving precise matching of the target audio sequence with the first music content, improving the accuracy and comprehensiveness of the candidate music content, and making the implementation scheme simple and efficient.

[0134] It should be noted that in S301, the electronic device determines at least one candidate music content in the music library based on the target audio feature, including: for each of the at least one first music content included in the music library, performing the following processing to obtain the at least one candidate music content: determining the audio feature matching degree between the target audio feature and the first audio feature of the first music content; when the audio feature matching degree is greater than a first threshold, determining the first music content as one of the candidate music contents.

[0135] Next, the process of the electronic device determining the audio feature matching degree between the target audio feature and the first audio feature of the first music content in S401 is described.

[0136] In the case where the target audio feature includes a target rhythm feature, a target melody feature, and a target timbre feature, and the first audio feature includes a first rhythm feature, a first melody feature, and a first timbre feature, reference Figure 5 As shown in the content, the process may include but is not limited to the following S501 to S503.

[0137] The target rhythm feature, target melody feature and target timbre feature refer to the rhythm feature, melody feature and timbre feature of the target audio feature; the first rhythm feature, first melody feature and first timbre feature refer to the rhythm feature, melody feature and timbre feature of the first audio feature.

[0138] In practice, the target audio feature and the first audio feature may also include other features, which are not listed here one by one.

[0139] S501: The electronic device determines a rhythm matching degree between the target rhythm feature and the first rhythm feature, a melody matching degree between the target melody feature and the first melody feature, and a timbre matching degree between the target timbre feature and the first timbre feature.

[0140] Rhythm matching refers to the matching degree between the target rhythm feature and the first rhythm feature.

[0141] Melody matching refers to the matching degree between the target melody feature and the first melody feature.

[0142] The timbre matching degree refers to the matching degree between the target timbre feature and the first timbre feature.

[0143] The embodiments of the present application do not limit the specific forms of expression of rhythm matching, melody matching, and timbre matching, which can be set according to actual conditions.

[0144] Exemplarily, S501 can be implemented as follows: the electronic device calculates the distance value or cosine similarity value between the target rhythm feature and the first rhythm feature to obtain a rhythm match; calculates the distance value or cosine similarity value between the target melody feature and the first melody feature to obtain a melody match; calculates the distance value or cosine similarity value between the target timbre feature and the first timbre feature to obtain a timbre match.

[0145] S502: The electronic device determines a first weight corresponding to the rhythm matching degree, a second weight corresponding to the melody matching degree, and a third weight corresponding to the timbre matching degree.

[0146] The first weight, the second weight, and the third weight refer to the weights corresponding to the rhythm matching degree, the melody matching degree, and the timbre matching degree, respectively. The specific values ​​of the first weight, the second weight, and the third weight are not limited in the present embodiment of the application, and can be set according to actual conditions.

[0147] For example, the first weight corresponding to the rhythm matching degree, the second weight corresponding to the melody matching degree, and the third weight corresponding to the timbre matching degree can be determined in advance based on human experience or a specific algorithm or a specific model, and stored in the electronic device. Correspondingly, S502 can be implemented as follows: the electronic device obtains the first weight, the second weight, and the third weight from a local device or a cloud.

[0148] S503: The electronic device performs weighted averaging on the rhythm matching degree, the melody matching degree and the timbre matching degree based on the first weight, the second weight and the third weight to obtain the audio feature matching degree.

[0149] Exemplarily, S503 may be implemented as follows: the electronic device calculates the product of the rhythm matching degree and the first weight, the product of the melody matching degree and the second weight, and the product of the timbre matching degree and the third weight; and determines the sum of these three products as the audio feature matching degree.

[0150] Simply put, by determining the rhythm matching degree between the target rhythm feature and the first rhythm feature, the melody matching degree between the target melody feature and the first melody feature, the timbre matching degree between the target timbre feature and the first timbre feature, and the first weight corresponding to the rhythm matching degree, the second weight corresponding to the melody matching degree, and the third weight corresponding to the timbre matching degree, the audio feature matching degree can be accurately determined, taking into account the matching of the target audio feature and the first audio feature in terms of rhythm, melody, timbre, etc., to achieve precise matching between the target audio feature and the first audio feature.

[0151] It should be noted that, when the target audio change feature includes a target rhythm change feature, a target melody change feature and a target timbre change feature, and the first audio change feature includes a first rhythm change feature, a first melody change feature and a first timbre change feature, the electronic device in S401 determines the audio change feature matching degree between the target audio change feature and the first audio change feature of the first music content, including: respectively determining the rhythm change feature matching degree between the target rhythm change feature and the first rhythm change feature, the melody change feature matching degree between the target melody change feature and the first melody change feature, and the timbre change feature matching degree between the target timbre change feature and the first timbre change feature; respectively determining a fourth weight corresponding to the rhythm change feature matching degree, a fifth weight corresponding to the melody change feature matching degree, and a sixth weight corresponding to the timbre change feature matching degree; based on the fourth weight, the fifth weight and the sixth weight, performing a weighted average of the rhythm change feature matching degree, the melody change feature matching degree and the timbre change feature matching degree to obtain the audio change feature matching degree.

[0152] Next, the process of the electronic device determining the at least one target music content from the at least one candidate music content in S302 is described.

[0153] refer to Figure 6 As shown in the content, the process may include but is not limited to the following S601 and S602.

[0154] S601: The electronic device determines a representative identifier of each of the at least one candidate music content.

[0155] The representative identifier is used to indicate whether the candidate music content is representative in the music genre to which it belongs.

[0156] The embodiments of the present application do not limit the specific content of the music type, which can be determined according to actual conditions. For example, in the case of dividing the music content by style, the music type may include pop, rock, classical, jazz, folk, electronic, hip-hop, metal, blues, etc.; in the case of dividing the music content by regional culture, the music type may include Chinese style music, European and American music, African music, Latin music, etc.; in the case of dividing the music content by historical period, the music type may include Baroque music, classical music, romantic music, modern music, etc.; in the case of dividing the music content by performance form, the music type may include solo, chorus, symphony, chamber music, etc.; in the case of dividing the music content by usage scenario, the music type may include dance music, film and television soundtrack, game music, etc.; in the case of dividing the music content by emotional expression, the music type may include cheerful music, sad music, passionate music, etc.

[0157] The specific content of the representative identifier is not limited in the present application embodiment and can be determined according to actual conditions. For example, the representative identifier can be "0" or "1", where "0" indicates not representative and "1" indicates representative.

[0158] For example, when the music genre is "pop music", if the candidate music content is "Happy Birthday to Me", then the representative identifier of "Happy Birthday to Me" can be "1", indicating that "Happy Birthday to Me" is representative in the music genre "pop music"; if the candidate music content is the classic song "Happy Birthday", then the representative identifier of "Happy Birthday" can be "0", indicating that "Happy Birthday" is not representative in the music genre "pop music".

[0159] Exemplarily, a representative identifier of each candidate music content in at least one candidate music content may be determined in advance based on human experience or a specific algorithm or a specific model and stored in the electronic device. Correspondingly, S601 may be implemented as follows: the electronic device obtains a representative identifier of each candidate music content in at least one candidate music content from a local device or a cloud device.

[0160] S602: The electronic device determines, among the at least one candidate music content, a candidate music content that is representatively identified as a first value as the at least one target music content.

[0161] The first value indicates that the candidate music content is representative in the corresponding music genre.

[0162] The first value is used to assist in determining the target music content. The specific content of the first value is not limited in the present embodiment of the application, and can be set according to actual conditions. For example, the first value can be "1", and "1" indicates representativeness.

[0163] Exemplarily, S602 can be implemented as follows: the electronic device traverses at least one candidate music content; for each candidate music content, determines whether the representative identifier of the candidate music content is the first value, and if so, determines the candidate music content as a target music content; if not, continues to determine whether the representative identifier of the next candidate music content is the first value until the traversal is completed; after the traversal is completed, obtains at least one target music content.

[0164] In short, by determining the representative identifier of each candidate music content, at least one representative target music content can be screened out from at least one candidate music content, so that high-value music content can be provided to users, effectively improving the quality and accuracy of music recommendations.

[0165] Next, the process of the electronic device sorting the at least one target music content to obtain the music recommendation list in S303 is described.

[0166] refer to Figure 7 As shown in the content, the process may include but is not limited to the following S701 and S702.

[0167] S701: The electronic device determines a timeliness score of each of the at least one target music content.

[0168] The timeliness score is used to characterize the closeness between the target music content and the current hot event.

[0169] The present application embodiment does not limit the specific content and quantity of the current hot events, and can be set according to actual conditions. For example, the current hot events may include Zibo tourism, etc.

[0170] For example, the electronic device can obtain the current hot events in real time based on a specific application program interface such as a news application program interface in advance. Correspondingly, S701 can be implemented as follows: the electronic device traverses at least one target music content; for each target music content, the closeness between the target music content and the current hot event is determined based on a specific algorithm or a specific model, and the timeliness score of the target music content is obtained; after the traversal is completed, the timeliness score of each target music content in the at least one target music content is obtained.

[0171] For example, when the target music content is "Zibo Zibo", if the current hot event is Zibo tourism, then the timeliness score of "Zibo Zibo" can be 100; if the current hot event is Wang Po's matchmaking, then the timeliness score of "Zibo Zibo" can be 15.

[0172] S702: The electronic device sorts the at least one target music content in descending order of timeliness score to obtain the music recommendation list.

[0173] Exemplarily, S702 can be implemented as follows: the electronic device traverses at least one target music content; for any two target music contents among the at least one target music content, performs pairwise comparisons according to the timeliness scores, determines the sorting results of the two target music contents, and adjusts the arrangement order of the two target music contents; after the traversal is completed, a music recommendation list is obtained.

[0174] Simply put, by determining the timeliness score of each target music content, at least one target music content can be sorted in descending order according to the timeliness score, so that music content with a high timeliness score is displayed first. In this way, the timeliness and pertinence of music recommendations can be effectively improved, allowing users to more quickly access music content that is closely related to current hot events, making the music recommendation system more intelligent and in line with actual conditions.

[0175] Next, the process of the electronic device determining the timeliness score of each of the at least one target music content in S701 is described.

[0176] refer to Figure 8 As shown in the content, the process may include but is not limited to the following S801 to S803.

[0177] For each of the at least one target music content, the following processing is performed:

[0178] S801: The electronic device obtains the release time of the target music content and the relevance between the target music content and the current hot event.

[0179] The relevance is used to characterize the degree of association or similarity between the target music content and the current hot event. The embodiment of the present application does not limit the specific form of the relevance, and it can be set according to the actual situation.

[0180] Exemplarily, S801 may be implemented as follows: the electronic device obtains the release time of the target music content from the music library; and determines the relevance of the target music content to the current hot event based on a specific algorithm or a specific model.

[0181] S802: The electronic device determines a target time effectiveness function of the target music content based on the target music type of the target music content.

[0182] The target music type refers to the music type of the target music content. The embodiment of the present application does not limit the specific content and classification method of the target music type, which can be determined according to actual conditions.

[0183] The timeliness function refers to a mathematical expression or algorithm used to determine the timeliness score of music content.

[0184] The target time-effect function refers to the time-effect function corresponding to the target music content. The embodiment of the present application does not limit the specific content of the target time-effect function, and it can be set according to the actual situation.

[0185] Exemplarily, the correspondence between the music type and the time-effect function can be determined in advance based on human experience or a specific algorithm or a specific model, and stored in the electronic device. Correspondingly, S802 can be implemented as follows: the electronic device determines the target music type of the target music content; searches for the target music type in the correspondence; and determines the time-effect function corresponding to the target music type as the target time-effect function.

[0186] S803: The electronic device determines a timeliness score of the target music content based on the release time, the relevance, and the target timeliness function.

[0187] Exemplarily, S803 may be implemented as follows: the electronic device calculates the difference between the release time and the current time, substitutes the difference and the correlation into the target timeliness function, and calculates the timeliness score of the target music content.

[0188] In simple terms, considering that the timeliness of music content of different music types has different changing trends, for example, the timeliness of pop music changes faster, and its timeliness will decay rapidly in a few days, while the timeliness of classical music and jazz music changes slower, and may only begin to decay after several months or even years. Therefore, by determining the timeliness functions corresponding to different music types, and obtaining the release time of the target music content, and the relevance of the target music content to the current hot events, the timeliness score of the target music content can be determined more accurately, thereby providing users with more comprehensive and higher-value music recommendation services.

[0189] The music recommendation method provided in the embodiment of the present application may also include a process of adjusting the target audio sequence.

[0190] After generating a target audio sequence for representing the music recommendation needs of the target user based on the fused modal features, referring to Figure 8 As shown in the content, the process may include but is not limited to the following S901 to S903.

[0191] S901: The electronic device obtains environmental data related to acoustic characteristics of the target user's current environment.

[0192] Environmental data may be any data related to the acoustic characteristics of the target user's current environment. The present application embodiment does not limit the specific content of the environmental data, and it can be set according to actual conditions. For example, the environmental data may include: weather, ambient sound, location, congestion, etc.

[0193] Exemplarily, S901 may be implemented as follows: the electronic device obtains environmental data of the target user's current environment through a specific sensor or a specific application program interface.

[0194] S902: The electronic device generates ambient audio corresponding to the ambient data based on the ambient data.

[0195] Ambient audio refers to audio used to create a specific environment. The specific content of the ambient audio is not limited in the present embodiment of the application and can be set according to actual conditions.

[0196] In a possible implementation, S902 may be implemented as follows: when the environmental data includes the ambient sound, the electronic device enhances or weakens the ambient sound to obtain the ambient audio corresponding to the environmental data.

[0197] In another possible implementation, S902 may be implemented as follows: when the environmental data does not include ambient sound, the electronic device uses an audio generation model to analyze and process the environmental data to generate ambient audio corresponding to the environmental data.

[0198] For example, when the environmental data includes weather and the weather is thunderstorm, the environmental audio generated by the audio generation model may include sounds of wind, rain, thunder and lightning.

[0199] S903: The electronic device adds the ambient audio to the target audio sequence to obtain an adjusted target audio sequence.

[0200] Exemplarily, S903 may be implemented as follows: the electronic device adds the ambient audio to the target audio sequence through audio editing software to obtain an adjusted target audio sequence.

[0201] Simply put, by generating ambient audio corresponding to the ambient data based on the ambient data related to the acoustic characteristics of the target user's current environment, and adding the ambient audio to the target audio sequence, the realism of the target audio sequence can be increased, and the dimension and level of the target audio sequence can be enriched, thereby making the target music content determined based on the target audio sequence more realistic and rich, thereby providing users with music recommendation services that are adapted to the real environment.

[0202] Below, the music recommendation method provided in the embodiment of the present application is explained through a complete embodiment.

[0203] With the rapid development of artificial intelligence technology, personalized recommendation systems have been widely used in various fields. In the field of in-car music, how to provide accurate music recommendation services based on the personalized needs and preferences of users has become an urgent problem to be solved. In-car music comes from different media sources. When a user plays a piece of music, it can recommend music that is close to or has a certain semantic relevance to the music based on the contextual semantics of the listening, providing users with a richer extended reading experience, thereby improving product stickiness and user activity. At the same time, in music recommendation, combined with the actual characteristics of large amount of music data, diverse content and more repetition, it is necessary to solve the problems of music duplication and content diversity in music-related recommendations, and provide objective and neutral music recommendation content. Most traditional music recommendation systems are based on rules or simple machine learning models, which cannot handle large-scale data and complex user behaviors. At the same time, the various links of the traditional music recommendation process are relatively scattered, resulting in poor recommendation results.

[0204] The big model can couple data feature learning, type prediction, recommendation calculation, etc. into an end-to-end form, learn better results during deep learning, and the big model can achieve tasks in an end-to-end efficient manner while obtaining better recommendation accuracy. It can be seen that the recommendation method based on the big model is of great application value. Therefore, in order to solve the above problems, the embodiment of the present application discusses the two aspects of the end-to-end efficient fusion of different link features of the big model and the more powerful high-dimensional user information learning ability, so as to help achieve better in-car music recommendations. The music recommendation method provided in the embodiment of the present application relates to the field of artificial intelligence and music recommendation technology, and in particular to a personalized in-car music recommendation design scheme based on a big model. The scheme uses big model technology to deeply mine and analyze massive data, build a user personalized model, and achieve more accurate music recommendations.

[0205] The overall business module of the personalized in-car music recommendation design solution provided in the embodiment of the present application is as follows: Fig.10 As shown, including:

[0206] Data collection module 1001: This module is responsible for collecting the user's personalized data, including but not limited to the user's music preferences, listening habits, driving habits, vehicle status, etc. This data can be obtained through vehicle sensors, user input, and external data sources.

[0207] Data preprocessing module 1002: This module performs preprocessing operations such as cleaning, deduplication, and classification on the collected data to ensure the quality and accuracy of the data. At the same time, this module is also responsible for feature extraction and representation learning of the data to better utilize large models for training and reasoning.

[0208] Data feature extraction module 1003: This module is used to perform data analysis on input data and extract features of the input data using an algorithm model.

[0209] Scene recognition module 1004: This module is used to accurately identify and distinguish various music scenes. This module can perceive and analyze user information, the user's environment information, etc., and identify various music scenes by capturing and understanding these scene features, providing an important basis for subsequent music recommendations, so as to match the most suitable music for users according to specific music scenes. It can also be used to dynamically adjust the music playback strategy according to the changes in the scene, so as to achieve more intelligent and personalized music recommendations.

[0210] Emotion recognition module 1005: This module can monitor the user's driving status and emotional changes in real time, recognize and understand the user's emotional state, and dynamically adjust the music recommendation strategy accordingly. For example, when the user is in a happy state, the system may recommend some cheerful pop music; when the user is frustrated, it may recommend some soothing light music or healing songs; when the user is detected to be driving fatigue, the system will automatically switch to a more relaxing and soothing music list to remind and cheer up the user; and when the user is detected to be in an excited driving state, the system may recommend some fast-paced and dynamic music.

[0211] Music recommendation module 1006: Based on the output of the large model, this module recommends music content that meets the user's personalized needs. The recommended music content not only takes into account the user's music preferences and listening habits, but also factors such as vehicle status and driving environment. For example, during long-distance driving, the system may recommend some relaxing and soothing music to relieve driving fatigue; while driving on congested roads in the city, the system may recommend some fast-paced music to improve driving mood.

[0212] User feedback module 1007: This module allows users to evaluate and provide feedback on the recommended music. The system continuously adjusts and optimizes the recommendation strategy based on user feedback to improve the accuracy of recommendations and user satisfaction. In addition, users can also input their own music preferences and listening habits through this module to further improve the personalized model.

[0213] Large model pre-training module 1008: This module uses the pre-processed data to train a large-scale deep learning model. The model uses an advanced neural network architecture to learn the laws of data and the characteristic distribution of user behavior patterns through the powerful semantic understanding and language generation capabilities of the large model, and build a general large language model to better understand the user's music preferences and needs.

[0214] In actual implementation, the outputs of the scene recognition module 1004, the emotion recognition module 1005, the music recommendation module 1006, and the user feedback module 1007 can be unified as the input of the big model, and based on the big model learning paradigm, an end-to-end recommendation system can be constructed through the big model pre-training module. In addition, it is worth mentioning that by utilizing the advantages of the big model learning data universality features, the embodiment of the present application maps the two different types of data features, text and speech, to a unified feature space, performs learning and unified prediction of the results, and realizes the ultimate end-to-end recommendation algorithm.

[0215] Next, the music recommendation system architecture of the personalized in-car music recommendation design solution provided in the embodiment of the present application is described. Fig.11 As shown, the music recommendation system architecture of the present application solution includes:

[0216] Data storage layer 1101: used to store the underlying original music text and system basic logs.

[0217] Feature mining layer 1102: responsible for music preprocessing and feature mining, as well as basic algorithms related to Natural Language Processing (NLP).

[0218] Data index layer 1103: used to construct indexes for the extracted music features and user behaviors to ensure high-performance retrieval.

[0219] Recommendation strategy layer 1104: used to implement recall, sorting and filtering strategies related to various recommendations.

[0220] Application interface layer 1105: used to provide encapsulated data interface services for the front end and ensure high availability of services.

[0221] In the above recommendation system architecture, the data preprocessing module of the feature mining layer 1102 is mainly responsible for mining various features in music, including but not limited to keywords, tag classifications, semantic vectors, structured fields, simhash signatures, etc. The above features will be stored in the data index layer 1103, and the big data index framework will provide fast conditional retrieval for the upper-level recommendation strategy. The labeling system and labeling strategy of semantic labeling can be configured and optimized in the music semantic labeling module, which will use part of the NLP algorithm service and front-end tags to configure and manage the platform.

[0222] The music similarity calculation and clustering module uses the characteristics of various dimensions of music to explore hot clusters of music on a global scale, and can support clustering requirements of different levels of granularity: music that is too similar (i.e., leaf nodes of hierarchical clustering) can be merged to avoid the upper-level recall strategy receiving a large amount of identical content, thereby improving recommendation efficiency. Clusters of music that are similar but not identical can provide music of different styles with the same theme, providing multi-angle recommendations.

[0223] The recommendation result recall, sorting and filtering module is responsible for recalling relevant music using different types of strategies: such as keyword recall, tag recall, semantic vector recall, etc., and integrating different types of recall results. The recall ratio or priority of each strategy can be adjusted through business configuration.

[0224] The sorting module provides default sorting logic, including similarity factor, heat factor, time decay factor, etc., and can update key parameters in the sorting algorithm based on the click feedback of the recommended results to optimize the recommendation click rate.

[0225] The filtering module is responsible for filtering the results that are not suitable for recommendation to the user based on what the user has listened to and business rules. If there are too few results after filtering, the result set can be supplemented through secondary recall.

[0226] The data interface module adds the title, author, source, time and other fields required for the front-end display of the recommendation results, and can also mark the reasons for the recommendation for some users, such as "late night playlist", "summer freshness", "mood playlist", etc. Finally, these data are encapsulated in the interface service, providing the necessary balancing, caching and exception handling functions.

[0227] Below, the music recommendation strategy in the personalized in-vehicle music recommendation design scheme provided in the embodiment of the present application is explained.

[0228] There can be different recommendation strategies for music recommendations for different product forms. Common strategies include: popularity-based recommendation, context-based recommendation, user interest-based recommendation, and community-based recommendation (such as collaborative filtering). The embodiment of the present application focuses on the relevance of music to the car usage scenario and the user's current mood to design a recommendation strategy.

[0229] Before making music recommendations, you can first mine music features. Music feature mining includes the following aspects:

[0230] (1) Keyword extraction. The core keywords of music titles and lyrics are extracted through models such as term frequency-inverse document frequency (TF-IDF) and entity recognition, and the relevant weights of each keyword are calculated. For keywords with specific meanings, targeted weighting can be performed through rules. The number of keyword features retained in the end is determined by the weight, and long-tail words with too low weights can be truncated.

[0231] (2) Entity and business tags. This application includes an algorithm service module for labeling music. The tags mainly include: a) style tags: such as jazz, classical, pop, rock, folk, etc.; b) emotion tags: such as cheerful, sad, quiet, excited, etc.; c) instrument tags: such as piano, guitar, violin, saxophone, etc.; d) theme tags: such as love, friendship, nature, history, etc.; e) era tags: such as classical music period, romantic music period, modern music period, etc.; f) region tags: such as European music, American music, Asian music, etc.; g) singer / band tags; h) language tags: such as Chinese, English, Japanese, etc.; i) speed tags: such as slow, medium, fast, etc.; j) emotion tags: such as love, friendship, happiness, etc. These tags will be used in the music recommendation engine as the basis for the underlying feature mining strategy.

[0232] (3) Topic vector. Above the vocabulary and label level, the semantic connotation behind the music can be extracted through pre-training. This semantic connotation can be represented by a topic vector. The mainstream technologies or models used in the industry include the Word to Vector (Word2Vec) model, the Bidirectional Encoder Representation from Transformers (BERT) model, and the Probabilistic Latent Semantic Analysis (PLSA). This application uses its own accumulated music data to train a deep learning semantic model in the field of music. It can extract the full text semantics of the music as a topic vector, and use it in subsequent relevance calculations, cluster mining, and music recommendations.

[0233] When making music recommendations, music recall strategies based on keywords, tags, semantic vectors, etc. can be used. After all music is preprocessed and features such as keywords, tags, and semantic vectors are extracted, the system will build these features into a big data index for the next step of recommendation recall. When a user listens to a piece of music a, the system will calculate the correlation between a and other music b1~bn in the global music library, and obtain the recommendation candidate set according to the following recall strategy:

[0234] (1) Keyword matching recall. This strategy is based on the keyword matching degree between music a and bn. Common calculation methods include TF-IDF, Best Matching 25 (BM25), keyword vector cosine angle, etc., and prioritizes the recall of music with higher keyword matching degree.

[0235] (2) Semantic tag recall. This strategy is based on the semantic tag matching between music a and bn. First, the music is converted into a tag vector or a semantic topic vector. Then the vector distance is calculated and the results with a closer distance are recalled first. Commonly used methods include vector angle cosine distance, Euclidean distance, Manhattan distance, etc. The matching of this recall strategy is not limited by word segmentation, but it requires the establishment of a relatively rich semantic or business tag system in advance and the ability to effectively label and classify the music. A piece of music can be associated with multiple tags or subject categories, and each category can also have a probability weight.

[0236] (3) Balanced recall content. The final result of the recall strategy can be mixed based on the above multiple recall strategies. In this process, the richness of the mixed results should be considered in particular. Otherwise, a large number of recommended music related to a but with little difference from each other will be recalled, causing users to listen fatigue or even give up. In order to balance the richness of the recall results, such as Fig.12As shown in the figure, music clustering, similarity removal, user-listened removal, and representative music recommendation can be performed in the recall stage. By clustering the recommendation results in the recall stage, the recommended music that is too similar can be removed, or music with representative views can be selected for recommendation to avoid redundant recommendation results.

[0237] When making music recommendations, you can also use music sorting strategies. Sorting is the core part of the entire recommendation model and the last link of the entire recommendation system. It plays an important role in re-sorting each user's recall set and displaying the content that users prefer in a higher position. Music sorting strategies include:

[0238] (1) Semantic relevance sorting. The core sorting logic of contextual recommendation is the relevance between the recommendation results and the current music. This relevance integrates various indicators such as keyword matching, tag matching, and semantic vector matching, and is weighted and summarized into comprehensive semantic relevance. The weights of various indicators can be configured based on business experience in the early stage; in the later stage, the optimal parameter configuration can be fitted based on the actual click-through rate of users and the click volume estimation model.

[0239] (2) Time decay sorting. The timeliness and popularity of music will weaken over time, and users are more inclined to listen to newer music. Therefore, in the actual sorting process, time decay penalties are imposed on different types of music. A time penalty function is shown in the following formula (1):

[0240]

[0241] Among them, t represents the time length since the release of the music to be recommended; T and G are adjustable parameters, T is the smoothing value, and G is the time penalty factor. The curve of the time penalty function under different T and G values ​​is as follows Fig.13 As shown in Figure 2, when T and G take different values, the penalty function curve will have different trend changes.

[0242] The penalty function is different for different types of music. Generally speaking, pop music is very effective, and its timeliness decays rapidly in a few days, which means that T is set to be small and G is set to be large; while for classical music and jazz music, which may become outdated in 1 to 3 months, T can be set to be medium and G to be small. In the application, different T and G can be selected to fit the actual curve according to the actual click feedback of different types of content or music, and then the optimal T and G values ​​of the column can be obtained.

[0243] (3) Music popularity ranking. Since the recommended music comes from different theme clusters, different music also represents different popularity. Therefore, this popularity can also be used as one of the factors for recommendation ranking. The higher the popularity, the higher the ranking.

[0244] When making music recommendations, you can also use music filtering strategies to reduce the redundancy of the final recommended music content. Specifically, it includes:

[0245] (1) Similar deduplication filtering. For music that is too similar, simhash calculation is used to remove duplicates to prevent users from seeing redundant information. The priority of filtering can be specified based on factors such as data source or music release time.

[0246] (2) Filtering by what the user has already listened to. It is generally not recommended to list music that the user has already listened to as a recommendation result. In order to be able to record and filter the music that the user has listened to in the shortest time possible. Two solutions can be considered: 1) Directly cache the identification (ID) of the music that has been listened to on the client, and perform post-filtering in the results returned by the recommendation engine. In order to ensure that there is enough music after filtering, the recommendation engine must return a candidate music set that exceeds the number of items in the list. 2) Cache the music that the user has listened to in a key-value (KV) manner in the background, and filter it directly before the recommendation engine returns the results to the front end. If the number of filtered music is insufficient, directly supplement it before responding to the request.

[0247] (3) Business rule filtering. You can specify specific business rules to filter the candidate set returned by the recommendation engine. The management backend will provide a configuration page to maintain these rules: a) Sensitive word filtering. Music containing specific sensitive words is filtered and not recommended. b) Tag filtering: Music containing specific tags is filtered and not recommended. c) Emotional score filtering: Music with a certain range of positive and negative emotional scores is not recommended. d) Combined condition filtering: Multiple conditions above are combined for filtering.

[0248] Below, the end-to-end multimodal large model algorithm result fusion and prediction process in the personalized in-car music recommendation design scheme provided in the embodiment of the present application is explained.

[0249] The overall process is to extract features from different types of text and speech data, fuse them into a unified semantic feature representation, and then learn through a unified codec Transformer. The overall flow chart of the end-to-end multimodal large model algorithm of this application is as follows: Fig.14As shown, the text features after Byte Pair Encoding (BPE) and Embedding and the audio features after Audio Spectrogram Transformer (AST) encoding and Perceiver processing (equivalent to at least one type of modal features mentioned above) are uniformly input into the Transformer model to learn a high-dimensional unified feature (equivalent to the above-mentioned fused modal feature). After that, the Transformer outputs discrete tokens that can represent text and speech tokens, and the results can be output based on these semantic tokens. The high-dimensional unified feature (equivalent to the above-mentioned fused modal feature) is input into the Vector Quantized Generative Adversarial Network (VQGAN) (equivalent to the above-mentioned first audio model and second audio model), and the output includes an audio sequence of speech, ambient sound, and music (equivalent to the above-mentioned target audio sequence).

[0250] The highlight of VQGAN is that it uses codebook to discretize the intermediate features of the encoding model, and uses the generative pre-trained model (Generative Pre-trained Transformer-2, GPT-2) as a coding generation tool. The idea of ​​codebook has been proposed in the Vector Quantized-Variational Auto Encoder (VQ-VAE), and the overall architecture of VQGAN is roughly to replace the VQVAE encoding generator from pixelCNN to Transformer, which is more conducive to processing text and speech serialized data feature learning, and use the discriminator of PatchGAN to add adversarial loss during the training process.

[0251] Among them, VAE is a powerful generative model, which can be understood from the perspective of VAE, that is, there is an Encoder that encodes the data into the latent space (z = encoder (x)), and then a Decoder is used to reconstruct the data from the latent space (x = decoder (z)). For VAE, a restriction is added on this basis, that is, z satisfies the isotropic Gaussian distribution (called prior). The advantage of this is that after the training is completed, the Encoder can be discarded, and a z can be randomly selected from this prior, and then an x ​​can be generated through the Decoder, which refers to the features that characterize the target text or audio.

[0252] The training of VAE should be understood from the perspective of probability. If we look at this process z=encoder(x) from the perspective of probability, it means that the Encoder learns a conditional probability The decoder learns another conditional probability p θ (x|z). At the same time, to make z obey a Gaussian prior, we can record this prior as p(x). In this way, the loss function can be written according to this logic, as shown in the following formula (2):

[0253]

[0254] Among them: ELBO is the evidence lower bound (ELBO), p θ (x|z) are conditional probabilities, and p(z) is the prior distribution.

[0255] Regarding the above mentioned rates Each quantity is a discrete integer, so we can write this number in one-hot form, which can be regarded as a probability distribution. There are K dimensions in total, and each dimension represents the corresponding e_ i From the perspective of VAE, a uniform distribution is given to this K-dimensional distribution as a priori, i.e., p i = 1 / K, so the evidence lower bound (ELBO) This term becomes a constant, as shown in the following formula (3):

[0256]

[0257] The first term represents the contribution of the dimension that is 1 in the one-hot to the KL divergence, and the second term represents the contribution of the other dimensions. ELBO stands for evidence low bound. Evidence refers to x, and ELBO represents the minimum expectation of evidence. By making this lower bound as large as possible, the resulting probability model will be more likely to produce the x value seen here.

[0258] Each dimension of the latent variable z of VAE is a continuous value, and the biggest feature of VQ-VAE is that each dimension of z is a discrete integer, which is consistent with some natural modes. For example, Language is a sequence of symbols, or reasoning, planning and predictive learning. Therefore, VQ-VAE can successfully model important features across multiple dimensions in the data space, such as a phoneme in audio that lasts for many samples / frames, without learning some particularly detailed things. In the discrete encoding method proposed in VQVAE, each dimension of the encoded feature is a discrete value, which is consistent with some natural modes.

[0259] The VQGAN used in this solution combines the VQVAE and GAN networks. At the same time, based on the specific requirements of this task, the network learning loss function is improved. Specifically, the loss function is as follows:

[0260]

[0261] Among them, L is the loss function, and three branches are used to form the total loss function. The first item is used to train the encoder and decoder. During bp, the gradient of z_q(x) is directly copied to z_e(x) instead of the embedding in the codebook, so this item only trains the encoder and decoder. The second item is used to train the codebook, so that the embeddings in the codebook are close to their respective nearest z_e(x) to minimize their loss. The third item is used to train the encoder, the purpose is to encourage the encoder's output to stay close to the selected codebook vector to prevent it from fluctuating too frequently from one code vector to another, that is, to prevent the encoder's output from frequently jumping between codebook embeddings.

[0262] At this point, the task based on the overall branch can be trained.

[0263] A music recommendation system with high recommendation accuracy first requires an accurate speech recognition algorithm. Accurate speech recognition and prediction determine the quality of the recommendation. In the speech recognition link of the large model, the embodiment of the present application adds a fine-grained VQ-GAN as a speech decoder, which can fuse the multimodal tokens output by the Transformer and predict the results. The mathematical description of the self-attention mechanism used by the Transformer is as follows:

[0264]

[0265] Among them, Q, K, and V are three Query, Key, and Value matrices for attention calculation after linear mapping.

[0266] The conditional probability is as shown in the following formula (6):

[0267] p(s|c) = ∏ i p(s i |s<i, c) Formula (6);

[0268] Among them, p(s|c) represents the probability of sequence s under the given condition c. p(s i |s<i, c) represents the probability of the i-th element s under the condition that the first i - 1 elements (i.e., s<i) in the known sequence and the condition c are known. i of.

[0269] Based on the speech feature c, the sequence feature sequence (equivalent to the above-mentioned target audio sequence) is obtained, which is called s here, and finally a likelihood function value (equivalent to the above-mentioned matching degree) is calculated. Specifically, in this task, the calculation goal is that for the expected speech prediction result, the likelihood function can estimate the speech feature sequence combination (equivalent to the above-mentioned target music content) when the maximum probability of obtaining this result can be achieved.

[0270] The beneficial effects of this solution include: starting from multi-modal learning, realizing end-to-end recommendation based on a large model, and can be applied to the in-vehicle music recommendation scenario.

[0271] In a second aspect, an embodiment of the present application provides a music recommendation device, as Fig.15 shown, the music recommendation device 150 includes: an acquisition unit 1501, a fusion unit 1502, a generation unit 1503, a determination unit 1504, and an output unit 1505.

[0272] Among them:

[0273] The acquisition unit 1501 is used to acquire at least one type of modal feature for music recommendation for the target user; each type of modal feature in the at least one type of modal feature is obtained by performing feature extraction on a type of modal data;

[0274] The fusion unit 1502 is used to perform feature fusion on the at least one type of modal feature to obtain a fused modal feature;

[0275] The generation unit 1503 is used to generate a target audio sequence for characterizing the music recommendation requirement of the target user based on the fused modal feature;

[0276] A determination unit 1504, configured to determine at least one target music content based at least on the target audio feature of the target audio sequence to form a music recommendation list;

[0277] The output unit 1505 is used to output the music recommendation list.

[0278] In some embodiments, the generation unit 1503 is specifically used to: use a first audio model to process the fusion modal features to predict the music recommendation type of the target user; the music recommendation type includes at least one of the following: melody type, rhythm type, emotion type and music style type; the first audio model belongs to an inference-based large model; use a second audio model to process the music recommendation type to generate a target audio sequence that is consistent with the music recommendation type; the second audio model belongs to a generative large model.

[0279] In some embodiments, the determination unit 1504 is specifically used to: determine at least one candidate music content in the music library based on the target audio feature; or, determine at least one candidate music content in the music library based on the target audio feature and the target audio change feature of the target audio sequence; determine the at least one target music content among the at least one candidate music content; and sort the at least one target music content to obtain the music recommendation list.

[0280] In some embodiments, the determination unit 1504 is also used to: for each of the at least one first music content included in the music library, perform the following processing to obtain the at least one candidate music content: determine the audio feature matching degree between the target audio feature and the first audio feature of the first music content, and the audio change feature matching degree between the target audio change feature and the first audio change feature of the first music content; when the audio feature matching degree is greater than a first threshold and the audio change feature matching degree is greater than a second threshold, determine the first music content as one of the candidate music contents.

[0281] In some embodiments, when the target audio feature includes a target rhythm feature, a target melody feature and a target timbre feature, and the first audio feature includes a first rhythm feature, a first melody feature and a first timbre feature, the determination unit 1504 is further used to: respectively determine the rhythm matching degree between the target rhythm feature and the first rhythm feature, the melody matching degree between the target melody feature and the first melody feature, and the timbre matching degree between the target timbre feature and the first timbre feature; respectively determine a first weight corresponding to the rhythm matching degree, a second weight corresponding to the melody matching degree, and a third weight corresponding to the timbre matching degree; based on the first weight, the second weight and the third weight, perform a weighted average of the rhythm matching degree, the melody matching degree and the timbre matching degree to obtain the audio feature matching degree.

[0282] In some embodiments, the determination unit 1504 is also used to: determine a representative identifier of each of the at least one candidate music content; the representative identifier is used to characterize whether the candidate music content is representative in the corresponding music type; and determine the candidate music content whose representative identifier is a first value in the at least one candidate music content as the at least one target music content; the first value characterizes that the candidate music content is representative in the corresponding music type.

[0283] In some embodiments, the determination unit 1504 is also used to: determine the timeliness score of each of the at least one target music content; the timeliness score is used to characterize the closeness between the target music content and the current hot event; sort the at least one target music content in descending order according to the timeliness score to obtain the music recommendation list.

[0284] In some embodiments, the determination unit 1504 is also used to: perform the following processing for each of the at least one target music content: obtain the release time of the target music content and the relevance of the target music content to the current hot event; determine the target timeliness function corresponding to the target music content based on the target music type of the target music content; determine the timeliness score of the target music content based on the release time, the relevance and the target timeliness function.

[0285] In some embodiments, the music recommendation device 150 further includes an adjustment unit. The adjustment unit is used to: obtain environmental data related to acoustic characteristics of the target user's current environment; generate environmental audio corresponding to the environmental data based on the environmental data; and add the environmental audio to the target audio sequence to obtain the adjusted target audio sequence.

[0286] It should be noted that the various units included in the music recommendation device provided in the embodiment of the present application can be implemented by a processor in an electronic device; of course, it can also be implemented by a specific logic circuit; in the implementation process, the processor can be a central processing unit (CPU, Central Processing Unit), a microprocessor (MPU, Micro Processor Unit), a digital signal processor (DSP, Digital Signal Processor) or a field programmable gate array (FPGA, Field-Programmable Gate Array), etc.

[0287] The description of the above device embodiment is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment. For technical details not disclosed in the device embodiment of the present application, please refer to the description of the method embodiment of the present application for understanding.

[0288] It should be noted that in the embodiments of the present application, if the above-mentioned music recommendation method is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application can be essentially or partly embodied in the form of a software product that contributes to the relevant technology. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a magnetic disk or an optical disk. In this way, the embodiments of the present application are not limited to any specific combination of hardware and software.

[0289] In a third aspect, an embodiment of the present application provides an electronic device, comprising at least a memory and a processor, wherein the memory stores a computer program that can be executed on the processor, and when the processor executes the program, the steps in the music recommendation method provided in the above embodiment are implemented.

[0290] Combine the following Fig.16 The electronic device 160 shown is a block diagram of the electronic device.

[0291] In one example, if Fig.16As shown, the electronic device 160 includes: a processor 1601, at least one communication bus 1602, at least one external communication interface 1603 and a memory 1604. The communication bus 1602 is configured to realize the connection and communication between these components. The external communication interface 1603 may include a standard wired interface and a wireless interface.

[0292] The memory 1604 is configured to store instructions and applications executable by the processor 1601, and can also cache data to be processed or processed by the processor 1601 and various modules in the electronic device (for example, image data, audio data, voice communication data, and video communication data), which can be implemented through flash memory (FLASH) or random access memory (Random Access Memory, RAM).

[0293] In another example, the electronic device may be a terminal device (vehicle device or mobile phone) or a controller. The terminal device or the controller is used to execute the steps in the music recommendation method provided in the above embodiment.

[0294] In a fourth aspect, an embodiment of the present application further provides a storage medium, that is, a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps in the music recommendation method provided in the above embodiment are implemented.

[0295] In a fifth aspect, an embodiment of the present application further provides a computer program product, including a computer program or instructions, which, when executed by a processor, implements the steps in the music recommendation method provided in the above embodiment.

[0296] It should be noted here that the description of the above storage medium and device embodiments is similar to the description of the above method embodiments, and has similar beneficial effects as the method embodiments. For technical details not disclosed in the storage medium and device embodiments of this application, please refer to the description of the method embodiments of this application for understanding.

[0297] It should be understood that "one embodiment" or "an embodiment" mentioned throughout the specification means that specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, "in one embodiment" or "in some embodiments" appearing throughout the specification may not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in various embodiments of the present application, the size of the sequence number of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application. The above-mentioned sequence numbers of the embodiments of the present application are for description only and do not represent the advantages and disadvantages of the embodiments.

[0298] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0299] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0300] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0301] In addition, all functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may be a separate unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0302] A person skilled in the art can understand that all or part of the steps of implementing the above method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiment; and the aforementioned storage medium includes: mobile storage devices, read-only memories (ROM), magnetic disks or optical disks, etc., various media that can store program codes.

[0303] Alternatively, if the above-mentioned integrated unit of the present application is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application can be essentially or partly embodied in the form of a software product that contributes to the relevant technology. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0304] The above is only an implementation method of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A music recommendation method, characterized in that: The music recommendation method comprises: Acquire at least one type of modal features for recommending music to a target user; each type of modal features in the at least one type of modal features is obtained by extracting features from a type of modal data; Performing feature fusion on the at least one type of modal features to obtain fused modal features; Based on the fused modal features, generating a target audio sequence for representing the music recommendation needs of the target user; determining at least one target music content based at least on the target audio feature of the target audio sequence to form a music recommendation list; Output the music recommendation list.

2. The music recommendation method according to claim 1, characterized in that: The step of generating a target audio sequence for representing the music recommendation needs of the target user based on the fused modal features includes: The fused modal features are processed using a first audio model to predict the music recommendation type of the target user; the music recommendation type includes at least one of the following: melody type, rhythm type, emotion type and music style type; the first audio model is an inference-based large model; The music recommendation type is processed using a second audio model to generate a target audio sequence that matches the music recommendation type; the second audio model belongs to a generative large model.

3. The music recommendation method according to claim 1, characterized in that: The step of determining at least one target music content based at least on the target audio feature of the target audio sequence to form a music recommendation list comprises: Based on the target audio feature, determining at least one candidate music content in the music library; or based on the target audio feature and the target audio variation feature of the target audio sequence, determining the at least one candidate music content in the music library; determining the at least one target music content among the at least one candidate music content; The at least one target music content is sorted to obtain the music recommendation list.

4. The music recommendation method according to claim 3, characterized in that: The determining, in the music library, the at least one candidate music content based on the target audio feature and the target audio variation feature of the target audio sequence comprises: For each of the at least one first music content included in the music library, the following processing is performed to obtain the at least one candidate music content: Determining an audio feature matching degree between the target audio feature and a first audio feature of the first music content, and an audio change feature matching degree between the target audio change feature and a first audio change feature of the first music content; When the audio feature matching degree is greater than a first threshold and the audio change feature matching degree is greater than a second threshold, the first music content is determined as one of the candidate music contents.

5. The music recommendation method according to claim 4, characterized in that: In a case where the target audio feature includes a target rhythm feature, a target melody feature, and a target timbre feature, and the first audio feature includes a first rhythm feature, a first melody feature, and a first timbre feature, determining an audio feature matching degree between the target audio feature and the first audio feature of the first music content includes: respectively determining a rhythm matching degree between the target rhythm feature and the first rhythm feature, a melody matching degree between the target melody feature and the first melody feature, and a timbre matching degree between the target timbre feature and the first timbre feature; respectively determining a first weight corresponding to the rhythm matching degree, a second weight corresponding to the melody matching degree, and a third weight corresponding to the timbre matching degree; Based on the first weight, the second weight and the third weight, the rhythm matching degree, the melody matching degree and the timbre matching degree are weighted averaged to obtain the audio feature matching degree.

6. The music recommendation method according to claim 3, characterized in that: The determining the at least one target music content from the at least one candidate music content comprises: Determine a representative identifier of each of the at least one candidate music content; the representative identifier is used to indicate whether the candidate music content is representative in the music genre to which it belongs; Among the at least one candidate music content, a candidate music content that is representatively identified as a first value is determined as the at least one target music content; the first value indicates that the candidate music content is representative in the corresponding music type.

7. The music recommendation method according to claim 3, characterized in that: The step of sorting the at least one target music content to obtain the music recommendation list includes: Determine the timeliness score of each of the at least one target music content; the timeliness score is used to characterize the closeness between the target music content and the current hot event; The at least one target music content is sorted in descending order according to the timeliness score to obtain the music recommendation list.

8. The music recommendation method according to claim 7, characterized in that: The determining of the timeliness score of each of the at least one target music content comprises: For each of the at least one target music content, the following processing is performed: Acquire the release time of the target music content and the relevance of the target music content to the current hot event; Determining a target time-effect function corresponding to the target music content based on a target music type of the target music content; Based on the release time, the relevance, and the target timeliness function, a timeliness score of the target music content is determined.

9. The music recommendation method according to claim 1, characterized in that: After generating a target audio sequence for representing the music recommendation demand of the target user based on the fused modal features, the method further includes: Acquiring environmental data related to acoustic characteristics of the target user's current environment; Based on the environmental data, generating environmental audio corresponding to the environmental data; The environmental audio is added to the target audio sequence to obtain the adjusted target audio sequence.

10. An electronic device, characterized in that: The electronic device comprises at least a memory and a processor, the memory stores a computer program executable on the processor, and the processor implements the method according to any one of claims 1 to 9 when executing the computer program.

Citation Information

Patent Citations

  • Music recommendation method and device

    CN106302678A

  • Music recommendation method, device and equipment and storage medium

    CN111506762A

  • Music generation method and related device

    CN116994544A

  • Music recommendation method and system and intelligent cabin

    CN117216315A

  • Personalized music recommendation method and device, equipment, medium and wearable equipment

    CN118013071A

Cited By

  • Sequence recommendation method based on positive and negative feedback and logic rule fusion

    CN120067449A

  • A sequence recommendation method based on the fusion of positive and negative feedback and logical rules

    CN120067449B

  • Commodity recommendation method and device, electronic equipment and program product

    CN120996914A