A text music matching method, device, and computer storage medium

Abstract

The embodiment of the present application discloses a method, device, and computer storage medium for composing music for text. The method relates to the natural language processing direction in the field of artificial intelligence. The method includes: obtaining sample text and multi-dimensional sample feature information corresponding to the sample text, predicting multi-dimensional user feedback information of browsing users for the sample text based on the text music matching model and the sample feature information, obtaining the loss corresponding to the user feedback information of each dimension based on the sample feature information and the user feedback information, training the text music matching model based on the loss corresponding to the user feedback information of each dimension, obtaining the trained text music matching model, and predicting the target music matching of the text to be matched based on the trained text music matching model. The scheme can improve the accuracy of text music matching by using the multi-dimensional sample feature information corresponding to the sample text as the model input and setting multiple optimization targets.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a text music matching method, device, and computer storage medium. Background Art

[0002] When users are reading paragraphs, articles, or chatting in chat applications, playing appropriate background music can create a good reading experience for users and significantly increase the reading time and number of user interactions. However, currently, when composing music for text, the author needs to select background music suitable for the text or the user when editing the text. This method of composing music for text is not only costly, but the selected background music may not be accurate. Summary of the invention

[0003] The embodiments of the present application provide a text music matching method, device, and computer storage medium, which can improve the accuracy of text music matching.

[0004] The present application provides a method for composing music to a text, comprising:

[0005] Acquire sample text and multi-dimensional sample feature information corresponding to the sample text;

[0006] Based on the text music model and the sample feature information, predict multi-dimensional user feedback information of browsing users on the sample text;

[0007] Based on the sample feature information and the user feedback information, obtaining the loss corresponding to the user feedback information of each dimension;

[0008] Based on the loss corresponding to the user feedback information of each dimension, the text-music matching model is trained to obtain a trained text-music matching model;

[0009] The target music for the text to be matched is predicted based on the trained text music matching model.

[0010] Accordingly, the embodiment of the present application further provides a text music matching device, including:

[0011] An acquisition model is used to acquire sample text and multi-dimensional sample feature information corresponding to the sample text;

[0012] A first prediction model is used to predict multi-dimensional user feedback information of browsing users on the sample text based on the text music model and the sample feature information;

[0013] A loss acquisition model, used to acquire the loss corresponding to the user feedback information of each dimension based on the sample feature information and the user feedback information;

[0014] A training model is used to train the text-music matching model based on the loss corresponding to the user feedback information of each dimension to obtain a trained text-music matching model;

[0015] The second prediction model is used to predict the target music of the text to be matched based on the trained text music matching model.

[0016] Optionally, in some embodiments, the acquisition module may include an acquisition submodule and an extraction submodule, as follows:

[0017] An acquisition submodule, used to acquire sample text and multiple sample soundtrack information corresponding to the sample text;

[0018] The extraction submodule is used to extract the features of the sample text and the sample music information to obtain multi-dimensional sample feature information.

[0019] At this time, the extraction submodule can be specifically used to extract the sample music feature information corresponding to the sample music information based on a preset database, extract the features of the sample text, and obtain the sample text feature information corresponding to the sample text.

[0020] Optionally, in some embodiments, the first prediction module may include a first prediction submodule, a second prediction submodule and a fusion submodule, as follows:

[0021] A first prediction submodule, configured to predict attribute prediction information of a browsing user for the sample text based on the linear submodel and the sample attribute information;

[0022] A second prediction submodule, configured to predict label prediction information of a browsing user for the sample text based on the deep neural network submodel and the sample label information;

[0023] The fusion submodule is used to fuse the attribute prediction information and the label prediction information to obtain multi-dimensional user feedback information.

[0024] At this time, the second prediction submodule can be specifically used to convert the sample label information into a sample label feature vector, and based on the deep neural network sub-model and the sample label feature vector, predict the label prediction information of the browsing user for the sample text.

[0025] Optionally, in some embodiments, the second prediction module may include a third prediction submodule and a determination submodule, as follows:

[0026] A third prediction submodule is used to predict multi-dimensional target user feedback information of each piece of music in the music library for the text to be matched with music based on the trained text matching model, the music library, and the text to be matched with music;

[0027] The determination submodule is used to determine the target music for the text to be composed from the music library according to the target user feedback information.

[0028] At this time, the third prediction submodule can be specifically used to obtain the text to be matched with music, and multiple text features corresponding to the text to be matched with music, obtain the music library, and music features corresponding to multiple music in the music library, and based on the trained text matching model, the text features, and the music features, predict the multi-dimensional target user feedback information of each music in the music library for the text to be matched with music.

[0029] At this time, the determination submodule can be specifically used to perform weighted fusion on the multi-dimensional target user feedback information corresponding to each piece of music in the music library to obtain the fused user feedback information corresponding to each piece of music, and determine the target music for the text to be matched from the multiple pieces of music in the music library based on the fused user feedback information.

[0030] In addition, an embodiment of the present application further provides a computer storage medium, which stores a plurality of instructions, and the instructions are suitable for a processor to load to execute the steps in any one of the text music composition methods provided in the embodiments of the present application.

[0031] The embodiment of the present application can obtain sample text and multi-dimensional sample feature information corresponding to the sample text, predict multi-dimensional user feedback information of browsing users for the sample text based on the text music model and the sample feature information, obtain the loss corresponding to the user feedback information of each dimension based on the sample feature information and the user feedback information, train the text music model based on the loss corresponding to the user feedback information of each dimension, obtain the trained text music model, and predict the target music of the text to be music-matched based on the trained text music model. The solution can improve the accuracy of text music by using the multi-dimensional sample feature information corresponding to the sample text as the model input and setting multiple optimization goals. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0033] Figure 1is a scene diagram of a text music matching system provided in an embodiment of the present application;

[0034] Figure 2 is a first flow chart of the text music matching method provided in an embodiment of the present application;

[0035] Figure 3 is a second flow chart of the text music matching method provided in an embodiment of the present application;

[0036] Figure 4 This is a flow chart of a text music matching model after application training provided in an embodiment of the present application;

[0037] Figure 5 is a flow chart of text music matching model training provided by an embodiment of the present application;

[0038] Figure 6 is a structural schematic diagram of a text music matching device provided in an embodiment of the present application;

[0039] Figure 7 It is a structural diagram of the network device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0040] Please refer to the drawings, where the same component symbols represent the same components, and the principles of the present application are illustrated by implementing them in an appropriate computing environment. The following description is based on the illustrated specific embodiments of the present application, which should not be considered as limiting other specific embodiments of the present application that are not described in detail herein.

[0041] In the following description, the specific embodiments of the present application will be described with reference to the steps and symbols performed by one or more computers, unless otherwise stated. Therefore, these steps and operations will be mentioned several times as being performed by a computer, and the computer execution referred to herein includes the operation of a computer processing unit by an electronic signal representing data in a structured form. This operation converts the data or maintains it at a location in the memory system of the computer, which can be reconfigured or otherwise change the operation of the computer in a manner familiar to testers in the field. The data structure maintained by the data is a physical location in the memory, which has specific characteristics defined by the data format. However, the principles of the present application are described in the above text, which does not represent a limitation, and testers in the field will understand that the various steps and operations described below can also be implemented in hardware.

[0042] The term "module" used herein can be regarded as a software object executed on the computing system. The different components, modules, engines and services described herein can be regarded as implementation objects on the computing system. The apparatus and method described herein can be implemented in software, and of course can also be implemented in hardware, all within the scope of protection of this application.

[0043] The terms "first", "second", and "third" in this application are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or device that includes a series of steps or modules is not limited to the listed steps or modules, but some embodiments also include steps or modules that are not listed, or some embodiments also include other steps or modules inherent to these processes, methods, products, or devices.

[0044] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0045] The embodiment of the present application provides a method for composing music to a text, and the execution subject of the method for composing music to a text may be a text composing music device provided in the embodiment of the present application, or a network device integrating the text composing music device, wherein the text composing music device may be implemented in hardware or software. Among them, the network device may be a smart phone, a tablet computer, a PDA, a laptop computer, or a desktop computer. The network device includes but is not limited to a computer, a network host, a single network server, a plurality of network server sets, or a cloud composed of a plurality of servers.

[0046] See also Figure 1 , Figure 1 A schematic diagram of an application scenario of the text music arrangement method provided in an embodiment of the present application, taking the text music arrangement device integrated in a network device as an example, the network device can obtain sample text and multi-dimensional sample feature information corresponding to the sample text, and predict multi-dimensional user feedback information of browsing users for the sample text based on the text music arrangement model and the sample feature information, and obtain the loss corresponding to the user feedback information of each dimension based on the sample feature information and the user feedback information, and train the text music arrangement model based on the loss corresponding to the user feedback information of each dimension to obtain the trained text music arrangement model, and predict the target music arrangement for the text to be arranged based on the trained text music arrangement model.

[0047] The text music matching method provided in the embodiment of the present application relates to the computer vision direction in the field of artificial intelligence. The embodiment of the present application can align multiple feature maps in the feature map group with the reference feature map through image processing technology, and then identify the pixel type information corresponding to the video image to be processed.

[0048] Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science. It attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making. Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level technology and software-level technology. Among them, artificial intelligence software technology mainly includes computer vision technology, machine learning / deep learning and other directions.

[0049] Among them, natural language processing (NLP) is an important direction in the fields of computer science and artificial intelligence. It studies various theories and methods that can achieve effective communication between people and computers using natural language. Natural language processing is a science that integrates linguistics, computer science, and mathematics. Therefore, research in this field will involve natural language, that is, the language people use in daily life, so it is closely related to the study of linguistics. Natural language processing technology usually includes text processing, semantic understanding, machine translation, robot question answering, knowledge graph and other technologies.

[0050] See also Figure 2 , Figure 2 The flowchart of the text music matching method provided in the embodiment of the present application is specifically described by the following embodiments:

[0051] 201. Obtain sample text and multi-dimensional sample feature information corresponding to the sample text.

[0052] The sample text can be a text that has been matched with music and can be used as a sample for model training. Text is a form of expression of written language. A sentence or a combination of multiple sentences with complete and systematic meaning can be called text. A text can be a sentence, a paragraph, or an article, etc. There can be many types of sample texts. For example, the sample text can be a complete article written by the author in a public account or mini program, a part of a paragraph excerpted or written by a user, or a conversation or chat conducted by a user through a chat application, etc.

[0053] The sample feature information may be feature information related to the sample text and may be used for model training. In order to enable the trained model to more accurately determine the music that matches the text, the model may be trained using sample feature information of multiple dimensions. For example, the sample feature information may include feature information corresponding to the sample music that matches the sample text, feature information corresponding to the user group that browses the sample text, feature information corresponding to the published content related to the sample text, and so on.

[0054] In practical applications, for example, before training the model, the training samples can be determined, and several texts can be extracted from the database as sample texts, and sample feature information corresponding to the sample music matching the sample text, sample feature information corresponding to the user group that has browsed the sample text, and sample feature information corresponding to the time and platform of the sample text, etc. In order to consider factors such as the user's experience of reading the text after the soundtrack, a variety of sample feedback information from users who have browsed the sample text can be collected as optimization targets to optimize the model. The sample feedback information can be reflected in the form of labels of the training samples. For example, the sample feedback information can include one or more of the average browsing time of users, the number of user comments, the amount of user rewards, the number of user shares, etc.

[0055] In one embodiment, the sample feedback information may not be limited to the various feedback information mentioned above. Other types of sample feedback information may be used as optimization targets to optimize the model according to actual conditions or as specified by the author.

[0056] In one embodiment, in order to improve the modeling capability of the model, it is necessary to obtain sample feature information of multiple dimensions. Therefore, multiple sample soundtrack information corresponding to the sample text can be obtained, and then feature extraction of the sample soundtrack information can be performed to obtain sample feature information. Specifically, the step of "obtaining sample text and sample feature information of multiple dimensions corresponding to the sample text" may include:

[0057] Acquire sample text and multiple sample soundtrack information corresponding to the sample text;

[0058] The features of the sample text and the sample music information are extracted to obtain multi-dimensional sample feature information.

[0059] The sample music information may be music-related information corresponding to the sample text. In order to improve the modeling capability of the model, the sample music information of multiple dimensions corresponding to the sample text may be obtained. For example, the sample music information may include the sample music information of the music dimension, that is, the sample music matching the sample text, the sample music information of the user dimension, that is, the user group that has browsed the sample text, and the sample music information of the context dimension, that is, the time and platform of the sample text.

[0060] In practical applications, for example, several texts can be extracted from the database as sample texts, and then sample music matching the sample text, the user group that has browsed the sample text, and the time and platform of the sample text publication can be obtained. Among them, the number of sample music matching the sample text is not limited. For example, when the sample text is "Happy New Year", the sample music can be "New Year is Coming", "Celebrating the New Year", "Happy New Year" and other songs related to the New Year. Among them, the user group that has browsed the sample text can be multiple users who have browsed the sample text. By using the information corresponding to multiple users who have browsed the sample text as training samples, the user's overall feeling for the text music can be optimized to prevent the model from being disturbed by the behavior of a single user. After obtaining the sample music information, feature extraction can be performed on the sample music information to obtain sample feature information.

[0061] In one embodiment, the sample music information may not only include the music dimension, the user dimension, and the context dimension, but may also add sample music information of other dimensions as training samples according to actual conditions, so as to train the model.

[0062] In one embodiment, since the sample music information includes multiple dimensions, the methods for extracting features of sample music information of different dimensions are also different. Therefore, different feature extraction methods can be used to extract sample feature information according to the type of sample music information. Specifically, the step of "extracting the features of the sample text and the sample music information to obtain multi-dimensional sample feature information" may include:

[0063] Extracting sample music feature information corresponding to the sample music information based on a preset database;

[0064] The features of the sample text are extracted to obtain sample text feature information corresponding to the sample text.

[0065] The preset database may be a pre-established database related to the sample music information. By searching the preset database, the purpose of obtaining the sample feature information corresponding to the sample music information can be achieved. For example, the preset database may include a music library, a music review library, a user portrait library, etc. The music library may be a library including multiple sample music. The music review library may be a database including user review information on the sample music. The user portrait library may be a database including user information corresponding to multiple users.

[0066] The sample feature information may include sample text feature information and sample music feature information.

[0067] The sample text feature information may be feature information extracted from the sample text. For example, the sample text feature information may include text label information, context attribute information, and the like.

[0068] The text label information may be a label that represents the characteristics of the sample text in the form of text. For example, the text label information corresponding to the sample text may be "emotional category", indicating that the sample text is an emotional text. The text label information corresponding to the sample text may also be "history category", "gossip category", and so on.

[0069] Among them, the contextual attribute information can be attribute information that represents the characteristics of the sample text in the form of a specific category. For example, the contextual attribute information corresponding to the sample text can be "publication time January 1, 2019", indicating that the publication time of the sample text is January 1, 2019. For another example, the contextual attribute information corresponding to the sample text can also be "publication platform xx website", indicating that the publication platform of the sample text is xx website.

[0070] The sample music feature information may be feature information extracted from the sample music feature information. For example, the sample music feature information may include music tag information, music attribute information, reader group tag information, reader group attribute information, and the like.

[0071] The music tag information may be a tag that indicates the characteristics of the sample music that matches the sample text in the form of text. For example, the music tag information corresponding to the sample music may be "Jazz", indicating that the sample music that matches the sample text is Jazz. The music tag information corresponding to the sample music may also be "Sad", "Instrumental", "Listen to when you are heartbroken", "Quiet", etc.

[0072] The music attribute information may be attribute information that indicates the characteristics of the sample music that matches the sample text in the form of a specific category. For example, the music attribute information corresponding to the sample music may be "duration 20 minutes", indicating that the duration of the sample music that matches the sample text is 20 minutes. The music attribute information corresponding to the sample music may also be "category Mandarin", "mode C major", and so on.

[0073] The reader group label information may be a label that indicates the characteristics of the user group that browses the sample text in the form of text. For example, the reader group label information corresponding to the user group may be "middle-aged people", indicating that the user group that browses the sample text is mostly middle-aged people. The reader group label information corresponding to the user group may also be "secondary", "cool", etc.

[0074] The reader group attribute information may be attribute information that indicates the characteristics of the user group that browses the sample text in the form of specific categories. For example, the reader group attribute information corresponding to the user group may be "average age 28 years old", indicating that the average age of the user group that browses the sample text is 28 years old. The reader group attribute information corresponding to the user group may also be "average height 170 cm", "major cities Beijing / Shanghai", and so on.

[0075] In practical applications, for example, Figure 5 As shown, after obtaining the sample text, the sample music matching the sample text, and the user group browsing the sample text, you can obtain the music tag information by searching the music library and the song name, lyrics, music reviews, etc. in the music review library; obtain the music attribute information by searching the music library; obtain the reader group tag information by searching the user portrait library for the articles the user has read and the comments posted; and obtain the reader group attribute information through the user portrait library.

[0076] Since the sample text is constantly increasing, the text label information is not obtained by pre-establishing a sample text library, but by mining the sample text as a training sample to obtain the text label information. Since the context attribute information is also obtained based on the sample text, the context attribute information can also be obtained by mining the sample text as a training sample.

[0077] 202. Based on the text music matching model and the sample feature information, predict the multi-dimensional user feedback information of the browsing user on the sample text.

[0078] Among them, the text music model can be a network model that can match the target music for the text to be matched. The embodiment of the present application does not limit the type of the text music model. As long as it is a supervised model that can match the target music for the text to be matched, it can be used as the text music model in the embodiment of the present application. For example, the text music model can be a wide&deep model, wherein the wide&deep model is a model for classification and regression by combining the memory ability of the linear model and the generalization ability of the deep neural network model. The wide&deep model includes a linear sub-model and a deep neural network sub-model. The linear sub-model part is a simple shallow model, such as logistic regression, svm (Support Vector Machine, support vector machine), etc., which can be used to process non-text features such as numerical and categorical types. The deep neural network sub-model part is DNN (Deep Neural Networks, deep neural network), which can be used to process label words, extract feature vectors and perform forward propagation. In the training process of the wide&deep model, the parameters of the two sub-models are optimized at the same time, so as to achieve the optimal prediction ability of the overall model.

[0079] Among them, user feedback information is information related to the feedback of users after reading the text with background music. For example, user feedback information may include one or more of the average browsing time of users, the number of user comments, the amount of user rewards, the number of user shares, etc.

[0080] In practical applications, for example, after obtaining sample feature information of multiple dimensions, the sample feature information of multiple dimensions can be input into a text music soundtrack model, and based on the text music soundtrack model, multi-dimensional user feedback information of users who browse the sample text for the sample text can be predicted. The user feedback information may include one or more of the average user browsing time, number of user comments, user reward amount, number of user sharing times, and so on.

[0081] In one embodiment, when the text music model is a wide & deep model, since the text music model includes a linear sub-model and a deep neural network sub-model, it is necessary to classify the sample feature information and then input it into the model. Specifically, the step of "predicting multi-dimensional user feedback information of browsing users for the sample text based on the text music model and the sample feature information" may include:

[0082] Based on the linear sub-model and the sample attribute information, predicting attribute prediction information of the browsing user for the sample text;

[0083] Based on the deep neural network sub-model and the sample label information, predicting label prediction information of the browsing user for the sample text;

[0084] The attribute prediction information and the label prediction information are integrated to obtain multi-dimensional user feedback information.

[0085] The sample feature information includes sample label information and sample attribute information.

[0086] The sample label information may be a label that represents sample features in the form of text. For example, the sample label information may include music label information, reader group label information, text label information, and the like.

[0087] The sample attribute information may be attribute information that represents sample characteristics in the form of specific categories. For example, the sample attribute information may include music attribute information, reader group attribute information, context attribute information, and the like.

[0088] Among them, the text music model includes a linear sub-model and a deep neural network sub-model.

[0089] Among them, the linear sub-model is a simple shallow model, such as logistic regression, SVM, etc., which can be used to process non-text features such as numerical and categorical. The linear sub-model has memory ability, that is, it can discover the correlation between features from historical data.

[0090] The deep neural network sub-model is DNN, which can be used to process label words, extract feature vectors and perform forward propagation. The deep neural network sub-model uses low-dimensional dense features as input through the embedding method, which can better generalize feature combinations that have not appeared in the training samples.

[0091] In practical applications, for example, Figure 5 As shown, the sample feature information can be divided into sample label information and sample attribute information. The sample attribute information is input into the linear sub-model to predict the attribute prediction information of the browsing user for the sample text. The sample label information is input into the deep neural network sub-model to predict the label prediction information of the browsing user for the sample text. Then, the attribute prediction information and the label prediction information are fused to obtain multi-dimensional user feedback information.

[0092] In one embodiment, in the deep neural network sub-model, the user feedback information can be predicted by converting the sample label information into a vector form. Specifically, the step of "predicting the label prediction information of the browsing user for the sample text based on the deep neural network sub-model and the sample label information" may include:

[0093] Converting the sample label information into a sample label feature vector;

[0094] Based on the deep neural network sub-model and the sample label feature vector, label prediction information of the browsing user for the sample text is predicted.

[0095] Among them, the deep neural network sub-model includes an embedding layer and a hidden layer.

[0096] Among them, the embedding layer is a network structure located in the deep neural network sub-model, which is used to process sparse features. It reduces the dimension by calculating the weight matrix of the embedding algorithm, thereby achieving the purpose of reducing the dimension of sparse features.

[0097] Among them, the hidden layer is located in the deep neural network sub-model. In the deep neural network sub-model, except for the input layer and the output layer, all other layers are hidden layers. The hidden layer does not directly receive external signals, nor does it directly send signals to the outside world. The hidden layer abstracts the input features at multiple levels and finally linearly divides the input features into different types of data.

[0098] In practical applications, for example, the sample label information can be input into a deep neural network sub-model, and the sample label information can be converted into a sample label feature vector through an embedding layer. The sample label feature vector can be expressed in the form of a vector, and then through a hidden layer, the label prediction information of the browsing user for the sample text is predicted.

[0099] 203. Based on the sample feature information and the user feedback information, obtain the loss corresponding to the user feedback information of each dimension.

[0100] The objective function may be a function used to achieve the desired goal in machine learning. In machine learning, in order to achieve a certain goal, it is necessary to construct an objective function, and then let the function take a maximum or minimum value, so as to obtain the model parameters of the machine learning algorithm.

[0101] Among them, since the supervised model needs an objective function during model training, in order to take various user feedback situations into account in the model in the embodiments of the present application, the framework of multi-task learning can be used for reference to establish multiple objective functions. For example, the embodiments of the present application can construct an objective function around the core idea of ​​"feedback from users after reading the text with background music", and the constructed objective functions are all related to the user's post-reading behavior. For example, the objective function can be constructed based on the average browsing time of users, the number of user comments, the amount of user rewards, the number of user shares, etc. Among them, the objective function is not restricted in the embodiments of the present application, as long as it is an objective function that meets the core idea, and the type of objective function can be adjusted according to the actual situation.

[0102] In practical applications, for example, after obtaining user feedback information of multiple dimensions based on the text music model, multiple objective functions corresponding to the user feedback information of multiple dimensions can be constructed according to the user feedback information and the sample feedback information corresponding to the sample feature information, and then the objective functions can be solved to obtain the solved results.

[0103] 204. Based on the loss corresponding to the user feedback information in each dimension, the text music matching model is trained to obtain a trained text music matching model.

[0104] In practical applications, for example, after solving multiple objective functions, the parameters in the text music soundtrack model can be adjusted according to the solution results of the objective functions to achieve the purpose of training the text music soundtrack model. When the text music soundtrack model is trained to convergence, the trained text music soundtrack model can be obtained.

[0105] 205. Predict the target music of the text to be matched based on the trained text matching model.

[0106] In practical applications, for example, after the text music matching model is trained to obtain a trained text music matching model, the trained text music matching model can be used to predict the target music matching of the text to be matched.

[0107] In one embodiment, the target music can be determined from the music library by predicting the target user feedback information of each piece of music in the music library for the text to be matched. Specifically, the step of "predicting the target music for the text to be matched based on the trained text matching model" may include:

[0108] Based on the trained text music matching model, the music library, and the text to be matched with music, predict multi-dimensional target user feedback information of each piece of music in the music library for the text to be matched with music;

[0109] According to the target user feedback information, a target music for the text to be composed is determined from the music library.

[0110] In practical applications, for example, a music library can be obtained, which includes multiple pieces of music, and the trained text music model can determine the target music from the music library. It is also possible to obtain a text to be matched with music, and then use the trained text music model to predict the target user feedback information generated by each piece of music in the music library for the text to be matched based on the information in the music library and the text to be matched, and then determine the target music for the text to be matched from the music library based on the obtained target user feedback information.

[0111] In one embodiment, the features corresponding to the text to be matched and the music library can be extracted to facilitate the prediction of target user feedback information by the trained text matching model. Specifically, the step of "predicting multi-dimensional target user feedback information of each piece of music in the music library for the text to be matched based on the trained text matching model, the music library, and the text to be matched" may include:

[0112] Acquire a text to be matched with music and a plurality of text features corresponding to the text to be matched with music;

[0113] Acquire a music library and music features corresponding to a plurality of music pieces in the music library;

[0114] Based on the trained text music matching model, the text features, and the music features, multi-dimensional target user feedback information of each music in the music library for the text to be matched with music is predicted.

[0115] In practical applications, for example, Figure 4 As shown, the music label information and music attribute information corresponding to each piece of music in the music library can be obtained through the preset music library and music review library. After the author finishes writing a text to be matched with music, the text features corresponding to the text to be matched with music can be obtained by the same method as the model training. The text features may include text feature information, reader group feature information, author feature information, and context feature information. Specifically, the text features may include text label information, reader group label information, reader group attribute information, context attribute information, and author feature information. Then the music label information corresponding to each piece of music in the music library, the music attribute information corresponding to each piece of music in the music library, the text label information corresponding to the text to be matched with music, the reader group label information corresponding to the text to be matched with music, and the context attribute information corresponding to the text to be matched with music are input into the trained text matching model to predict the multi-dimensional target user feedback information of each piece of music in the music library for the text to be matched with music, such as one or more of the average browsing time of users, the number of user comments, the amount of user rewards, and the number of user shares.

[0116] In the model training phase, only the sample music that matches the sample text is input into the model together with the sample text for model training. In the model use phase, each piece of music in the music library can be input into the model together with the text to be matched with music to predict user feedback information. This method can predict the target user feedback information that may be generated when each piece of music in the music library is played as background music for the text to be matched with music.

[0117] In one embodiment, in order to improve the flexibility of target music selection, the target music can be selected based on the author's settings and user feedback information that the author is more concerned about. Specifically, the step of "determining the target music for the text to be matched with music from the music library based on the target user feedback information" may include:

[0118] Performing weighted fusion on the multi-dimensional target user feedback information corresponding to each piece of music in the music library to obtain fused user feedback information corresponding to each piece of music;

[0119] According to the fused user feedback information, a target music for the text to be composed is determined from the multiple pieces of music in the music library.

[0120] In practical applications, for example, after obtaining multi-dimensional target user feedback information corresponding to each piece of music in the music library based on the trained text soundtrack model, such as the average browsing time of the target users, the number of target user comments, the amount of target user rewards, and the number of target user shares corresponding to each piece of music in the music library, multiple pieces of music in the music library can be sorted according to the author's specifications and the target soundtrack can be recommended.

[0121] For example, after obtaining the target user's average browsing time, target user's comment times, target user's reward amount, and target user's sharing times corresponding to each piece of music in the music library, the author hopes to increase the user's reward amount. At this time, the author can set it to make recommendations based on the target user's reward amount. The system can sort multiple pieces of music in the music library according to the target user's reward amount predicted for each piece of music in the music library, and then recommend one or several pieces of music with high target user's reward amount as target soundtracks to the author. The author can select the soundtrack of the text to be matched from the recommended target soundtracks. In this way, the author can easily understand which background music to match the text to be matched with, which may get more rewards. The type of target user feedback information can be adjusted according to the actual situation, such as deleting user feedback information that the author does not care about, or adding new user feedback information.

[0122] For example, after obtaining the target user's average browsing time, target user comment times, target user reward amount, and target user sharing times corresponding to each piece of music in the music library, the author can also set the author's own attention level for each type of user feedback information. For example, the author can set the target user sharing times as the highest level of attention, followed by the target user reward amount, then the target user comment amount, and then the target user's average browsing time. Then, the system can set a corresponding weight for each user feedback information according to the degree of attention set by the author for each user feedback information. User feedback information with high attention can be given a higher weight, such as the weight of the number of target user shares can be set to 40%, the weight of the target user reward amount can be set to 30%, the weight of the number of target user comments can be set to 20%, and the weight of the average browsing time of the target user can be set to 10%. Then, according to the weight, the multiple target user feedback information is weighted and fused to obtain the fused user feedback information corresponding to each piece of music. Then, according to the fused user feedback information, multiple pieces of music in the music library are sorted, and one or several pieces of music with larger fused user feedback information are recommended to the author as target soundtracks. The author can select the soundtrack of the text to be matched from the recommended target soundtracks. In this way, the author can select the target music according to the degree of attention paid to different user feedback information.

[0123] In one embodiment, the author can not only specify the ranking of the author's attention level for each type of user feedback information, but also directly specify the weight of each type of user feedback information, such as the author can directly set the weight of each type of user feedback information. In addition, the system's allocation of weights is not limited to the above-mentioned weight values, as long as it can ensure that the weight required by the author is achieved.

[0124] This text music matching method can be applied in text music matching scenarios. On the one hand, it can save the author's time and assist the author in finding the target music that best matches the text to be matched. On the other hand, it can also help the author find more potential target music that can be used as background music through the system's intelligent recommendation.

[0125] In one embodiment, the text music matching method can be applied not only to the scenario of text music matching, but also to a multimodal chat system. For example, when a user is chatting through a chat application, the system can automatically recommend music or audio books for the user to choose from. The user can play music or audio books that match the current chat content according to the system's recommendation. For example, when the user enters "Happy Birthday" in the chat system, the system can automatically recommend songs related to Happy Birthday, such as "Happy Birthday to You", "Happy Birthday", and "Birthday". In this way, the user can choose a song to play, which greatly improves the user experience during the chat.

[0126] As can be seen from the above, the embodiment of the present application can obtain sample text and multi-dimensional sample feature information corresponding to the sample text, predict the multi-dimensional user feedback information of the browsing user for the sample text based on the text soundtrack model and the sample feature information, obtain the loss corresponding to the user feedback information of each dimension based on the sample feature information and the user feedback information, train the text soundtrack model based on the loss corresponding to the user feedback information of each dimension, obtain the trained text soundtrack model, and predict the target soundtrack of the text to be soundtracked based on the trained text soundtrack model. This scheme can improve the modeling ability of the text soundtrack model by taking the multi-dimensional sample feature information corresponding to the sample text as the model input and setting multiple optimization goals. Matching music to the soundtrack text according to the text soundtrack model can not only assist the author in finding suitable background music and save the author's time, but also find more potential music that can be used as background music for the author, so that the author's behavior of selecting target music is more flexible, and the accuracy of matching music for the text to be soundtracked is improved.

[0127] According to the method described in the previous embodiment, the text music matching device will be further described in detail below by taking the example of being specifically integrated into a network device.

[0128] refer to Figure 3 The specific process of the text music matching method of the embodiment of the present application can be as follows:

[0129] 301. A network device obtains a training sample, where the training sample includes a sample text, sample music matching the sample text, and a sample reader group that has browsed the sample text.

[0130] In practical applications, for example, texts with music already arranged can be extracted from a database as sample texts, and songs matching the sample texts can be obtained as sample music, as well as information about a sample reader group that has browsed the sample texts. The sample texts, sample music, and sample reader group can together constitute a training sample.

[0131] In one embodiment, the accuracy of the text music model can be improved by enriching the training samples. Therefore, multiple sample texts can be obtained from the database to form multiple training samples for training the text music model. In addition, the sample music that matches a sample text is not limited to one piece, that is, the sample music is not limited to one piece of music that has been used as the background music of the sample text in history. Multiple pieces of music that are related to the content of the sample text and can be used as the background music of the sample text can also be used as sample music to enrich the training samples. At the same time, in order to avoid the individual behavior of a single reader affecting the effect of the model training, the sample reader group information can include multiple readers who have browsed the sample text. By expanding the number of sample reader groups, the generality and accuracy of the sample reader groups are guaranteed.

[0132] In one embodiment, in order to construct objective functions in multiple dimensions according to various user feedback information predicted by the text music matching model and real sample feedback information during the text music matching model training process, and optimize the target through the objective functions in multiple dimensions, therefore, in the training stage, it is necessary to collect various sample feedback information after readers who have browsed the sample text, and the collected information is the real sample feedback information corresponding to the sample text, such as the average browsing time of readers, the number of reader comments, the amount of reader rewards, and the number of reader shares. These real sample feedback information can be reflected as labels of the training model.

[0133] 302. The network device obtains sample feature information, where the sample feature information includes music tag information, reader group tag information, text tag information, music attribute information, reader group attribute information, and context attribute information.

[0134] In practical applications, for example, after obtaining sample text, sample music, and sample reader groups, music label information, reader group label information, music attribute information, and reader group attribute information can be obtained by searching the pre-set music library, music review library, and reader portrait library. Since the sample text in the database is constantly increasing, there is no need to establish a database in advance, but to directly mine the sample text to obtain text label information and context attribute information.

[0135] For example, Figure 5 As shown, you can search the music library and the music review library, and mine the music tag information through the music name, lyrics, music reviews, etc.; by searching the music library, you can get the music attribute information; search the reader portrait library, and mine the reader group tag information and reader group attribute information from the texts such as the articles the readers have read and the comments they have posted.

[0136] Among them, tag information is a feature expressed in text form. Tag information can be expressed in the form of words. For example, music tag information can be "jazz", "sadness", "instrumental music", "listen when you break up", "quiet", etc.; reader group tag information can be "secondary", "middle-aged people", "cool", etc.; text tag information can be "emotional", "historical", "gossip", etc.

[0137] Among them, attribute information is a feature expressed in the form of a specific category, which can be expressed in the form of "feature name and feature value". For example, music attribute information can be "duration 20 minutes", "category Mandarin", "mode C major", etc.; reader group attribute information can be "average age 28 years old", "average height 170cm", "major cities Beijing / Shanghai", etc.; context attribute information can be "publication time January 1, 2019", "publication platform xx website", etc.

[0138] 303. The network device inputs the sample feature information into the text music matching model to predict multi-dimensional user feedback information corresponding to the sample text.

[0139] In practical applications, for example, when the text soundtrack model is a wide&deep model, the wide&deep model includes a wide part and a deep part, where the deep part is a deep neural network used to process text features such as label information, do embedding and forward propagation, and the wide part is a simple shallow model, such as logistic regression, SVM, etc., used to process non-text features such as attribute information. Therefore, the sample feature information can be divided into sample label information and sample attribute information, where the sample label information includes music label information, reader group label information, and text label information; the sample attribute information includes music attribute information, reader group attribute information, and context attribute information.

[0140] Then, the sample label information is input into the deep part of the wide&deep model, and the sample label information is converted into a sample label feature vector in vector form through the embedding algorithm. Then, the label prediction information corresponding to the sample label information is predicted through the multi-layer neural network. The sample attribute information is input into the wide part of the wide&deep model, and the attribute prediction information corresponding to the sample attribute information is predicted through the linear model of the wide part. Then, the obtained label prediction information and attribute prediction information are fused to obtain multi-dimensional user feedback information.

[0141] Among them, the embodiment of the present application does not limit the structure or type of the text soundtrack model. As long as it is a supervised model, a network model that can predict multi-dimensional user feedback information corresponding to the sample text can be used.

[0142] 304. The network device constructs multiple objective functions according to the multi-dimensional user feedback information and the sample labels corresponding to the training samples.

[0143] In practical applications, since the objective function is needed to enable the model to obtain the required target during the training of the network model, the embodiment of the present application can draw on the framework of multi-task learning and build multiple objective functions related to the user's post-reading behavior around the core idea of ​​"feedback from users after reading articles with background music".

[0144] For example, after obtaining multi-dimensional user feedback information, it can be compared with the sample feedback information corresponding to the training sample label, and based on the multi-dimensional user feedback information and the sample feedback information corresponding to the training sample, a multi-dimensional objective function is constructed, where each objective function represents the optimization of a type of user feedback information, that is, multiple objective functions represent the optimization of user feedback information such as the average browsing time of readers, the number of reader comments, the amount of reader rewards, and the number of reader shares. In order to improve the flexibility of the text soundtrack model, the type of objective function can be adjusted according to the actual situation. As long as the objective function meets the core idea of ​​"user feedback after reading the article with background music", it can be included in the system.

[0145] 305. The network device solves multiple objective functions and adjusts the parameters of the text-music matching model according to the solution results to obtain a trained text-music matching model.

[0146] In practical applications, for example, after constructing multiple objective functions, the multiple objective functions can be solved, and according to the solution results of the objective functions, the parameters in the text music model can be adjusted until the text music model converges to obtain the trained text music model.

[0147] 306. The network device obtains text features corresponding to the text to be matched with music, and music features corresponding to each piece of music in the music library.

[0148] In practical applications, for example, after the model training is completed and the trained text music model is obtained, the trained text music model can be used to predict the target music for the music text. The music text to be matched with the background music can be obtained, and the text features corresponding to the music text to be matched can be mined. The text features can include text label information, reader group label information, reader group attribute information, context attribute information, and author feature information corresponding to the music text to be matched. In addition, a music library can be obtained, which includes multiple music pieces, and then the music features corresponding to each music piece in the music library can be mined according to the preset music library, music review library, and reader portrait library. The music features can include music label information and music attribute information corresponding to each music piece in the music library.

[0149] 307. The network device inputs the text features and the music features into the trained text-music model to predict and obtain multi-dimensional target user feedback information.

[0150] In practical applications, for example, the text features corresponding to the text to be matched with music and the music features corresponding to each piece of music in the music library can be input into the trained text matching model. Then, based on the trained text matching model, the multi-dimensional target user feedback information of each piece of music in the music library for the text to be matched with music can be predicted.

[0151] For example, if the music library includes multiple pieces of music such as music 1, music 2, music 3, etc., the trained text music model can predict the multi-dimensional target user feedback information when music 1 is used as the background music of the text to be matched; the multi-dimensional target user feedback information when music 2 is used as the background music of the text to be matched; the multi-dimensional target user feedback information when music 3 is used as the background music of the text to be matched... Among them, the multi-dimensional target user feedback information includes the average browsing time of readers, the number of reader comments, the amount of reader rewards, and the number of reader shares.

[0152] Among them, when training the text soundtrack model, the music input as training samples is the sample music that matches the sample text. When applying the trained text soundtrack model for information prediction, the input music is not limited to the music that matches the text to be soundtracked, but all the music in the music library can be input. At this time, multi-dimensional target user feedback information for each music in the music library for the text to be soundtracked can be obtained.

[0153] 308. The network device determines a target soundtrack from multiple pieces of music in the music library according to the target user feedback information.

[0154] In practical applications, for example, after obtaining multi-dimensional target user feedback information for each piece of music in the music library for the text to be matched with the music, a variety of sorting methods can be provided for the author of the text to be matched with the music. When the author of the text to be matched with the music is more concerned about the amount of user rewards, the multiple pieces of music in the music library can be sorted according to the value of the user's reward amount, and one or several pieces of music in the music library can be recommended to the author as target music, so that the author can select more suitable music as the background music for the text to be matched with the music.

[0155] In one embodiment, for example, after obtaining multi-dimensional target user feedback information for each piece of music in the music library for the text to be matched with music, the multi-dimensional target user feedback information can be fused according to the weights specified by the author of the text to be matched with music to obtain fused user feedback information, and then the multiple pieces of music in the music library are sorted according to the values ​​of the fused user feedback information, and one or several pieces of music in the music library are recommended to the author as target music, so that the author can select more suitable music as background music for the text to be matched with music.

[0156] As can be seen from the above, the embodiment of the present application can obtain training samples through a network device, the training samples include sample text, sample music matching the sample text, and sample readers who have browsed the sample text, obtain sample feature information, the sample feature information includes music tag information, reader tag information, text tag information, music attribute information, reader attribute information, and context attribute information, input the sample feature information into the text soundtrack model, predict the multi-dimensional user feedback information corresponding to the sample text, and construct multiple objective functions according to the multi-dimensional user feedback information and the sample labels corresponding to the training samples, solve the multiple objective functions, and adjust the parameters of the text soundtrack model according to the solution results to obtain the trained text soundtrack model, obtain the text features corresponding to the text to be soundtracked, and the music features corresponding to each piece of music in the music library, input the text features and music features into the trained text soundtrack model, predict the multi-dimensional target user feedback information, and determine the target soundtrack from multiple pieces of music in the music library according to the target user feedback information. This scheme can improve the modeling ability of the text soundtrack model by using the multi-dimensional sample feature information corresponding to the sample text as the model input and setting multiple optimization goals. Matching music for the soundtrack text according to the text soundtrack model can not only assist the author in finding suitable background music and save the author's time, but also help the author find more potential music that can be used as background music, making the author's behavior of selecting target music more flexible and improving the accuracy of matching music for the text to be soundtracked.

[0157] In order to better implement the above method, an embodiment of the present application may also provide a text music arrangement device, which may be specifically integrated in a network device, which may include a server, a terminal, etc., wherein the terminal may include: a mobile phone, a tablet computer, a laptop computer or a personal computer (PC, Personal Computer), etc.

[0158] For example, Figure 6 As shown, the text music matching device may include an acquisition module 61, a first prediction module 62, a loss acquisition module 63, a training module 64 and a second prediction module 65, as follows:

[0159] An acquisition module 61 is used to acquire sample text and multi-dimensional sample feature information corresponding to the sample text;

[0160] A first prediction module 62, configured to predict multi-dimensional user feedback information of browsing users on the sample text based on the text music matching model and the sample feature information;

[0161] A loss acquisition module 63, configured to acquire the loss corresponding to the user feedback information of each dimension based on the sample feature information and the user feedback information;

[0162] A training module 64 is used to train the text-music matching model based on the loss corresponding to the user feedback information of each dimension to obtain a trained text-music matching model;

[0163] The second prediction module 65 is used to predict the target music of the text to be composed based on the trained text music composition model.

[0164] In one embodiment, the acquisition module 61 may include an acquisition submodule 611 and an extraction submodule 612, as follows:

[0165] An acquisition submodule 611 is used to acquire a sample text and a plurality of sample soundtrack information corresponding to the sample text;

[0166] The extraction submodule 612 is used to extract the features of the sample text and the sample music information to obtain multi-dimensional sample feature information.

[0167] In one embodiment, the extraction submodule 612 may be specifically used for:

[0168] Extracting sample music feature information corresponding to the sample music information based on a preset database;

[0169] The features of the sample text are extracted to obtain sample text feature information corresponding to the sample text.

[0170] In one embodiment, the first prediction module 62 may include a first prediction submodule 621, a second prediction submodule 622 and a fusion submodule 623, as follows:

[0171] A first prediction submodule 621, configured to predict attribute prediction information of a browsing user for the sample text based on the linear submodel and the sample attribute information;

[0172] A second prediction submodule 622, configured to predict label prediction information of a browsing user for the sample text based on the deep neural network submodel and the sample label information;

[0173] The fusion submodule 623 is used to fuse the attribute prediction information and the label prediction information to obtain multi-dimensional user feedback information.

[0174] In one embodiment, the second prediction submodule 622 may be specifically used for:

[0175] Converting the sample label information into a sample label feature vector;

[0176] Based on the deep neural network sub-model and the sample label feature vector, label prediction information of the browsing user for the sample text is predicted.

[0177] In one embodiment, the second prediction module 65 may include a third prediction submodule 651 and a determination submodule 652 as follows:

[0178] The third prediction submodule 651 is used to predict multi-dimensional target user feedback information of each piece of music in the music library for the text to be matched with music based on the trained text matching model, the music library, and the text to be matched with music;

[0179] The determination submodule 652 is used to determine the target music for the text to be composed from the music library according to the target user feedback information.

[0180] In one embodiment, the third prediction submodule 651 may be specifically used for:

[0181] Acquire a text to be matched with music and a plurality of text features corresponding to the text to be matched with music;

[0182] Acquire a music library and music features corresponding to a plurality of music pieces in the music library;

[0183] Based on the trained text music matching model, the text features, and the music features, multi-dimensional target user feedback information of each music in the music library for the text to be matched with music is predicted.

[0184] In one embodiment, the determining submodule 652 may be specifically used for:

[0185] Performing weighted fusion on the multi-dimensional target user feedback information corresponding to each piece of music in the music library to obtain fused user feedback information corresponding to each piece of music;

[0186] According to the fused user feedback information, a target music for the text to be composed is determined from the multiple pieces of music in the music library.

[0187] In specific implementation, the above units can be implemented as independent entities, or can be arbitrarily combined to be implemented as the same or several entities. The specific implementation of the above units can refer to the previous method embodiments, which will not be repeated here.

[0188] As can be seen from the above, the embodiment of the present application can obtain sample text and multi-dimensional sample feature information corresponding to the sample text through the acquisition module 61, predict the multi-dimensional user feedback information of the browsing user for the sample text based on the text soundtrack model and the sample feature information through the first prediction module 62, obtain the loss corresponding to the user feedback information of each dimension based on the sample feature information and the user feedback information through the loss acquisition module 63, train the text soundtrack model based on the loss corresponding to the user feedback information of each dimension through the training module 64, and obtain the trained text soundtrack model, and predict the target soundtrack of the text to be soundtracked based on the trained text soundtrack model through the second prediction module 65. This scheme can improve the modeling ability of the text soundtrack model by taking the multi-dimensional sample feature information corresponding to the sample text as the model input and setting multiple optimization targets. Matching music to the soundtrack text according to the text soundtrack model can not only assist the author in finding suitable background music and save the author's time, but also find more potential music that can be used as background music for the author, so that the author's behavior of selecting target music is more flexible, and the accuracy of matching music for the text to be soundtracked is improved.

[0189] The embodiment of the present application also provides a network device, which can integrate any text music matching device provided in the embodiment of the present application.

[0190] For example, Figure 7 As shown, it shows a schematic diagram of the structure of the network device involved in the embodiment of the present application, specifically:

[0191] The network device may include one or more processing core processors 71, one or more computer-readable storage media memories 72, a power supply 73, an input unit 74 and other components. Those skilled in the art will appreciate that Figure 7 The network device structure shown in the figure does not constitute a limitation on the network device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0192] The processor 71 is the control center of the network device. It uses various interfaces and lines to connect various parts of the entire network device. By running or executing software programs and / or modules stored in the memory 72 and calling data stored in the memory 72, it executes various functions of the network device and processes data, thereby monitoring the network device as a whole. Optionally, the processor 71 may include one or more processing cores; preferably, the processor 71 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 71.

[0193] The memory 72 can be used to store software programs and modules. The processor 71 executes various functional applications and data processing by running the software programs and modules stored in the memory 72. The memory 72 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the network device, etc. In addition, the memory 72 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 72 may also include a memory controller to provide the processor 71 with access to the memory 72.

[0194] The network device also includes a power supply 73 for supplying power to each component. Preferably, the power supply 73 can be logically connected to the processor 71 through a power management system, so as to manage charging, discharging, and power consumption through the power management system. The power supply 73 can also include one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0195] The network device may further include an input unit 74, which may be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function controls.

[0196] Although not shown, the network device may also include a display unit, etc., which will not be described in detail here. Specifically in this embodiment, the processor 71 in the network device will load the executable files corresponding to the processes of one or more applications into the memory 72 according to the following instructions, and the processor 71 will run the application stored in the memory 72 to achieve various functions, as follows:

[0197] Obtain sample text and multi-dimensional sample feature information corresponding to the sample text, predict multi-dimensional user feedback information of browsing users for the sample text based on the text music matching model and the sample feature information, obtain the loss corresponding to the user feedback information of each dimension based on the sample feature information and the user feedback information, train the text music matching model based on the loss corresponding to the user feedback information of each dimension, obtain the trained text music matching model, and predict the target music matching of the text to be matched based on the trained text music matching model.

[0198] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0199] As can be seen from the above, the embodiment of the present application can obtain sample text and multi-dimensional sample feature information corresponding to the sample text, predict the multi-dimensional user feedback information of the browsing user for the sample text based on the text soundtrack model and the sample feature information, obtain the loss corresponding to the user feedback information of each dimension based on the sample feature information and the user feedback information, train the text soundtrack model based on the loss corresponding to the user feedback information of each dimension, obtain the trained text soundtrack model, and predict the target soundtrack of the text to be soundtracked based on the trained text soundtrack model. This scheme can improve the modeling ability of the text soundtrack model by taking the multi-dimensional sample feature information corresponding to the sample text as the model input and setting multiple optimization goals. Matching music to the soundtrack text according to the text soundtrack model can not only assist the author in finding suitable background music and save the author's time, but also find more potential music that can be used as background music for the author, so that the author's behavior of selecting target music is more flexible, and the accuracy of matching music for the text to be soundtracked is improved.

[0200] A person of ordinary skill in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0201] To this end, an embodiment of the present application provides a computer device in which a plurality of instructions are stored, and the instructions can be loaded by a processor to execute the steps in any of the text music matching methods provided in the embodiments of the present application. For example, the instructions can execute the following steps:

[0202] Obtain sample text and multi-dimensional sample feature information corresponding to the sample text, predict multi-dimensional user feedback information of browsing users for the sample text based on the text music matching model and the sample feature information, obtain the loss corresponding to the user feedback information of each dimension based on the sample feature information and the user feedback information, train the text music matching model based on the loss corresponding to the user feedback information of each dimension, obtain the trained text music matching model, and predict the target music matching of the text to be matched based on the trained text music matching model.

[0203] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0204] The storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0205] Since the instructions stored in the storage medium can execute the steps in any text music arrangement method provided in the embodiments of the present application, the beneficial effects that can be achieved by any text music arrangement method provided in the embodiments of the present application can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0206] The above is a detailed introduction to a text music arrangement method, device, and computer storage medium provided in the embodiments of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for technical personnel in this field, according to the ideas of the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A method for composing music to a text, characterized in that: include: Acquiring sample text and multi-dimensional sample feature information corresponding to the sample text, including: acquiring sample text and multiple sample music information corresponding to the sample text; extracting features of the sample text and the sample music information to obtain multi-dimensional sample feature information, wherein the sample text is a text that has been matched with music and can be used as a sample for model training; Based on the text music model and the sample feature information, predict multi-dimensional user feedback information of browsing users for the sample text, wherein the user feedback information is information related to the feedback of users after reading the text with background music, including one or more of the average browsing time of users, the number of user comments, the amount of user rewards, and the number of user shares; Based on the sample feature information and the user feedback information, obtaining the loss corresponding to the user feedback information of each dimension; Based on the loss corresponding to the user feedback information of each dimension, the text-music matching model is trained to obtain a trained text-music matching model; Predicting a target music for the text to be matched with music based on the trained text music matching model includes: predicting multi-dimensional target user feedback information of each piece of music in the music library for the text to be matched with music based on the trained text music matching model, a music library, and the text to be matched with music; determining the target music for the text to be matched with music from the music library according to the target user feedback information; Among them, based on the trained text music arrangement model, the music library, and the text to be accompanied by music, predicting multi-dimensional target user feedback information of each piece of music in the music library for the text to be accompanied by music, including: obtaining the text to be accompanied by music, and multiple text features corresponding to the text to be accompanied by music; obtaining the music library and music features corresponding to multiple pieces of music in the music library; based on the trained text music arrangement model, the text features, and the music features, predicting multi-dimensional target user feedback information of each piece of music in the music library for the text to be accompanied by music.

2. The text music matching method according to claim 1, characterized in that: The sample feature information includes sample text feature information and sample music feature information; Extracting features of the sample text and the sample music information to obtain multi-dimensional sample feature information includes: Extracting sample music feature information corresponding to the sample music information based on a preset database; The features of the sample text are extracted to obtain sample text feature information corresponding to the sample text.

3. The text music matching method according to claim 1, characterized in that: The sample feature information includes sample label information and sample attribute information; The text music matching model includes a deep neural network sub-model and a linear sub-model; Based on the text music model and the sample feature information, predict multi-dimensional user feedback information of browsing users on the sample text, including: Based on the linear sub-model and the sample attribute information, predicting attribute prediction information of the browsing user for the sample text; Based on the deep neural network sub-model and the sample label information, predicting label prediction information of the browsing user for the sample text; The attribute prediction information and the label prediction information are integrated to obtain multi-dimensional user feedback information.

4. The text music matching method according to claim 3, characterized in that: Based on the deep neural network sub-model and the sample label information, predicting label prediction information of the browsing user for the sample text includes: Converting the sample label information into a sample label feature vector; Based on the deep neural network sub-model and the sample label feature vector, label prediction information of the browsing user for the sample text is predicted.

5. The text music matching method according to claim 1, characterized in that: Determining a target music for the text to be composed of music from the music library according to the target user feedback information includes: Performing weighted fusion on the multi-dimensional target user feedback information corresponding to each piece of music in the music library to obtain fused user feedback information corresponding to each piece of music; According to the fused user feedback information, a target music for the text to be composed is determined from the multiple pieces of music in the music library.

6. A text music matching device, characterized in that: include: An acquisition model is used to acquire sample text and multi-dimensional sample feature information corresponding to the sample text, including: acquiring sample text and multiple sample music information corresponding to the sample text; extracting features of the sample text and the sample music information to obtain multi-dimensional sample feature information, wherein the sample text is a text that has been matched with music and can be used as a sample for model training; A first prediction model is used to predict multi-dimensional user feedback information of browsing users on the sample text based on the text music model and the sample feature information, wherein the user feedback information is information related to the feedback of users after reading the text with background music, including one or more of the average browsing time of users, the number of user comments, the amount of user rewards, and the number of user shares; A loss acquisition model, used to acquire the loss corresponding to the user feedback information of each dimension based on the sample feature information and the user feedback information; A training model is used to train the text-music matching model based on the loss corresponding to the user feedback information of each dimension to obtain a trained text-music matching model; A second prediction model is used to predict a target soundtrack for the text to be matched with music based on the trained text soundtrack model; The second prediction model is also used to predict multi-dimensional target user feedback information of each piece of music in the music library for the text to be accompanied by music based on the trained text music arrangement model, the music library, and the text to be accompanied by music; determine the target music for the text to be accompanied by music from the music library according to the target user feedback information; wherein, based on the trained text music arrangement model, the music library, and the text to be accompanied by music, predicting multi-dimensional target user feedback information of each piece of music in the music library for the text to be accompanied by music includes: obtaining the text to be accompanied by music, and multiple text features corresponding to the text to be accompanied by music; obtaining the music library and music features corresponding to multiple pieces of music in the music library; based on the trained text music arrangement model, the text features, and the music features, predicting multi-dimensional target user feedback information of each piece of music in the music library for the text to be accompanied by music.

7. A computer storage medium having a computer program stored thereon, characterized in that: When the computer program is run on a computer, the computer is enabled to execute the text music matching method according to any one of claims 1 to 5.

Images