Music generation method, electronic device, storage device and product

Abstract

The present disclosure relates to a music generation method, electronic device, storage device and product, and relates to the field of computer technology. The music generation method includes: displaying a first message sent by a user in a dialogue interface between a user and an intelligent agent; generating music features corresponding to one or more music elements based on the first message; determining prompt information according to the music features; generating music based on the prompt information; and displaying the music sent by the intelligent agent to the user. The embodiments of the present disclosure improve the efficiency and flexibility of generating music.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a music generation method, electronic device, storage device and product. Background Art

[0002] A generative model is one that can create samples similar to pre-learned samples based on input, thereby improving the efficiency of sample generation. In fields such as machine learning, technicians can use generative models to generate training samples for other machine learning models to enrich the training set of machine learning models and reduce the workload of manual labeling.

[0003] With the development of computer technology and artificial intelligence technology, the application of generative models is becoming more and more extensive. It is no longer limited to technicians. More users can use generative models for creation. For example, ordinary users can also use the open interfaces of various applications to create text, images, or music. Summary of the invention

[0004] This summary is provided to introduce concepts in a brief form that will be described in detail in the detailed description below. This summary is not intended to identify key features or essential features of the claimed technical solution, nor is it intended to limit the scope of the claimed technical solution.

[0005] According to some embodiments of the present disclosure, a music generation method is provided, comprising: displaying a first message sent by the user in a dialogue interface between the user and an intelligent agent; generating music features corresponding to one or more music elements based on the first message; determining prompt information based on the music features; generating music based on the prompt information; and displaying the music sent by the intelligent agent to the user.

[0006] According to some embodiments of the present disclosure, an electronic device is provided, comprising: a memory; and a processor coupled to the memory, wherein the processor is configured to execute the music generation method of any embodiment described in the present disclosure based on instructions stored in the memory.

[0007] According to some embodiments of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the program is executed by a processor, the music generation method of any embodiment described in the present disclosure is performed.

[0008] According to some embodiments of the present disclosure, a computer program product is provided. When the computer program product is executed on a computer, the computer implements the music generating method of any embodiment described in the present disclosure.

[0009] Other features, aspects and advantages of the present disclosure will become apparent from the following detailed description of exemplary embodiments of the present disclosure with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The preferred embodiments of the present disclosure are described below with reference to the accompanying drawings. The drawings described herein are used to provide a further understanding of the present disclosure, and each drawing together with the following detailed description is included in this specification and forms a part of the specification for explaining the present disclosure. It should be understood that the drawings described below only relate to some embodiments of the present disclosure and do not constitute a limitation on the present disclosure.

[0011] Figure 1 A flowchart of a music generation method according to some embodiments of the present disclosure is shown.

[0012] Figure 2 A flowchart of a method for generating music features according to some embodiments of the present disclosure is shown.

[0013] Figure 3 A flowchart of a method for supplementing music features according to some embodiments of the present disclosure is shown.

[0014] Figure 4 A flowchart of a method for identifying music generation intent according to some embodiments of the present disclosure is shown.

[0015] Figure 5 A flowchart of a music adjustment method according to some embodiments of the present disclosure is shown.

[0016] Figure 6 A schematic diagram of a conversation interface according to some embodiments of the present disclosure is exemplarily shown.

[0017] Figure 7 A schematic structural diagram of a music generating device according to some embodiments of the present disclosure is shown.

[0018] Figure 8 A schematic structural diagram of an electronic device according to some embodiments of the present disclosure is shown.

[0019] Fig. 9 A schematic diagram of the structure of a computer system according to some embodiments of the present disclosure is shown.

[0020] It should be understood that, for ease of description, the sizes of the various parts shown in the drawings are not necessarily drawn according to the actual proportional relationship. The same or similar reference numerals are used in the various drawings to represent the same or similar parts. Therefore, once an item is defined in one drawing, it may not be further discussed in subsequent drawings. DETAILED DESCRIPTION

[0021] The following will be combined with the drawings in the embodiments of the present disclosure to clearly and completely describe the technical solutions in the embodiments of the present disclosure, but it is obvious that the described embodiments are only part of the embodiments of the present disclosure, rather than all the embodiments. The following description of the embodiments is actually only illustrative and is by no means intended to limit the present disclosure and its application or use. It should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein.

[0022] It should be understood that the various steps recorded in the method embodiments of the present disclosure can be performed in different orders, and / or performed in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown in the execution. The scope of the present disclosure is not limited in this respect. Unless otherwise specifically stated, the relative arrangement, numerical expressions and numerical values ​​of the parts and steps set forth in these embodiments should be interpreted as being merely exemplary and do not limit the scope of the present disclosure.

[0023] The term "include" and its variations used in the present disclosure mean an open term that includes at least the following elements / features but does not exclude other elements / features, that is, "including but not limited to". In addition, the term "include" and its variations used in the present disclosure mean an open term that includes at least the following elements / features but does not exclude other elements / features, that is, "including but not limited to". Therefore, include is synonymous with include. The term "based on" means "based at least in part on".

[0024] References throughout this specification to "one embodiment," "some embodiments," or "an embodiment" mean that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present invention. For example, the term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; and the term "some embodiments" means "at least some embodiments." Furthermore, the appearances of the phrases "in one embodiment," "in some embodiments," or "in an embodiment" in various places throughout this specification do not necessarily all refer to the same embodiment, but may do so.

[0025] It should be noted that the concepts of "first", "second", etc. mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units. Unless otherwise specified, the concepts of "first", "second", etc. are not intended to imply that the objects described in this way must be in a given order in time, space, ranking, or any other manner.

[0026] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0027] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0028] The embodiments of the present disclosure are described in detail below in conjunction with the accompanying drawings, but the present disclosure is not limited to these specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. In addition, in one or more embodiments, specific features, structures or characteristics can be combined in any suitable manner that will be clear from the present disclosure by a person of ordinary skill in the art.

[0029] When using machine learning models such as generative models to automatically generate various works, users often need to provide prompts. The model will process the prompts and generate the works that the user wants by combining the model's knowledge base.

[0030] Compared with text and images, music is more complex, for example, it involves multiple dimensions such as rhythm, beat, melody, mode, instruments, voice, emotion, atmosphere, etc. Music is not a visual object, and non-music professionals may find it difficult to accurately and clearly describe the characteristics of the music they want to generate.

[0031] Therefore, in the related art, music generation is relatively difficult and inefficient.

[0032] The present disclosure provides a music generation method, which mines the music features that the user wants to use in the music to be generated through the dialogue between the user and the intelligent agent, and then generates music based on these music features.

[0033] The agent can generate corresponding content based on the dialogue sent by other subjects in the dialogue scene (such as one or more users, other agents participating in the dialogue). It can be implemented in software, hardware, or a combination of software and hardware. The agent can also be called a robot, a digital person, or a virtual agent of a machine learning model. The agent can be implemented based on one or more machine learning models, such as a large language model (LLM) or a foundation model. The machine learning model can be a generative model, which is used to output target content based on input information. The information input to the generative model includes the processing basis of the generative model in the generation process, such as which information is referenced to perform the generation process, the requirements of the output target content, etc. The generative model includes, for example, a model generated based on text, or a model generated based on images, and the output of the generative model can include one or more of text, images, and music. Of course, the input or output of the generative model can also be data of other modalities, such as audio, video, or a combination of multiple types of data. A generative model can be a single-modal model, such as a model that generates text from text (referred to as a "text-to-text model") or a model that generates images from images (referred to as a "image-to-image model"). Alternatively, a generative model can be a cross-modal model, that is, a model whose input and output belong to different modalities, such as a model that generates music from text. Alternatively, the input of a generative model can include multiple modalities, and the output can also include multiple modalities.

[0034] Figure 1 FIG. 1 is a flow chart showing a method for generating music according to some embodiments of the present disclosure. Figure 1 As shown, the music generation method of this embodiment includes steps S102 to S110.

[0035] In step S102, a first message sent by the user is displayed on the dialogue interface between the user and the agent. The first message is, for example, a message sent by the user to the agent.

[0036] The first message sent by the user can be in unstructured language. For example, the user can provide a structured natural language description such as "rhythm: fast; instrument: piano; atmosphere: relaxed" without following the existing template. The user can provide a natural language sentence and describe the creation needs in unstructured language, such as "help me generate a piece of content that describes a person walking on a rainy street, feeling a little sad." In this way, the constraints on the information provided by the user can be reduced, allowing the user to input more freely.

[0037] In step S104, a music feature corresponding to one or more music elements is generated based on the first message.

[0038] Musical elements can also be called musical elements or musical feature items. Musical elements refer to some reference information when generating music. One or more musical elements include, for example, at least one of the following: style, timbre, melody, rhythm, atmosphere, instrument, and lyrics information. Other types of information may also be included as needed, which will not be described in detail here.

[0039] The musical characteristics of a musical element can be understood as the specific content of the musical element. For example, for the musical element "music style", the corresponding musical characteristics may be "classical", "popular", "rock", "folk", etc.

[0040] Some or all of these music features may be directly included in the first message, or some or all of these music features may be indirectly obtained through the first message. For example, the music features corresponding to the words and sentences in the first message may be obtained by semantically understanding or mapping the words and sentences. Therefore, even if the user does not use professional vocabulary in the first message, if the user intends to express the music features but the description in the first message is not very accurate, a more accurate description may be generated based on the first message.

[0041] For example, the content of the first message sent by the user is "Generate a piece of music suitable for walking on the street on a rainy day", then based on the message, the musical feature of the music element "instrument" can be determined to be "piano", the musical feature of the music element "atmosphere" can be determined to be "sad", the musical feature of the music element "style" can be determined to be "popular", and so on.

[0042] Of course, music features can be expressed in natural language or in non-natural language (but can correspond to natural language, such as using codes or vectors to represent the content of natural speech), and this application does not impose any restrictions on this.

[0043] Through step S104, structured music features can be extracted from the unstructured first message, that is, the elements contained in the first message can be disassembled, understood, and information extracted to form music features of various music elements.

[0044] In step S106, prompt information is determined according to the music features.

[0045] The prompt information can be used as an input to a generative model for generating music. The prompt information can be the music feature itself, or it can be the result of processing the music feature in a specified manner, such as formatting, etc. Alternatively, other methods can be used to convert the music feature into the prompt information.

[0046] When the input of the generative model is natural language, the prompt information is natural language. Of course, the specific form of the prompt information can be determined according to the requirements of the generative model.

[0047] In step S108, music is generated based on the prompt information.

[0048] For example, the prompt information can be input into a generative model for generating music to obtain music output by the model. The generative model can generate music matching each music element according to each music element represented by the prompt information.

[0049] During the training process, the generative model can learn the correspondence between each music sample and the prompt information reflecting the characteristics of the music. Therefore, when generating music, the content of the music can be determined based on the prompt information. For example, first determine the overall rhythm and beat, then determine the melody (such as the pitch and duration of each phrase), and then determine the instrument to perform each phrase, and then generate preliminary music based on this basic information. Then add sound effects, vocals, etc. Finally, modify or adjust as needed.

[0050] In step S110, the music sent by the agent to the user is displayed.

[0051] For example, music can be played directly to the user and information corresponding to the music can be displayed; for another example, a link corresponding to the music can be displayed so that the user can obtain or play the music after accessing the link; for another example, a playback control corresponding to the music can be displayed so that the user can play the music after performing a play operation on the playback control. The displayed content can be displayed in the form of a message sent by the agent to the user.

[0052] In addition, the cover, lyrics, title and other information of the music may also be displayed. The cover of the music may be an image generated based on the music or the first message. For example, prompt information may be generated based on the first message, and the prompt information may be processed using the Vincent graph model to generate the cover of the music. For another example, if the generated music includes lyrics, the cover of the music may also be generated based on the lyrics and the Vincent graph model.

[0053] The above embodiment can generate music according to the message sent by the user during the conversation between the user and the intelligent agent. In addition, the user can express the content of the music to be generated through the message sent, that is, express the demand using natural language sentences. The music features can be extracted from the message sent by the user, and prompt information can be generated based on it, and then the music is generated. In this way, the user can freely express the creation needs through natural language sentences, which improves the efficiency and flexibility of generating music.

[0054] In some embodiments, music features can be generated by semantic understanding. Figure 2 A method for generating music features according to some embodiments of the present disclosure is described.

[0055] Figure 2 FIG. 2 is a flow chart showing a method for generating music features according to some embodiments of the present disclosure. Figure 2 As shown, the generation method of this embodiment includes steps S202 to S204.

[0056] In step S202, one or more keywords are determined according to the result of semantic understanding of the first message.

[0057] The keywords determined according to the result of semantic understanding may be keywords originally included in the first message, or words expressing the same or similar meanings. That is, one or more keywords may be directly or indirectly extracted from the first message, but are all related to the actual meaning of the first message. For example, one or more keywords are extracted from the first message and the result of semantic understanding of the first message.

[0058] In some embodiments, the result of the semantic understanding of the first message can be considered as expressing the content of the first message from another perspective. The first message is expressed using a more standardized language, or the first message is expressed using vocabulary in a specified vocabulary. For example, the first message can be processed using a model for generating text, and the first message and processing instructions can be input into a text generation model for processing instructions. The processing instructions, for example, are to express the first message using a language related to music. The language related to music, for example, is some specified music features, and some commonly used words, and commonly used words include, for example, pronouns, verbs, conjunctions, and the like.

[0059] Since the content of the first message sent by the user may be limited, in some embodiments, the first message may be expanded and then keywords may be extracted from it. For example, for the text in the first message, the text model may be used to expand the text to generate the expanded text. For example, when the model is used to process the text, the model may be instructed to "expand the text in the language that describes music." Thus, when the information provided by the user is relatively limited, more information related to the intention that the user wants to express can be obtained through the expansion result.

[0060] In step S204, music features corresponding to one or more music elements are generated according to the keywords.

[0061] That is, determine which music element the keyword corresponds to. After determining the correspondence between the keyword and the music element, the keyword itself can be used as the content of the music feature, or the keyword can be converted (for example, expressed in a specified manner, or formatted) to obtain the music feature.

[0062] In some embodiments, for each keyword, a music element matching the keyword is determined from one or more music elements; for each music element, a music feature is generated according to the keyword matching the music element type.

[0063] The correspondence between the keywords and the music elements can be determined by a preset correspondence table, which includes the correspondence between one or more music elements and one or more words. Alternatively, the keywords can be classified by a classification model, and each category corresponds to a music element. Alternatively, the music element corresponding to the keyword can be determined by using a semantic understanding model.

[0064] There may be one or more keywords corresponding to each music element. For any music element, if there is one keyword corresponding to it, a music feature can be directly generated based on the keyword, for example, the keyword itself can be used as a music feature, or the keyword can be converted to generate a music feature. In the case where there are multiple keywords corresponding to a certain music element, these words can be retained in parallel as music features; or each keyword can be converted into a music feature and the converted results can be used together as music features; or multiple keywords can be fused to generate music features, for example, the keywords can be fused into a smaller number of words, or a phrase, or a sentence, and music features can be generated based on the fusion results.

[0065] Through the above embodiment, the user can use conversational natural language to describe the music that he wants to generate, so that the keywords can be determined by semantically understanding the first message sent by the user. In this way, even for users who do not have a deep understanding of professional knowledge of music, music can be efficiently generated for them, thereby improving the efficiency of user-generated music and user experience.

[0066] Since users may lack professional knowledge of music, or may not have thought about how to express their needs, the messages sent by users may not fully cover the music features required for generating music. In some embodiments, the characteristics of the dialogue scene can be used to guide users to supplement. Figure 3 An embodiment of a method for supplementing music features of the present disclosure is described.

[0067] Figure 3 FIG. 2 is a flow chart showing a method for supplementing music features according to some embodiments of the present disclosure. Figure 3 As shown, the method of this embodiment includes steps S302 to S304.

[0068] In step S302, based on the first message sent by the user, music features of one or more designated music elements are determined.

[0069] One or more designated music elements may be necessary music elements for generating music, or important music elements. Which music elements belong to the designated music elements may be configured in advance.

[0070] Although the music features can be mined from the first message in the manner of the above embodiment, sometimes the music features of some music elements cannot be obtained. For example, when the information contained in the first message is too little, the music features of some music elements may not be mined.

[0071] In step S304, in response to the existence of at least one target music element without music characteristics, the first guidance information sent by the agent to the user is displayed. The first guidance information is used to guide the user to provide the music characteristics of the target music element.

[0072] The first guide information may be displayed in the form of a message sent by the agent.

[0073] Assume that the music elements include A, B, C, and D. Through the first message, the music features of the music elements A, B, and C can be obtained, but the music features of the music element D cannot be obtained. Then, the first guidance information can be sent based on the music element D.

[0074] In some embodiments, the first guide information is used to inquire the user about the music characteristics of the target music element, or to provide alternative music characteristics of the target music element.

[0075] For example, the agent can directly send a message to the user, the content of the message is to ask the user what the musical characteristics of the target musical element in the music that the user wants to create are. For example, for the musical element "rhythm", the first guiding information can be "What is the rhythm of this song?" This helps guide the user to accurately describe the missing musical features.

[0076] For another example, the agent can generate alternative music features based on the target music element. The alternative music features can be default settings. For example, for the music element rhythm, the music feature can represent fast, normal, slow, and so on. For some music elements, there may be multiple possibilities for their music features. In this case, alternative music features can be generated based on the music element and the content of the first message. For example, if the user's first message describes "walking in a bustling city", if it is determined that the music element "style" is missing, the first guidance information can be sent to the user. Based on the content of the first message, it can be determined that the song describes a modern scene and is not suitable for classical music unless the user specifically states it. Therefore, the agent can send a message to the user, "Do you want this song to be in rock style or folk style?" This method can provide more information to users who lack professional knowledge of music, thereby further improving the user's creation speed.

[0077] The dialogue between the agent and the user can be carried out in one or more rounds. That is, the agent can send one or more guiding messages to the user to improve the musical features of the music to be generated.

[0078] The above embodiment guides the user to provide music features through dialogue, so the user can provide relevant information of the music to be generated through multiple dialogues. In this way, the processing pressure of the user when generating music will be reduced and the processing efficiency will be improved.

[0079] In the process of guiding the user to supplement the information related to the music, the user can also be guided to provide lyrics. In some embodiments, when one or more music features do not indicate whether the music has lyrics, the second guidance information is displayed, and the second guidance information is generated according to the music features to guide the user to provide the lyrics information of the music. That is, if the first message does not include lyrics, and the lyrics are not obtained by analyzing the first message, the user can be guided to supplement the lyrics information. The intelligent agent can send a message including the second guidance information to the user. The second guidance information is, for example, "Does this song have lyrics?" "Please write a few lyrics."

[0080] In some embodiments, in response to a user sending lyrics information to an agent, music is generated based on the prompt information and the lyrics information. The lyrics information sent by the user may be complete lyrics, partial lyrics, or keywords in the lyrics, or the theme of the lyrics, etc. After obtaining the lyrics information, the prompt information and the lyrics information may be processed using a text generation model, such as rewriting, expanding, or a combination of the two, to generate complete lyrics.

[0081] Since the user can instruct the agent to create music while chatting with the agent in the dialogue interface, the conversation between the user and the agent may also involve non-music creation content. During the conversation, the intent of the message sent by the user can be identified to determine whether to generate music.

[0082] Figure 4 FIG. 2 is a flow chart showing a method for identifying music generation intention according to some embodiments of the present disclosure. Figure 4 As shown, the music generation intention recognition method of this embodiment includes steps S402 to S404.

[0083] In step S402, the intent of the first message is identified.

[0084] The intention of generating music in the first message can be explicit or implicit. That is, the first message can directly contain words related to music generation or music creation, such as "generate a piece of music, the requirement is..."; or, the first message may not contain relevant words, but the user's intention conveyed by the first message is more suitable for creating music. A list can be pre-set, including intentions suitable for creating music, such as the intention to express current mood or feelings, the intention to communicate with other contacts, the intention to describe scene pictures, etc.

[0085] When performing intent recognition, a keyword matching method can be used, for example, extracting keywords from the first message and determining the intent corresponding to the keywords as the intent of the first message; the intent of the first message can also be identified using an intent recognition model to determine whether the first message has the intention to generate music.

[0086] In step S404, in response to the first message indicating an intention to generate music, a music feature corresponding to one or more music elements is generated based on the first message sent by the user.

[0087] That is, in response to the first message having the intention of generating music, the process of generating music is triggered. If the first message does not have the intention of generating music, no music is generated, but the agent is controlled to continue the conversation with the user. For example, based on the first message, a response message of the agent is generated, and the response message includes at least one of text, image, video, and link. For example, based on the reply text, at least one of images, videos, and links can be added as needed; or only text can be replied.

[0088] The above embodiment can first detect the intention of the message sent by the user, and execute the music generation process when the message contains the intention of generating music. Therefore, when the music generation function is configured for the intelligent agent, the normal conversation between the intelligent agent and the user is not affected. In addition, the user does not need to use fixed instructions to trigger music generation, but only needs to express the intention of music generation in the conversation. Therefore, the efficiency of music generation is improved.

[0089] After the music is generated, the user can also adjust the generated music as needed. Figure 5 An embodiment of a music adjustment method is described.

[0090] Figure 5 FIG. 2 shows a flow chart of a music adjustment method according to some embodiments of the present disclosure. Figure 5 As shown, the music adjustment method of this embodiment includes steps S502 to S508.

[0091] In step S502, the second message sent by the user to the agent is displayed.

[0092] The second message is sent by the user after the agent sends music to the user.

[0093] In step S504, the intent of the second message is identified. The manner of identifying the intent of the second message may refer to the manner of identifying the intent of the first message, which will not be described in detail here.

[0094] In step S506, in response to the second message including the intention to adjust the music, the music is adjusted based on the second message.

[0095] For example, if the user indicates through the second message that the generated music prelude is too long, the chorus is too short, the instrument is inappropriate, etc., corresponding adjustments can be made based on the second message. In some embodiments, the second message is semantically understood, and the adjustment object and adjustment method therein are identified and determined. For example, for "the prelude is too long", the adjustment object is "the prelude", and the adjustment method is determined to be "shortening" based on "too long".

[0096] In step S508, the adjusted music sent by the agent to the user is displayed.

[0097] After adjusting the music, the agent can send the adjusted music to the user by sending a message.

[0098] Through the above embodiments, the user can flexibly adjust the generated music in a dialogue manner, thereby making the generated music more compatible with the user's expectations and improving the user experience of music generation.

[0099] Figure 6The following is a schematic diagram of a dialog interface of some embodiments of the present disclosure. Figure 6 As shown, in the dialogue interface 6 between the user and agent A, the user sends a message 61 "Help me generate music, the content is friends talking freely around a bonfire by the sea, the atmosphere is relaxed and happy, there are the sounds of waves hitting the beach, and there are some guitar melodies". Through semantic understanding, some musical elements of the music to be generated can be determined. For example, the atmosphere is relaxed, happy, and warm (determined based on "friends" and "bonfire"), the rhythm is brisk (determined based on "relaxed and happy"), the melody is soothing and warm (based on "friends talking freely around a bonfire by the sea, the atmosphere is relaxed and happy"), the instruments are guitar, ukulele (determined based on "seaside" and "waves"), tambourine (determined based on "the sound of waves hitting the beach"), and so on.

[0100] If necessary, the user can be guided to make further supplements. Figure 6 The middle agent sends a message 62 to the user, "OK, what music style do you want?" to guide the user to add a music style. The user replies with a message 63, "Folk music, male voice, romantic", providing information that the music style is folk music. In addition, the timbre is male voice, and the atmosphere adds a romantic feature.

[0101] Based on the above extracted information, music is generated and displayed through a message 64 sent by the agent. Message 64 carries the player of the generated music, as well as information about the theme, author, and lyrics.

[0102] If the user has any adjustments, he or she may further send adjustment instructions through the message sending control 65 .

[0103] The above describes the methods of various embodiments of the present disclosure. The following further describes the apparatus for executing the methods of various embodiments of the present disclosure.

[0104] Figure 7 FIG. 2 shows a schematic diagram of the structure of a music generating device according to some embodiments of the present disclosure. Figure 7 As shown, the music generating device 70 of this embodiment includes: a display module 701, configured to display a first message sent by a user in a dialogue interface between the user and the agent; a feature generating module 702, configured to generate a music feature corresponding to one or more music elements based on the first message; a determination module 703, configured to determine prompt information according to the music feature; a music generating module 704, configured to generate music based on the prompt information; the display module 701 is further configured to display the music sent by the agent to the user.

[0105] In some embodiments, the feature generation module 702 is further configured to: determine one or more keywords based on the result of semantic understanding of the first message; and generate music features corresponding to one or more music elements based on the keywords.

[0106] In some embodiments, the feature generation module 702 is further configured to: for each keyword, determine a music element matching the keyword from one or more music elements; for each music element, generate a music feature according to the keyword matching the music element type.

[0107] In some embodiments, the feature generation module 702 is further configured to extract one or more keywords from the first message and the result of semantic understanding of the first message.

[0108] In some embodiments, the feature generation module 702 is further configured to: determine the music features of one or more specified music elements based on a first message sent by the user; in response to the presence of at least one target music element that does not have the music features, display a first guidance message sent by the intelligent agent to the user, the first guidance message being used to guide the user to provide the music features of the target music elements.

[0109] In some embodiments, the first guide information is used to inquire the user about the music characteristics of the target music element, or to provide alternative music characteristics of the target music element.

[0110] In some embodiments, the one or more music elements include at least one of style, timbre, melody, rhythm, atmosphere, instrument, and lyric information.

[0111] In some embodiments, the feature generation module 702 is further configured to: identify the intent of the first message; and in response to the first message having the intent to generate music, generate a music feature corresponding to one or more music elements based on the first message sent by the user.

[0112] In some embodiments, the display module 701 is further configured to display a second message sent by the user to the agent; the music generating device 70 also includes an adjustment module, which is configured to: identify the intention of the second message, and in response to the second message including the intention to adjust the music, adjust the music based on the second message; the display module 701 is further configured to display the adjusted music sent by the agent to the user.

[0113] In some embodiments, the display module 701 is further configured to display second guidance information when one or more music features do not indicate whether the music has lyrics, and the second guidance information is generated based on the music features to guide the user to provide lyrics information of the music.

[0114] In some embodiments, the music generation module 704 is further configured to: in response to the user sending lyrics information to the agent, generate music according to the prompt information and the lyrics information.

[0115] It should be noted that the above-mentioned units are only logical modules divided according to the specific functions they implement, and are not used to limit the specific implementation methods. For example, they can be implemented in software, hardware, or a combination of software and hardware. In actual implementation, the above-mentioned units can be implemented as independent physical entities, or can also be implemented by a single entity (for example, a processor (CPU or DSP, etc.), an integrated circuit, etc.). In addition, the above-mentioned units are shown with dotted lines in the accompanying drawings to indicate that these units may not actually exist, and the operations / functions they implement can be implemented by the processing circuit itself.

[0116] In addition, although not shown, the device may also include a memory, which may store various information generated by the device, various units contained in the device during operation, programs and data for operation, data to be sent by the communication unit, etc. The memory may be a volatile memory and / or a non-volatile memory. For example, the memory may include, but is not limited to, a random access memory (RAM), a dynamic random access memory (DRAM), a static random access memory (SRAM), a read-only memory (ROM), and a flash memory. Of course, the memory may also be located outside the device. Optionally, although not shown, the device may also include a communication unit, which may be used to communicate with other devices. In one example, the communication unit may be implemented in an appropriate manner known in the art, such as including communication components such as an antenna array and / or a radio frequency link, various types of interfaces, communication units, etc. It will not be described in detail here. In addition, the device may also include other components not shown, such as a radio frequency link, a baseband processing unit, a network interface, a processor, a controller, etc. It will not be described in detail here.

[0117] Some embodiments of the present disclosure also provide an electronic device. Figure 8 A schematic diagram of the structure of an electronic device according to some embodiments of the present disclosure is shown. For example, in some embodiments, the electronic device 8 can be various types of devices, for example, can include but not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. For example, the electronic device 8 can include a display panel for displaying data and / or execution results used in the scheme according to the present disclosure. For example, the display panel can be in various shapes, such as a rectangular panel, an elliptical panel, or a polygonal panel, etc. In addition, the display panel can be not only a flat panel, but also a curved panel, or even a spherical panel.

[0118] like Figure 8 As shown, the electronic device 8 of this embodiment includes: a memory 81 and a processor 82 coupled to the memory 81. It should be noted that Figure 8 The components of the electronic device 8 shown are only exemplary and non-restrictive. The electronic device 8 may also have other components according to actual application requirements. The processor 82 may control other components in the electronic device 8 to perform desired functions.

[0119] In some embodiments, the memory 81 is used to store one or more computer-readable instructions. When the processor 82 is used to execute the computer-readable instructions, the computer-readable instructions are executed by the processor 82 to implement the method according to any of the above embodiments. For the specific implementation of each step of the method and the related explanation content, please refer to the above embodiments, and the repetition will not be repeated here.

[0120] For example, the processor 82 and the memory 81 may communicate with each other directly or indirectly. For example, the processor 82 and the memory 81 may communicate with each other through a network. The network may include a wireless network, a wired network, and / or any combination of a wireless network and a wired network. The processor 82 and the memory 81 may also communicate with each other through a system bus, which is not limited by the present disclosure.

[0121] For example, the processor 82 may be embodied as various appropriate processors, processing devices, etc., such as a central processing unit (CPU), a graphics processing unit (GPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The central processing unit (CPU) may be an X86 or ARM architecture, etc. For example, the memory 81 may include any combination of various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The memory 81 may include, for example, a system memory, which may store, for example, an operating system, an application program, a boot loader (Boot Loader), a database, and other programs. Various applications and various data may also be stored in the storage medium.

[0122] In addition, according to some embodiments of the present disclosure, various operations / processes according to the present disclosure, when implemented by software and / or firmware, can be transmitted from a storage medium or a network to a computer system having a dedicated hardware structure, such as Fig. 9The computer system 90 shown installs the programs constituting the software, and when the various programs are installed, the computer system can execute various functions, including the functions described above. Fig. 9 A schematic diagram of the structure of a computer system according to some embodiments of the present disclosure is shown.

[0123] exist Fig. 9 In the embodiment, a central processing unit (CPU) 901 performs various processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage section 908 to a random access memory (RAM) 903. In the RAM 903, data required when the CPU 901 performs various processes, etc., is also stored as needed. The central processing unit is merely exemplary, and it may also be other types of processors, such as the various processors described above. The ROM 902, the RAM 903, and the storage section 908 may be various forms of computer-readable storage media, as described below. It should be noted that although Fig. 9 ROM 902, RAM 903 and storage section 908 are shown separately in the figure, but one or more of them may be combined or located in the same or different memory or storage modules.

[0124] The CPU 901, the ROM 902, and the RAM 903 are connected to one another via a bus 904. An input / output interface 905 is also connected to the bus 904.

[0125] The following components are connected to the input / output interface 905: an input section 906, such as a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output section 907, including a display, such as a cathode ray tube (CRT), a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage section 908, including a hard disk, a magnetic tape, etc.; and a communication section 909, including a network interface card such as a LAN card, a modem, etc. The communication section 909 allows communication processing to be performed via a network such as the Internet. It is easy to understand that although Fig. 9 The various devices or modules in the computer system 90 are shown to communicate via a bus 904, but they may also communicate via a network or other means, wherein the network may include a wireless network, a wired network, and / or any combination of a wireless network and a wired network.

[0126] A drive 910 is also connected to the input / output interface 905 as needed. A removable medium 911 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory or the like is mounted on the drive 910 as needed so that a computer program read therefrom is installed into the storage section 908 as needed.

[0127] In the case where the above-described series of processing is realized by software, a program constituting the software can be installed from a network such as the Internet or a storage medium such as the removable medium 911 .

[0128] According to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 909, or installed from the storage part 908, or installed from the ROM 902. When the computer program is executed by the CPU 901, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0129] It should be noted that in the context of the present disclosure, a computer-readable medium may be a tangible medium that may contain or store a program for use by an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. A computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. A computer-readable storage medium may be, for example, but not limited to: an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program that may be used by an instruction execution system, device or device or used in combination with it. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on a computer-readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0130] The computer-readable medium may be included in the electronic device, or may exist independently without being installed in the electronic device.

[0131] In some embodiments, a computer program is further provided, comprising: instructions, which, when executed by a processor, cause the processor to execute any of the methods of the above embodiments. For example, the instructions may be embodied as computer program codes.

[0132] In an embodiment of the present disclosure, computer program code for performing the operations of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including but not limited to object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" language or similar programming languages. The program code may be executed entirely on a user's computer, partially on a user's computer, as a separate software package, partially on a user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0133] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0134] The modules, components or units involved in the embodiments described in the present disclosure may be implemented by software or hardware, wherein the names of the modules, components or units do not, in some cases, limit the modules, components or units themselves.

[0135] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, exemplary hardware logic components that may be used include, without limitation, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0136] The above descriptions are only some embodiments of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other to form a technical solution.

[0137] In the description provided herein, many specific details are set forth. However, it is understood that embodiments of the present invention can be implemented without these specific details. In other cases, well-known methods, structures, and techniques are not presented in detail in order not to obscure the understanding of the description.

[0138] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0139] Although some specific embodiments of the present disclosure have been described in detail by way of example, it should be understood by those skilled in the art that the above examples are for illustration only and are not intended to limit the scope of the present disclosure. It should be understood by those skilled in the art that the above embodiments may be modified without departing from the scope and spirit of the present disclosure. The scope of the present disclosure is defined by the appended claims.

Claims

1. A music generation method, comprising: On the dialogue interface between the user and the agent, displaying the first message sent by the user; Based on the first message, generating music features corresponding to one or more music elements includes: determining the music features of one or more designated music elements based on the first message sent by the user; in response to the existence of at least one target music element that does not have the music features, displaying first guidance information sent by the agent to the user, the first guidance information is used to provide alternative music features of the target music element, the alternative music features are generated according to the target music element and the content of the first message; Determining prompt information according to the music features; Generate music based on the prompt information; Display the music sent by the agent to the user.

2. The music generation method according to claim 1, wherein: The generating, based on the first message, a music feature corresponding to one or more music elements comprises: Determining one or more keywords based on a result of semantic understanding of the first message; The music features corresponding to the one or more music elements are generated according to the keywords.

3. The music generation method according to claim 2, wherein: Generating the music features corresponding to the one or more music elements according to the keywords includes: For each keyword, determining a music element matching the keyword from one or more music elements; For each music element, a music feature is generated according to a keyword matching the type of the music element.

4. The music generation method according to claim 2, wherein: The determining one or more keywords according to the result of the semantic understanding of the first message includes: One or more keywords are extracted from the first message and a result of semantic understanding of the first message.

5. The music generating method according to any one of claims 1 to 4, wherein: The one or more music elements include at least one of music style, timbre, melody, rhythm, atmosphere, instrument, and lyrics information.

6. The music generating method according to any one of claims 1 to 4, wherein: The generating of music features corresponding to one or more music elements based on the first message sent by the user includes: Performing intent recognition on the first message; In response to the first message indicating an intention to generate music, the music features corresponding to the one or more music elements are generated based on the first message sent by the user.

7. The music generation method according to any one of claims 1 to 4, further comprising: displaying a second message sent by the user to the agent; identifying the intent of the second message; In response to the second message including an intention to adjust the music, adjusting the music based on the second message; The adjusted music sent by the agent to the user is displayed.

8. The music generation method according to any one of claims 1 to 4, further comprising: In a case where the one or more music features do not indicate whether the music has lyrics, second guidance information is displayed, where the second guidance information is generated based on the music features and is used to guide the user to provide lyrics information of the music.

9. The music generating method according to claim 8, wherein: Generating music based on the prompt information includes: In response to the user sending lyrics information to the agent, the music is generated according to the prompt information and the lyrics information.

10. An electronic device, comprising: Memory; as well as A processor coupled to the memory, the processor being configured to execute the music generating method according to any one of claims 1 to 9 based on instructions stored in the memory.

11. A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the music generating method according to any one of claims 1 to 9.

12. A computer program product, when the computer program product is run on a computer, enables the computer to implement the music generation method according to any one of claims 1 to 9.

Images