Method and apparatus for generating music, device, and storage medium

Abstract

A method and apparatus for generating music, a device, and a storage medium. The method comprises: on the basis of a received music generation request, constructing a target mask corresponding to the music generation request; on the basis of a mask matrix and a first encoded representation of reference music content, generating an intermediate encoded representation; using a target model to process the intermediate encoded representation, so as to generate a second encoded representation, the second encoded representation being generated by means of replacing a target part in the intermediate encoded representation, and the target part being indicated by the mask matrix; and, on the basis of the second encoded representation, generating target music content.

Description

Method, apparatus, device and storage medium for music generation

[0001] The present application claims priority to the Chinese Patent Application No. 202411253460.5, filed on September 6, 2024, and entitled "Method, apparatus, device and storage medium for music generation", the content of which is incorporated herein by reference in its entirety. TECHNICAL FIELD

[0002] Example embodiments of the present disclosure generally relate to the field of computers, and in particular, to a method, apparatus, device and computer readable storage medium for music generation. BACKGROUND

[0003] With the development of computer technology, music generation technology, as a frontier innovation in the field of artificial intelligence, is gradually changing the traditional mode of music creation and generation. The traditional music generation model has limited functions and cannot guarantee that the background sound or vocals remain unchanged during the music generation process. SUMMARY

[0004] In a first aspect of the present disclosure, a method for music generation is provided. The method comprises: constructing a target mask corresponding to a music generation request based on the received music generation request; generating an intermediate encoded representation based on the target mask and a first encoded representation of a reference music content; processing the intermediate encoded representation using a target model to generate a second encoded representation, the second encoded representation being generated by replacing a target portion in the intermediate encoded representation, the target portion being indicated by the target mask; and generating a target music content based on the second encoded representation.

[0005] In a second aspect of the present disclosure, an apparatus for music generation is provided. The apparatus comprises: a target mask construction module configured to construct a target mask corresponding to a music generation request based on the received music generation request; an intermediate encoded representation generation module configured to generate an intermediate encoded representation based on the target mask and a first encoded representation of a reference music content; a second encoded representation generation module configured to process the intermediate encoded representation using a target model to generate a second encoded representation, the second encoded representation being generated by replacing a target portion in the intermediate encoded representation, the target portion being indicated by the target mask; and a music content generation module configured to generate a target music content based on the second encoded representation.

[0006] In a third aspect of the present disclosure, an electronic device is provided. The device comprises at least one processor; and at least one memory coupled to the at least one processor and storing instructions for execution by the at least one processor. The instructions, when executed by the at least one processor, cause the device to perform the method of the first aspect.

[0007] In a fourth aspect of the present disclosure, a computer-readable storage medium is provided. The computer-readable storage medium has stored thereon computer-executable instructions that, when executed by a processor, cause the processor to implement the method of the first aspect.

[0008] In a fifth aspect of the present disclosure, a computer program product is provided. The computer program product is tangibly stored in a computer storage medium and includes computer-executable instructions that, when executed by a device, cause the device to perform the method according to the first aspect.

[0009] It should be understood that nothing in the Summary is to be construed as a limitation on the scope of the embodiments of the present disclosure. Other features, aspects, and advantages of the present disclosure will become apparent from the following description, which is given by way of example in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0010] The above and other features, aspects, and advantages of embodiments of the present disclosure will become more apparent from the following detailed description in conjunction with the accompanying drawings. In the drawings, like reference numerals denote like elements, in which:

[0011] FIG. 1 shows a schematic diagram of an example environment in which embodiments of the present disclosure can be implemented;

[0012] FIG. 2 shows a flowchart of music generation according to some embodiments of the present disclosure;

[0013] FIGS. 3A-3E show example diagrams of target masks according to some embodiments of the present disclosure;

[0014] FIGS. 4A-4D show flowcharts of generating music based on different target masks according to some embodiments of the present disclosure;

[0015] FIG. 5 shows a flowchart of training a music generation model according to some embodiments of the present disclosure;

[0016] FIG. 6 shows a block diagram of an apparatus for music generation according to some embodiments of the present disclosure; and

[0017] FIG. 7 shows a block diagram of an electronic device capable of implementing various embodiments of the present disclosure. DETAILED DESCRIPTION

[0018] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While certain embodiments of the present disclosure are shown in the drawings, it is understood that the present disclosure can be embodied in various forms and should not be construed as being limited to the embodiments set forth herein; rather, these embodiments are provided so that the present disclosure will be thoroughly and completely understood. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0019] It should be noted that the titles of any sections / sub-sections provided herein are not limiting. Various embodiments are described throughout this document and any type of embodiment can be included under any section / sub-section. Furthermore, embodiments described in any section / sub-section can be combined with any other embodiments described in the same section / sub-section and / or different section / sub-section in any manner.

[0020] In the description of embodiments of the present disclosure, the term "includes" and its derivatives, such as "including," should be understood in an open, inclusive sense, that is, "including, but not limited to." The term "based on" should be understood as "based at least in part on." The term "one embodiment" or "an embodiment" should be understood as "at least one embodiment." The term "some embodiments" should be understood as "at least some embodiments." Other explicitly and implicitly recited definitions can also be found below. The terms "first," "second," and the like can refer to different or identical objects. Other explicit and implicit definitions can also be found below.

[0021] Data of users, acquisition and / or use of data, etc. can be involved in embodiments of the present disclosure. These aspects all comply with corresponding laws and regulations and relevant provisions. In embodiments of the present disclosure, all collection, acquisition, processing, processing, forwarding, use, etc. of data are performed on the premise that users are aware of and confirm. Accordingly, when implementing embodiments of the present disclosure, the type of data or information that can be involved, the range of use, the scenario of use, etc. should be informed to users and authorized by users in a proper manner according to relevant laws and regulations. The specific informing and / or authorization manner can vary according to actual situations and application scenarios, and the scope of the present disclosure is not limited in this respect.

[0022] In the present specification and embodiments, if personal information processing is involved, it will be processed on the premise of legality (for example, obtaining the consent of the subject of personal information, or being necessary for the performance of a contract, etc.), and only within the prescribed or agreed range. Users refuse to process personal information other than the necessary information required for basic functions, which will not affect the user's use of basic functions.

[0023] The data involved in the schemes in the specification and examples, such as the training and inference of the model, including but not limited to the data itself, the acquisition and / or use of the data, complies with the requirements of the corresponding laws and regulations.

[0024] As briefly mentioned above, with the development of computer technology, music generation technology, as a frontier innovation in the field of artificial intelligence, is gradually changing the traditional mode of music creation and generation. The music generation function of the traditional music generation model is limited, and it cannot guarantee that the background sound or vocals remain unchanged during the music generation process.

[0025] To this end, an embodiment of the present disclosure proposes a scheme for music generation. According to various embodiments of the present disclosure, based on a received music generation request, a target mask corresponding to the music generation request is constructed; based on the target mask and a first encoded representation of reference music content, an intermediate encoded representation is generated; the intermediate encoded representation is processed using a target model to generate a second encoded representation, the second encoded representation being generated by replacing a target portion in the intermediate encoded representation, the target portion being indicated by the target mask; and based on the second encoded representation, a target music content is generated.

[0026] In this way, the embodiments of the present disclosure can construct a corresponding mask matrix based on a music generation request, and can further generate corresponding music content using the mask matrix. Thus, the embodiments of the present disclosure can enrich the music generation scene and improve the quality of the music content.

[0027] In addition, the embodiments of the present disclosure indicate the target portion in the intermediate encoded representation corresponding to the content to be generated by using the mask matrix. Thus, the embodiments of the present disclosure can not interfere with other audio data during the music generation process, and can ensure the unity of the overall music style.

[0028] Example embodiments of the present disclosure are described below with reference to the accompanying drawings.

[0029] Example environment

[0030] FIG. 1 shows a schematic diagram of an example environment 100 in which embodiments of the present disclosure can be implemented. As shown in FIG. 1, the example environment 100 can include an electronic device 110 and a target model 136.

[0031] In the environment 100 of FIG. 1, the electronic device 110 can receive input data 130 from a user 140, the input data 130 can include audio data and / or text data. Further, the electronic device 110 can invoke the target model 136 to process the input data 130 to obtain target audio 120. The target model may, for example, be a non-causal deep learning model such as a diffusion model, which can generate synthesized audio content based on input text content or lyrics content.

[0032] In some embodiments, the electronic device 110 can include various types of computing systems / servers capable of providing computing capabilities, and the electronic device 110 can include an end device. Such an end device can be any type of mobile terminal, fixed terminal, or portable terminal including a mobile handset, a desktop computer, a laptop computer, a notebook computer, a netbook computer, a tablet computer, a media computer, a multimedia tablet, a personal communication system (PCS) device, a personal navigation device, a personal digital assistant (PDA), a voice / video recorder, a digital camera / camcorder, a positioning device, a television receiver, a radio broadcast receiver, an electronic book device, a game device, or any combination thereof, including accessories and peripherals of such devices or any combination thereof. The electronic device 110 may, for example, include various types of computing systems / servers capable of providing computing capabilities, such as mainframes, edge computing nodes, computing devices in a cloud environment, virtual machines, and the like. Although shown as a single device, the electronic device 110 can include multiple physical devices.

[0033] It should be appreciated that the structure and functionality of the various elements in the environment 100 are described for illustrative purposes only and do not imply any limitation on the scope of the present disclosure.

[0034] Some example embodiments of the present disclosure will be described below with continued reference to the drawings.

[0035] Music generation process examples

[0036] FIG. 2 illustrates a flowchart of a speech processing process 200 according to some embodiments of the present disclosure. The process 200 can be implemented at the electronic device 110. The process 200 is described below with reference to FIG. 1.

[0037] As shown in FIG. 2, at block 210, the electronic device 110 can construct a target mask corresponding to the music generation request based on the received music generation request.

[0038] In some embodiments, the target mask, also referred to as a mask matrix, can be constructed accordingly based on the content to be generated by the music generation request. In some implementations, the target mask can correspond to preset mask patterns corresponding to different types of generation requests. In some embodiments, the mask patterns include, but are not limited to, a first mask pattern, a second mask pattern, a third mask pattern, a fourth mask pattern, and a fifth mask pattern as described below.

[0039] In some embodiments, the first mask pattern indicates, for example, generating subsequent track data corresponding to an existing track based on the reference music content. In some scenarios, the first mask pattern can also be referred to as a continuation mask. FIG. 3A illustrates a schematic diagram of a continuation mask according to some embodiments of the present disclosure. As shown in FIG. 3A, the continuation mask can mask the reference audio content over the entire track in the time dimension, and can indicate that the model generates subsequent music content backward, thereby realizing the continuation capability of the music content.

[0040] In some embodiments, the second mask pattern indicates generating additional track data corresponding to a new track based on the reference music content. In some scenarios, the second mask pattern can also be referred to as a track conversion mask. FIG. 3B illustrates a schematic diagram of a track conversion mask according to some embodiments of the present disclosure. As shown in FIG. 3B, the track conversion mask can realize the conversion of other tracks from one track or multiple tracks. Such music generation scenarios can include, for example, adding an accompaniment track from a vocal track, adding a vocal track from an accompaniment track, and the like.

[0041] In some embodiments, the third mask pattern can indicate generating subsequent track data corresponding to at least one existing track and additional track data corresponding to a new track based on the reference music content. In some scenarios, the third mask pattern can also be referred to as an expansion mask. FIG. 3C illustrates a schematic diagram of an expansion mask according to some embodiments of the present disclosure. As shown in FIG. 3C, the expansion mask can realize the continuation of the content on the main track and supplement the content of other tracks. Such music generation scenarios can include, for example, generating subsequent music content and accompanying tracks given a humming content.

[0042] In some embodiments, the fourth mask pattern can indicate generating additional track data corresponding to multiple tracks based on the existing track data of the multiple tracks of the reference music content, the additional track data of different tracks corresponding to different time periods. In some scenarios, the fourth mask pattern can also be referred to as a mixing mask. FIG. 3D illustrates a schematic diagram of a mixing mask according to some embodiments of the present disclosure. As shown in FIG. 3D, the mixing mask can indicate taking the content of a certain track as the starting content and the content of another track as the ending content, and accordingly generating the intermediate content of the two tracks. Such music generation scenarios can include, for example, transitioning from a specified piece of music to another specified piece of music.

[0043] In some embodiments, the fifth mask pattern can indicate generating intermediate track data corresponding to at least one existing track and additional track data corresponding to a new track based on the reference music content. In some scenarios, unlike the third mask pattern, the track content generated by the fifth mask pattern is the intermediate portion data of a specific track. In some scenarios, the fifth mask pattern is also referred to as a time-track expansion mask.

[0044] FIG. 3E shows a schematic diagram of a time-track augmentation mask, according to some embodiments of the present disclosure. As shown in FIG. 3E, the time-track augmentation mask can indicate to generate intermediate content of a specific track segment and augment other tracks. Such a music generation scenario can include, for example, augmenting content in an accompaniment track and augmenting other tracks.

[0045] The target model can process different input audio data based on different mask patterns to generate different music content, enriching the music generation function of the target model.

[0046] In some embodiments, the electronic device 110 may, for example, match a corresponding mask pattern according to a music generation request, and generate a corresponding target mask.

[0047] Specifically, the electronic device 110 can determine at least one track and / or at least one track segment to be generated based on the music generation request. Taking the music expansion scenario as an example, when the electronic device 110 receives a music expansion request, the electronic device 110 can determine a track to be generated based on the prelude audio in the music expansion request. For example, as shown in FIG. 3C, the track to be generated can be the track corresponding to the expansion mask in FIG. 3C.

[0048] Further, the electronic device 110 can construct a target mask based on the at least one track and / or the at least one track segment. In some examples, the mask portion in the target mask corresponding to the at least one track and / or the at least one track segment is set to a preset value (e.g., 0). Continuing the example of the music expansion scenario, the electronic device 110 can construct the expansion mask shown in FIG. 3C based on the received prelude audio encoding and the determined track to be generated.

[0049] The specific process of determining the track and / or track segment to be generated based on the music generation request will be further determined below.

[0050] In some embodiments, the music generation request may, for example, indicate lyrics content to be added. Taking the music continuation request as an example, the user may, for example, indicate lyrics content corresponding to the subsequent music content to be generated.

[0051] Accordingly, the electronic device 110 can determine first lyrics content to be added based on the music generation request. Further, the electronic device 110 can determine a first time length corresponding to the first lyrics content based on the speech rate information of the reference music content.

[0052] For example, the electronic device 110 can determine the singing speech rate in the reference music content by sampling, and can determine the expected time length corresponding to the first lyrics content to be added based on the singing speech rate.

[0053] Accordingly, the electronic device 100 can determine that the at least one audio track segment to be generated includes a first audio track segment in the vocal track that corresponds to the first time length. Taking the music continuation scenario as an example, the electronic device 110 may, for example, construct a continuation mask 300A as shown in FIG. 3A based on the determined first time length.

[0054] In some scenarios, the reference music content may, for example, not include a vocal track. Accordingly, in a case where the music generation request indicates newly added lyric content, the electronic device may, based on the music generation request, determine the newly added second lyric content. Further, the electronic device 110 may, in response to the reference music content not including a vocal track, determine that the at least one audio track to be generated can include the vocal track.

[0055] Taking the music generation request as an example, the music generation request may, for example, indicate generation of a vocal track corresponding to the second lyric content based on the existing accompaniment content. Accordingly, the electronic device 110 may, for example, construct an audio track conversion mask 300B as shown in FIG. 3B.

[0056] In some embodiments, the music generation request may, for example, indicate expansion of a specific content of a certain audio track. Specifically, the electronic device 110 may, based on the music generation request, obtain the reference music content, the reference music content including a plurality of audio track segments corresponding to a target audio track. Further, the electronic device 110 may determine that the at least one audio track segment to be generated includes an audio track segment corresponding to a preset time period of the target audio track.

[0057] Taking the accompaniment expansion scenario as an example, the music generation request may, for example, indicate input of a start segment and an end segment of an accompaniment audio track, and may specify a preset time period to be expanded. Accordingly, the electronic device 110 may, for example, construct a time-audio track expansion mask as shown in FIG. 3E.

[0058] In some scenarios, the preset time period to be expanded may, for example, also be determined based on lyric content indicated by the music generation request. For example, a user may indicate insertion of third lyric content in existing lyric content, and accordingly, the electronic device 110 may determine that the preset time period to be expanded is a time period corresponding to the third lyric content.

[0059] In some embodiments, taking the time-audio track expansion mask as shown in FIG. 3E as an example, in the accompaniment expansion scenario, the music generation request also indicates a set of audio tracks to be newly added, for example, a vocal track, etc.

[0060] In some embodiments, the music generation request can indicate two different audio track segments of two different audio tracks to indicate a transition between the two audio track segments. Specifically, the electronic device 110 may, for example, based on the music generation request, obtain the reference music content including at least a second audio track segment corresponding to a first audio track and a third audio track segment corresponding to a second audio track.

[0061] For example, the first audio track can be a vocal track, and the second audio track can be an accompaniment track. The music generation request may, for example, indicate a starting audio track segment of the vocal track, and can indicate an ending audio track segment of the accompaniment track.

[0062] Accordingly, the electronic device 110 can determine that the at least one audio track segment to be generated includes a fourth audio track segment (e.g., subsequent content of the second audio track segment) corresponding to the first audio track (e.g., vocal) and a fifth audio track segment (e.g., preceding content of the third audio track segment) corresponding to the second audio track.

[0063] In some embodiments, a time length of the fourth audio track segment to be augmented may, for example, be determined based on the indicated new lyric content (also referred to as fourth lyric content). Accordingly, the electronic device 110 can construct a mixed mask 300D as shown in FIG. 3D based on the fourth audio track segment and the fifth audio track segment to be generated.

[0064] Based on the above mask construction process, embodiments of the present disclosure can generate corresponding masks according to the needs of different music generation, thereby enriching the music generation scenarios.

[0065] With reference to FIG. 2, at block 220, the electronic device 110 can generate an intermediate encoded representation based on the mask matrix and the first encoded representation of the reference music content.

[0066] In some embodiments, the electronic device 110 may, in response to the reference audio content including a plurality of audio tracks, determine audio track data corresponding to the plurality of audio tracks using a separation module, and may, using an audio encoder, process the audio track data corresponding to the plurality of audio tracks to determine the first encoded representation.

[0067] Different music generation processes corresponding to different music generation scenarios will be described below in conjunction with FIGS. 4A-4D.

[0068] In some scenarios, as shown in FIG. 4A, FIG. 4A illustrates a music generation process in a music expansion scenario in an embodiment of the present application. The electronic device 110 can receive a music generation request, which can indicate the lyrics content to be added. The electronic device 110 can obtain the prelude audio 401 (i.e., reference music content), music expansion request, and first lyrics content 407 added by the user 140. It should be understood that the prelude audio 401 can be multi-track data or only single-track data. When the prelude audio 401 includes multi-track data, the electronic device 110 can also include the target track specified by the user 140, which can be, for example, a vocal track.

[0069] After receiving the prelude audio track 401, the electronic device 110 can input the prelude audio 401 to the audio separation model 402 to separate the multiple tracks in the prelude audio 401, obtaining the prelude audio encoding 403 corresponding to the vocal track (i.e., the first encoded representation). Moreover, the electronic device 110 can determine the singing speed in the prelude audio 401 based on the average of the number of words appearing in the reference music content and the duration of the reference music content.

[0070] Further, the electronic device 110 can calculate the subsequent audio duration based on the singing speed of the prelude audio 401 and the length of the first lyrics content 407, and generate a corresponding expansion mask based on the subsequent audio duration and the music expansion request. After generating the expansion mask, the electronic device 110 can combine the prelude audio encoding 403 and the expansion mask to obtain the intermediate encoded representation 406.

[0071] In other scenarios, as shown in FIG. 4B, FIG. 4B illustrates a music generation process in a track conversion scenario in an embodiment of the present application. The electronic device 110 can receive a music generation request, which can indicate the vocal track to be generated. The electronic device 110 can obtain the accompaniment audio 421 (i.e., reference music content), second lyrics content 426, and track conversion request. The electronic device 110 can input the accompaniment audio 421 to the audio separation model to separate the multiple tracks in the accompaniment audio 421, and encode the multiple tracks in the accompaniment audio by the audio encoder 423 to obtain the accompaniment audio encoding 424 (i.e., the first encoded representation). It should be understood that the accompaniment audio 421 can include multiple tracks, such as a piano track, a violin track, and a cello track, etc.

[0072] Further, the electronic device 110 can determine the vocal track to be generated based on the track conversion request, and generate a track conversion mask based on the vocal track to be generated.

[0073] Finally, the electronic device 110 can combine the accompaniment audio track encoding 424 with the audio track conversion mask to obtain an intermediate encoded representation 425.

[0074] In other scenarios, as shown in FIG. 4C, which illustrates a music generation process in an accompaniment augmentation scenario in an embodiment of the present application. The electronic device 110 can receive a music generation request, which can indicate to augment a certain content of an audio track. The electronic device 110 can obtain a reference music content, which includes a starting audio segment 431-1 and an ending audio segment 431-2 of a vocal track, and a preset time period specified by the user 140 to be augmented. The electronic device 110 can input the starting audio segment 431-1 and the ending audio segment 431-2 into audio encoders 432-1 and 432-2, respectively, to obtain a starting audio encoding 433-1 and an ending audio encoding 433-2. The electronic device 110 can combine the starting audio encoding 433-1 and the ending audio encoding 433-2 into a first encoded representation.

[0075] Further, the electronic device 110 can generate a corresponding time-audio track augmentation mask based on the preset time period specified by the user 140 to be augmented. The time-audio track augmentation mask can include an accompaniment mask and an intermediate audio encoding between the starting audio encoding 433-1 and the ending audio encoding 433-2.

[0076] Finally, the electronic device 110 can combine the starting audio encoding 433-1, the ending audio encoding 433-2, and the time-audio track augmentation mask to obtain an intermediate encoded representation 435.

[0077] In other scenarios, as shown in FIG. 4D, which illustrates a music generation process in an audio mixing scenario in an embodiment of the present application. The electronic device 110 can receive a music generation request, which can indicate audio segments of two different audio tracks to indicate a transition between the two audio track segments. The electronic device 110 can obtain a reference music content, which includes a starting audio segment 441-1 and an ending audio segment 441-2. The starting audio segment 441-1 indicates a vocal track, and the ending audio segment 441-2 indicates an accompaniment track.

[0078] After obtaining the starting audio segment 441-1 and the ending audio segment 441-2, the electronic device 110 can input the starting audio segment 441-1 and the ending audio segment 441-2 into an audio encoder 442-1 and an audio encoder 442-2, respectively, to obtain a starting audio encoding 443-1 and an ending audio encoding 443-2. The electronic device 110 can combine the starting audio encoding 443-1 and the ending audio encoding 443-2 into a first encoded representation.

[0079] Further, the electronic device 110 can determine, based on the start track encoding 443-1 and the end track encoding 443-2, that the vocal track needs to generate a subsequent portion of the start track encoding 443-1 and the accompaniment track needs to generate a preceding portion of the end track encoding 443-2. Based on this, the electronic device 110 can generate a corresponding mix mask to indicate the partial track to be generated.

[0080] Finally, the electronic device 110 can fill the mix mask to the partial track to be generated to obtain the intermediate encoded representation 435.

[0081] In some embodiments, the electronic device 110 can further generate the second encoded representation based on text content indicated by the music generation request, the text content describing including lyric content, style information, timbre information, and melody information.

[0082] At block 230, the electronic device 110 can process the intermediate encoded representation with the target model to generate the second encoded representation, the second encoded representation being generated by replacing the target portion in the intermediate encoded representation, the target portion being indicated by the target mask.

[0083] In some embodiments, the electronic device 110 can further generate the second encoded representation based on text content indicated by the music generation request, the text content describing including lyric content, style information, timbre information, and melody information.

[0084] For example, the electronic device 110 can receive a music description input by the user 140. The electronic device 110 can input the music description content into the target model when generating the second encoded representation to instruct the target model to generate the second encoded representation with reference to the music description content.

[0085] As an example, as shown in FIG. 4A, in a music augmentation scenario, the electronic device 110 can combine the first lyric content 407 obtained and the second lyric content to obtain the third lyric content 408. Further, the electronic device 110 can input the third lyric content 408 and the intermediate mask representation 406 into the target model 409 to obtain the second encoded representation. The second encoded representation includes the prelude audio encoding 403, the subsequent audio encoding of the vocal track, and the audio encoding of the accompaniment track.

[0086] As an example, as shown in FIG. 4B, in a track conversion scenario, the electronic device 110 can input the second lyric content 426 input by the user and the intermediate encoded representation 425 into the target model 427 to obtain the second encoded representation. The second encoded representation includes the accompaniment track encoding 424 and the vocal track generated by the target model 427 based on the track mask and the second lyric content 426.

[0087] As an example, as shown in FIG. 4C, in the accompaniment augmentation scenario, the electronic device 110 can input the intermediate surface representation 435 and the third lyric content 436 input by the user to the target model to obtain a second encoded representation. The second encoded representation includes the start audio encoding 433-1, the end audio encoding 433-2, and audio encodings of one or more audio tracks generated by the target model 437 based on the time-audio track augmentation mask.

[0088] As an example, as shown in FIG. 4D, in the audio mixing scenario, the electronic device 110 can input the intermediate surface representation 435 and the third lyric content 436 input by the user to the target model to obtain a second encoded representation. The second encoded representation includes the start audio encoding 443-1, the end audio encoding 443-2, an audio encoding of a prequel portion of the end audio encoding 443-2 generated by the target model 447 based on the mixing mask, and an audio encoding of a sequel portion of the start audio encoding 443-1 generated by the target model 447 based on the mixing mask.

[0089] With continued reference to FIG. 2, at block 240, the electronic device 110 can generate the target music content based on the second encoded representation.

[0090] In particular, the electronic device 110 can invoke an arbitrary audio decoder to decode the second encoded representation to generate the target music content. The present disclosure is not intended to limit the specific decoding process.

[0091] In some embodiments, the target model can be trained based on the following process: first, determine training audio track data corresponding to a plurality of audio tracks of a music sample; further, encode the training audio track data to generate first audio features; then, generate second audio features based on a constructed training mask and the first audio features, the training mask indicating audio tracks and / or audio track segments to be generated; further, process the second audio features using the target model to generate third audio features; and finally, train the target model based on a comparison of the first audio features and the third audio features.

[0092] In some embodiments, the third audio features are further generated based on text content associated with the music sample.

[0093] The training process of the target model discussed above will be further introduced below. FIG. 5 illustrates a training process 500 of a target model of an embodiment of the present disclosure. The training process 500 can be performed by a suitable training device, which can be the same as or different from the electronic device 110.

[0094] As shown in FIG. 5, the training device can obtain audio data 510 and text data 550 for training of a target model 570. For the text data 550, the training device can encode the text data 550 through a text encoder 560 to obtain text features 555.

[0095] For the audio data 510, the training device inputs the audio data to the audio track separation model 520 to separate multiple audio tracks of the audio data 510 to obtain multi-track audio data 514 (i.e., training track data). Further, the electronic device 100 can call the audio encoder 530 to encode the multi-track audio data 514 to obtain multi-track audio features 516 (i.e., first audio features). Still further, the training device can perform multi-functional mask processing 540 on the multi-track audio features 516 by using different types of masks to obtain audio training features (i.e., second audio features) of the target model 570.

[0096] Further, the training device can input the audio training data and the text features 555 to the target model 570 to train the target model 570 to obtain multi-track audio features 575 generated by the target model 570. Finally, the training device can calculate a loss function 580 of the target model 570 based on the multi-track audio features 516 and the generated multi-track audio features 575 (i.e., third audio features). The training device can further adjust parameters of the target model 570 according to the loss function 580.

[0097] In this way, the embodiments of the present disclosure can construct a corresponding mask matrix based on a music generation request, and can further generate corresponding music content by using the mask matrix. Thus, the embodiments of the present disclosure can enrich the music generation scenario and improve the quality of the music content.

[0098] In addition, the embodiments of the present disclosure indicate the target part corresponding to the content to be generated in the intermediate coding representation by using the mask matrix. Thus, the embodiments of the present disclosure can not interfere with other audio data in the music generation process, and can ensure the unity of the overall music style.

[0099] Example apparatus and device

[0100] Embodiments of the present disclosure also provide a corresponding apparatus for implementing the above method or process. FIG. 6 shows a schematic structural block diagram of an apparatus 600 for music generation according to certain embodiments of the present disclosure. The apparatus 600 can be implemented as or included in the electronic device 110. Various modules / components in the apparatus 600 can be implemented by hardware, software, firmware, or any combination thereof.

[0101] As shown in FIG. 6, the apparatus 600 includes a target mask generation module 610 configured to construct, based on the received music generation request, a target mask corresponding to the music generation request; an intermediate coded representation generation module 620 configured to generate, based on the target mask and a first coded representation of a reference music content, an intermediate coded representation; a second coded representation generation module 630 configured to process the intermediate coded representation by using a target model to generate a second coded representation, the second coded representation being generated by replacing a target portion in the intermediate coded representation, the target portion being indicated by the target mask; and a music content generation module 640 configured to generate, based on the second coded representation, a target music content.

[0102] In some embodiments, the target mask generation module 610 is further configured to determine, based on the music generation request, at least one audio track and / or at least one audio track segment to be generated, and construct the target mask based on the at least one audio track and / or the at least one audio track segment, wherein a mask portion in the target mask corresponding to the at least one audio track and / or the at least one audio track segment is set to a preset value.

[0103] In some embodiments, the target mask generation module 610 is further configured to determine, based on the music generation request, a first lyric content to be added, determine, based on tempo information of the reference music content, a first time length corresponding to the first lyric content, and determine that the at least one audio track segment to be generated includes a first audio track segment in a vocal track corresponding to the first time length.

[0104] In some embodiments, the target mask generation module 610 is further configured to determine, based on the music generation request, a second lyric content to be added, and in response to the reference music content not including a vocal track, determine that the at least one audio track to be generated includes the vocal track.

[0105] In some embodiments, the target mask generation module 610 is further configured to obtain, based on the music generation request, the reference music content, the reference music content including a plurality of audio track segments corresponding to a target audio track, and determine that the at least one audio track segment to be generated includes an audio track segment corresponding to a preset time period of the target audio track.

[0106] In some embodiments, the preset time period includes a time period specified by a user, or a time period determined based on a third lyric content indicated by the music generation request.

[0107] In some embodiments, the music generation request further indicates a set of audio tracks to be added.

[0108] In some embodiments, the target mask generation module 610 is further configured to: based on the music generation request, obtain reference music content, the reference music content comprising at least a second track segment corresponding to the first track and a third track segment corresponding to the second track; and determine that the at least one track segment to be generated comprises a fourth track segment corresponding to the first track and a fifth track segment corresponding to the second track.

[0109] In some embodiments, the first track is a vocal track, and a time length of the fourth track segment is determined based on fourth lyric content indicated by the music generation request.

[0110] In some embodiments, the target mask corresponds to a mask pattern determined based on the music generation request, the mask pattern comprising: a first mask pattern indicating to generate subsequent track data corresponding to the existing tracks based on the reference music content; a second mask pattern indicating to generate additional track data corresponding to the new track based on the reference music content; a third mask pattern indicating to generate subsequent track data corresponding to the existing at least one track and additional track data corresponding to the new track based on the reference music content; a fourth mask pattern indicating to generate additional track data corresponding to a plurality of tracks based on existing track data of the plurality of tracks of the reference music content, the additional track data of different tracks corresponding to different time periods; or a fifth mask pattern indicating to generate intermediate track data corresponding to the existing at least one track and additional track data corresponding to the new track based on the reference music content.

[0111] In some embodiments, the apparatus 600 further comprises a first encoded representation generation module configured to: in response to the reference audio content comprising a plurality of tracks, determine track data corresponding to the plurality of tracks using the separation module; and determine the first encoded representation by processing the track data corresponding to the plurality of tracks using an audio encoder.

[0112] In some embodiments, the second encoded representation is further generated based on text content indicated by the music generation request, the text content describing at least one of: lyric content, style information, timbre information, melody information.

[0113] In some embodiments, the target model is trained based on a process comprising: determining training track data corresponding to a plurality of tracks of a music sample; encoding the training track data to generate first audio features; generating second audio features based on a constructed training mask and the first audio features, the training mask indicating tracks and / or track segments to be generated; processing the second audio features using the target model to generate third audio features; and training the target model based on a comparison of the first audio features and the third audio features.

[0114] In some embodiments, the third audio features are further generated based on text content associated with the music sample.

[0115] The units included in the apparatus 600 can be implemented utilizing various means, including software, hardware, firmware, or any combination thereof. In some embodiments, one or more units can be implemented using software and / or firmware, e.g., machine-executable instructions stored on a storage medium. In addition or as an alternative, some or all of the units in the apparatus 600 can be implemented, at least partially, by one or more hardware logic components. For example, and without limitation, illustrative types of hardware logic components that can be used include field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip systems (SOCs), complex programmable logic devices (CPLDs), etc.

[0116] FIG. 7 illustrates a block diagram of an electronic device 700 in which one or more embodiments of the disclosure can be implemented. It should be understood that the electronic device 700 illustrated in FIG. 7 is merely exemplary and should not be construed as limiting the functionality and scope of the embodiments described herein. The electronic device 700 illustrated in FIG. 7 can be used to implement the electronic device 110 of FIG. 1.

[0117] As illustrated in FIG. 7, the electronic device 700 is in the form of a general electronic device. Components of the electronic device 700 can include, but are not limited to, one or more processors 710 or processing units, a memory 720, a storage device 730, one or more communication units 740, one or more input devices 750, and one or more output devices 760. The processor 710 can be a real or virtual processor and is capable of performing various processing according to programs stored in the memory 720. In a multi-processor system, multiple processors perform computer-executable instructions in parallel to improve parallel processing capabilities of the electronic device 700.

[0118] The electronic device 700 typically includes a number of computer storage media. Such media can be any available media that is accessible by the electronic device 700 and includes both volatile and non-volatile media, removable and non-removable media. The memory 720 can be a volatile memory (e.g., registers, cache, random access memory (RAM)), a non-volatile memory (e.g., read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. The storage device 730 can be a removable or non-removable medium and can include machine-readable media, such as a flash drive, a magnetic disk, or any other medium that can be used to store information and / or data and that can be accessed by the electronic device 700.

[0119] The electronic device 700 can further include additional detachable / non-detachable, volatile / non-volatile storage media. Although not shown in FIG. 7, a disk drive for reading from or writing to a detachable, non-volatile magnetic disk (e.g., a "floppy disk"), and an optical disk drive for reading from or writing to a detachable, non-volatile optical disk (e.g., a CD-ROM) can be provided. In these cases, each drive can be connected to the bus (not shown) by one or more data media interfaces. The memory 720 can include a computer program product 725 having one or more program modules configured to carry out the various methods or acts of the various embodiments of the present disclosure.

[0120] The communication unit 740 enables communication with other electronic devices through communication media. Additionally, the functionality of the components of the electronic device 700 can be implemented in a single computing cluster or a plurality of computer machines capable of communicating over a communication connection. As such, the electronic device 700 can operate in a networked environment using logical connections to one or more other servers, network personal computers (PCs), or another network nodes in the networking environment.

[0121] The input device 750 can be one or more input devices, such as a mouse, a keyboard, a trackball, etc. The output device 760 can be one or more output devices, such as a display, a speaker, a printer, etc. The electronic device 700 can also communicate with one or more external devices (not shown) such as a storage device, a display device, etc., one or more devices that enable a user to interact with the electronic device 700, or any devices (e.g., a network card, a modem, etc.) that enable the electronic device 700 to communicate with one or more other electronic devices, through the communication unit 740, as desired. Such communication can be carried out via an input / output (I / O) interface (not shown).

[0122] According to an example implementation of the present disclosure, there is provided a computer-readable storage medium having computer-executable instructions stored thereon, where the computer-executable instructions are executed by a processor to implement the method described above. According to an example implementation of the present disclosure, there is also provided a computer program product tangibly stored on a non-transitory computer-readable medium and comprising computer-executable instructions, where the computer-executable instructions are executed by a processor to implement the method described above.

[0123] Various aspects of the disclosure are now described with reference to the drawings. In general, the drawings described below are diagrammatic and schematic representations of actual or conceptual structures and processes, and are not limiting of the scope of the present disclosure. In the drawings, the size and relative positioning of components can be exaggerated for clarity and / or descriptive purposes. Also, the drawings represent examples of apparatuses and / or methods in accordance with the present disclosure. In some instances, various aspects of the disclosure can be shown in a diagram, or by a series of diagrams, and can include color, shading, and other graphical features that can make machines, components of machines, and / or computer program products appear as if they are real machines, components of machines, and / or computer program products. However, such graphical features are not intended to be limiting of the scope of the present disclosure. In addition, various aspects of the disclosure can be described in terms of logical, procedural, and / or functional steps that can be performed by a computer program product, such as a computer program product 725, as described above. These logical, procedural, and / or functional steps can be implemented in hardware, software, or a combination of hardware and software.

[0124] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0125] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0126] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0127] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0127] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0127] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0127] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0127] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0127] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0127] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0127] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0127] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0127] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0127] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0127] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0127] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0127] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0127] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0127] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0127] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0127] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0127] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the

Claims

1. A method for music generation, comprising constructing a mask matrix corresponding to a received music generation request based on the music generation request; generating an intermediate encoded representation based on the target mask and a first encoded representation of a reference music content; processing the intermediate encoded representation with a target model to generate a second encoded representation, the second encoded representation being generated by replacing target portions in the intermediate encoded representation, the target portions being indicated by the target mask; and generating a target music content based on the second encoded representation. 2.The method of claim 1, wherein the constructing a target mask corresponding to a received music generation request based on the music generation request comprises: determining at least one audio track and / or at least one audio track segment to be generated based on the music generation request; and constructing the target mask based on the at least one audio track and / or the at least one audio track segment, wherein mask portions in the target mask corresponding to the at least one audio track and / or the at least one audio track segment are set to a preset value. 3.The method of claim 2, wherein the determining at least one audio track and / or at least one audio track segment to be generated based on the music generation request comprises: determining a first lyric content to be added based on the music generation request; determining a first time length corresponding to the first lyric content based on tempo information of the reference music content; and determining the at least one audio track segment to be generated comprises a first audio track segment in a vocal audio track corresponding to the first time length. 4.The method of claim 2, wherein the determining at least one audio track and / or at least one audio track segment to be generated based on the music generation request comprises: determining a second lyric content to be added based on the music generation request; and in response to the reference music content not including a vocal audio track, determining the at least one audio track to be generated comprises the vocal audio track. 5.The method of claim 2, wherein the determining at least one audio track and / or at least one audio track segment to be generated based on the music generation request comprises: obtaining the reference music content based on the music generation request, the reference music content including a plurality of audio track segments corresponding to a target audio track; and determining the at least one audio track segment to be generated comprises an audio track segment corresponding to a preset time period of the target audio track. 6.The method of claim 5, wherein the preset time period comprises: a time period specified by a user; or a time period determined based on a third lyric content indicated by the music generation request. 7.The method of claim 5, wherein the music generation request further indicates a set of audio tracks to be added. 8.The method of claim 2, wherein the determining at least one audio track and / or at least one audio track segment to be generated based on the music generation request comprises: obtaining the reference music content based on the music generation request, the reference music content including at least a second audio track segment corresponding to a first audio track and a third audio track segment corresponding to a second audio track; and ​ ​ ​ ​ ​ determining the at least one audio track segment to be generated comprises a fourth audio track segment corresponding to the first audio track and a fifth audio track segment corresponding to the second audio track. 9.The method of claim 8, wherein the first audio track is a human voice track, and a time length of the fourth audio track segment is determined based on fourth lyric content indicated by the music generation request. 10.The method of claim 1, wherein the target mask corresponds to a mask pattern determined based on the music generation request, the mask pattern comprising: a first mask pattern indicating to generate subsequent audio track data corresponding to an existing audio track based on the reference music content; a second mask pattern indicating to generate additional audio track data corresponding to a new audio track based on the reference music content; a third mask pattern indicating to generate subsequent audio track data corresponding to at least one existing audio track and additional audio track data corresponding to a new audio track based on the reference music content; a fourth mask pattern indicating to generate additional audio track data corresponding to a plurality of audio tracks of the reference music content based on existing audio track data of the plurality of audio tracks, the additional audio track data of different audio tracks corresponding to different time periods; or a fifth mask pattern indicating to generate intermediate audio track data corresponding to at least one existing audio track and additional audio track data corresponding to a new audio track based on the reference music content. 11.The method of any one of claims 1 to 10, further comprising: determining audio track data corresponding to a plurality of audio tracks using a separation module in response to the reference audio content comprising the plurality of audio tracks; and processing the audio track data corresponding to the plurality of audio tracks using an audio encoder to determine the first encoded representation. 12.The method of claim 1, wherein the second encoded representation is further generated based on text content indicated by the music generation request, the text content describing at least one of: lyric content, style information, timbre information, melody information. 13.The method of claim 1, wherein the target model is trained based on a process comprising: determining training audio track data corresponding to a plurality of audio tracks of a music sample; encoding the training audio track data to generate first audio features; generating second audio features based on a constructed training mask and the first audio features, the training mask indicating audio tracks and / or audio track segments to be generated; processing the second audio features using the target model to generate third audio features; and training the target model based on a comparison of the first audio features and the third audio features. 14.The method of claim 13, wherein the third audio features are further generated based on text content associated with the music sample. 15.An apparatus for music generation, comprising: a target mask construction module configured to construct a target mask corresponding to a received music generation request based on the music generation request; an intermediate encoded representation generation module configured to generate an intermediate encoded representation based on the target mask and a first encoded representation of reference music content; and an audio generation module configured to generate a second encoded representation based on the intermediate encoded representation. ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ a second encoding representation generation module configured to process the intermediate encoding representation with a target model to generate a second encoding representation, the second encoding representation being generated by replacing a target portion in the intermediate encoding representation, the target portion being indicated by the target mask; and a target music generation module configured to generate target music content based on the second encoding representation.

16. An electronic device, comprising: at least one processor; and at least one memory coupled to the at least one processor and storing instructions for execution by the at least one processor, the instructions, when executed by the at least one processor, cause the electronic device to perform the method according to any one of claims 1-14.

17. A computer-readable storage medium having computer-executable instructions stored thereon that are executable by a processor to implement the method according to any one of claims 1-14.

18. A computer program product tangibly stored in a computer storage medium and comprising computer-executable instructions that, when executed by a device, cause the device to perform the method according to any one of claims 1-14.

Images