Method and device for generating music content, equipment and storage medium

Abstract

The embodiment of the invention relates to a method and device for generating music content, equipment and a storage medium. The method provided herein includes: acquiring a set of tokens generated based on input information; providing a set of tokens to the target model to generate a plurality of encoded representations corresponding to the plurality of tiles, where a target encoded representation corresponding to a first tile is generated based on a first set of attention parameters associated with a second tile, the second tile being earlier in time than the first tile; and generating the target music content by decoding the plurality of encoded representations. Thus, the embodiments of the present disclosure can generate music content in a streaming manner.

Description

Technical Field

[0001] The exemplary embodiments disclosed herein generally relate to the field of computers, and particularly to methods, apparatus, devices, and computer-readable storage media for generating music content. Background Technology

[0002] With the development of computer technology, music generation technology is gradually becoming a key technology in the fields of human-computer interaction and digital entertainment. Music generation technology refers to enabling machines to simulate the human music creation process through algorithms. Consequently, through music generation technology, people can create music more conveniently, even generating musical works through simple instructions or text descriptions. Summary of the Invention

[0003] In a first aspect of this disclosure, a method for generating music content is provided. The method includes: acquiring a set of tokens generated based on input information; providing the set of tokens to a target model to generate multiple encoded representations corresponding to multiple blocks, wherein the target encoded representation corresponding to a first block is generated based on a first set of attention parameters associated with a second block, the second block being earlier in time than the first block; and generating target music content by decoding the multiple encoded representations.

[0004] In a second aspect of this disclosure, an apparatus for generating music content is provided. The apparatus includes: an acquisition module configured to acquire a set of tokens generated based on input information; a providing module configured to provide the set of tokens to a target model to generate multiple encoded representations corresponding to multiple blocks, wherein a target encoded representation corresponding to a first block is generated based on a first set of attention parameters associated with a second block, the second block being earlier in time than the first block; and a generation module configured to generate target music content by decoding the multiple encoded representations.

[0005] In a third aspect of this disclosure, an electronic device is provided. The device includes at least one processing unit; and at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit. When executed by the at least one processing unit, the instructions cause the device to perform the method of the first aspect.

[0006] In a fourth aspect of this disclosure, a computer-readable storage medium is provided. The computer-readable storage medium stores a computer program that can be executed by a processor to implement the method of the first aspect.

[0007] It should be understood that the content described in this content section is not intended to limit the key or essential features of the embodiments of this disclosure, nor is it intended to restrict the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0008] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. In the drawings, the same or similar reference numerals denote the same or similar elements, wherein:

[0009] Figure 1 A schematic diagram of an example system for generating music content according to some embodiments of the present disclosure is shown;

[0010] Figure 2 A flowchart illustrating an example process for generating music content according to some embodiments of this disclosure is shown;

[0011] Figures 3A to 3C A schematic diagram of an attention mechanism according to some embodiments of the present disclosure is shown;

[0012] Figure 4 A schematic structural block diagram of an apparatus for generating music content according to some embodiments of the present disclosure is shown; and

[0013] Figure 5 A block diagram of an electronic device capable of implementing several embodiments of the present disclosure is shown. Detailed Implementation

[0014] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0015] It should be noted that the headings of any section / subsection provided herein are not limiting. Various embodiments are described throughout this document, and embodiments of any type may be included under any section / subsection. Furthermore, embodiments described in any section / subsection may be combined in any way with any other embodiments described in the same section / subsection and / or different sections / subsections.

[0016] In the description of embodiments of this disclosure, the term "comprising" and similar terms should be understood as open-ended inclusion, i.e., "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". Other explicit and implicit definitions may also be included below. The terms "first", "second", etc., may refer to different or the same objects. Other explicit and implicit definitions may also be included below.

[0017] As used in this paper, the term "model" refers to a system that learns the relationship between inputs and outputs from training data, enabling it to generate corresponding outputs for a given input after training. Model generation can be based on machine learning techniques. Deep learning is a machine learning algorithm that uses multiple layers of processing units to process inputs and provide corresponding outputs. In this paper, "model" may also be referred to as a "machine learning model," a "machine learning network," or simply a "network," and these terms are used interchangeably. A model can also include different types of processing units or networks.

[0018] As used herein, a “unit,” “operation unit,” or “subunit” can consist of any suitable machine learning model or network. As used herein, a set of elements or similar expressions can include one or more such elements. For example, “a set of convolutional units” can include one or more convolutional units.

[0019] The embodiments of this disclosure may involve user data, data acquisition, and / or use. All of these aspects comply with applicable laws, regulations, and relevant provisions. In the embodiments of this disclosure, all data collection, acquisition, processing, manipulation, forwarding, and use are conducted with the user's knowledge and confirmation. Accordingly, in implementing the embodiments of this disclosure, the type, scope of use, and usage scenarios of any data or information that may be involved should be communicated to the user and their authorization obtained in accordance with relevant laws and regulations through appropriate means. The specific methods of notification and / or authorization may vary depending on the actual situation and application scenario, and the scope of this disclosure is not limited in this respect.

[0020] In this specification and the embodiments, any processing of personal information will be carried out only under the premise of legality (such as obtaining the consent of the personal information subject, or being necessary for the performance of a contract), and will only be carried out within the scope stipulated or agreed upon. A user's refusal to process personal information other than that necessary for basic functions will not affect the user's use of basic functions.

[0021] In this specification and the embodiments, if the training and inference of the model are involved, the data involved (including but not limited to the data itself, the acquisition and / or use of the data) comply with the requirements of relevant laws, regulations and rules.

[0022] Despite advancements in music generation technology, challenges remain regarding sound quality improvement, generation speed, and the generation of long audio files. Traditional music generation models have limitations in generating high-quality, high-fidelity music clips and are inefficient in real-time response and processing of long audio data. Furthermore, generating long audio works requires models to understand and process extended audio information, which places higher demands on the diversity and richness of training data.

[0023] In view of this, embodiments of the present disclosure propose a scheme for generating music content. According to this scheme, a set of tokens generated based on input information can be obtained. Further, a set of tokens can be provided to a target model to generate multiple encoded representations corresponding to multiple blocks, wherein the target encoded representation corresponding to a first block is generated based on a first set of attention parameters associated with a second block, the second block being earlier than the first block. Additionally, target music content can be generated by decoding the multiple encoded representations.

[0024] Therefore, embodiments of this disclosure can generate music content in a streaming manner, thereby supporting partial playback of music content during the generation process. Furthermore, embodiments of this disclosure can reduce reliance on long audio training data, reducing the cost of data acquisition and training resources.

[0025] The following section provides a detailed description of various example implementations of this scheme, with reference to the accompanying drawings.

[0026] Example System

[0027] Figure 1 A schematic diagram of an example system 100 for generating music content according to some embodiments of the present disclosure is shown. System 100 may be deployed in or implemented using suitable electronic devices.

[0028] In some embodiments, the electronic device may include various types of computing systems / servers capable of providing computing power, and the electronic device may include terminal devices. Such terminal devices may be any type of mobile terminal, fixed terminal, or portable terminal, including mobile phones, desktop computers, laptop computers, notebook computers, netbook computers, tablet computers, media computers, multimedia tablets, personal communication system (PCS) devices, personal navigation devices, personal digital assistants (PDAs), audio / video players, digital cameras / camcorders, positioning devices, television receivers, radio receivers, e-book devices, gaming devices, or any combination thereof, including accessories and peripherals of these devices or any combination thereof. The electronic device may, for example, include various types of computing systems / servers capable of providing computing power, such as mainframes, edge computing nodes, computing devices in cloud environments, virtual machines, etc. Although shown as a single device, the electronic device may include multiple physical devices.

[0029] like Figure 1 As shown, system 100 may include language model 114. Language model 114 can acquire various types of input information, such as lyrics 102, music tags 104, audio 106, sheet music 108, and other input data 110.

[0030] In some embodiments, the language model 114 may process the above input information to generate a set of tokens.

[0031] In some embodiments, system 100 may also include an audio compressor (e.g., an audio tokenizer) 112. The audio compressor may, for example, process audio 106 to generate a set of audio tokens.

[0032] In some embodiments, the set of audio tokens may be provided as part of a token sequence input to the target model 116. Alternatively, the set of audio tokens may also be used as input to the language model 114 to generate a token sequence to be provided to the target model 116.

[0033] Furthermore, the target model 116 can process the received set of tokens and generate a corresponding encoded representation. In some embodiments, the target model 116 can generate multiple encoded representations corresponding to multiple blocks, each block for example corresponding to a preset duration.

[0034] As an example, target model 116 can sequentially generate coded representations of 0 to 4 seconds, and subsequently generate coded representations of 4 to 8 seconds. Further, system 100 may include audio decoder 118. Audio decoder 118 can decode the generated coded representations into corresponding audio segments, ultimately obtaining the generated music content 120.

[0035] In this way, by sequentially generating the encoded representations corresponding to multiple blocks, the music content 120 can be generated in a streaming manner, thereby improving the efficiency of music content generation.

[0036] The following will combine Figure 2 Let us describe in detail the specific processing procedure of target model 116.

[0037] Example process

[0038] Figure 2 A flowchart of an example process 200 for generating music content according to some embodiments of the present disclosure is shown. Process 200 can be implemented at system 100. Reference is made below. Figure 1 To describe process 200.

[0039] As shown in the figure, in box 210, system 100 obtains a set of tokens generated based on the input information.

[0040] For reference Figure 1 As discussed, system 100 can use audio compressor 112 to generate the encoded representation corresponding to the audio information, and can use language model 114 to generate a set of tokens. Further, system 110 can provide this set of tokens to target model 116.

[0041] In some embodiments, the target model 116 may include, for example, a diffusion model.

[0042] In box 220, system 100 provides a set of tokens to target model 114 to generate multiple encoded representations corresponding to multiple blocks, wherein the target encoded representation corresponding to the first block is generated based on a first set of attention parameters associated with the second block, the second block being earlier than the first block.

[0043] In some embodiments, the target model 114 can generate corresponding encoded representations in blocks. For example, the target model 114 can first generate encoded representations for 0 to 4 seconds, and then generate encoded representations for 4 to 8 seconds.

[0044] In some embodiments, the 4- to 8-second encoded representation can be generated based on the 0- to 4-second encoded representation. As will be described in detail below, when generating the 4- to 8-second encoded representation, the target model 114 can determine the attention information for the 4- to 8-second period based on the relevant attention parameters for the 0- to 4-second period to generate the 4- to 8-second encoded representation.

[0045] The following will be referenced Figures 3A to 3B To describe the specific process of box 220. Figure 3A A schematic diagram is shown of the encoded representation of target model 114 for 0 to 4 seconds (i.e., the second block).

[0046] like Figure 3A As shown, system 100 can obtain input 302 from 0 to 4 seconds and can determine the value parameter 304, key parameter 306 and query parameter 308 accordingly.

[0047] Furthermore, system 100 can determine query-key pairs 310 based on key parameter 306 and query parameter 308, and thus determine attention information 312. Accordingly, system 100 can generate an encoded representation of 0 to 4 seconds, i.e., output 314, based on value parameter 304 and attention information 312.

[0048] In some embodiments, such as Figure 3A As shown, system 100 can also write a set of attention parameters corresponding to blocks of 0 to 4 seconds to the cache module, such as cache value parameter 316 and cache key parameter 318.

[0049] Figure 3B A schematic diagram shows the encoded representation of target model 114 for 4 to 8 seconds (i.e., the first block). For example... Figure 3B As shown, similar to Figure 3A As shown in the process, system 100 can obtain input 320 for 4 to 8 seconds and determine the value parameter 322, key parameter 324 and query parameter 326 corresponding to the block.

[0050] Different from Figure 3A As shown in the process, system 100 can obtain the cached value parameter 316 and key parameter 318 from the cache module, and update the value parameter 322 and key parameter 330 corresponding to the first block (i.e., 4 to 8 seconds) accordingly.

[0051] For example, system 100 can concatenate the value parameter 316 for 0 to 4 seconds to the value parameter 322 for 4 to 8 seconds to obtain an updated value parameter. Additionally, system 100 can concatenate the key parameter 318 for 0 to 4 seconds to the key parameter 324 for 4 to 8 seconds to obtain an updated key parameter.

[0052] Furthermore, system 100 can determine query-key pairs 332 based on the updated key parameters and query parameters 326, and accordingly determine attention information 334 corresponding to 4 to 8 seconds.

[0053] Furthermore, system 100 can determine output 336, i.e., the encoded representation corresponding to 4 to 8 seconds, based on attention information 334 and updated value parameters.

[0054] Similarly, system 100 can also write the value parameter and key parameter corresponding to 4 to 8 seconds into the cache module as the cached value parameter 338 and the cached key parameter 340.

[0055] Based on a similar process, in the process of generating an 8 to 12-second encoded representation, the system 100 can also obtain value parameters for 0 to 4 seconds and value parameters for 4 to 8 seconds, as well as key parameters for 0 to 4 seconds and key parameters for 4 to 8 seconds from the cache, to generate an 8 to 12-second encoded representation.

[0056] In this way, embodiments of the present disclosure can achieve the generation of block-based coded representations based on a local attention mechanism.

[0057] Continue to refer to Figure 2 In box 230, system 100 generates target music content by decoding multiple encoded representations.

[0058] As an example, system 100 may use audio decoder 118 to decode multiple encoded representations to generate target music content. In some embodiments, the target music content may be provided, for example, in a streaming manner.

[0059] The training process of the target model 116 will be further described below. In some embodiments, the system 100 may use training audio content to train the target model 116.

[0060] Specifically, system 100 can determine a reference coded representation of the training audio content. For example, system 100 can use a trained audio encoder to generate a reference coded representation of the training audio content.

[0061] Furthermore, system 100 can utilize target model 116 to process a set of training tokens of the training audio content to generate a training coded representation. In some embodiments, system 100 can utilize a trained audio compressor 112 to generate a coded representation corresponding to the training audio content, and can utilize language model 114 to generate a set of tokens.

[0062] Furthermore, system 100 can provide the set of training tokens to target model 116 to generate training encoded representations. Additionally, system 100 can train target model 116 based on the difference between the training encoded representation and the reference encoded representation.

[0063] For example, system 100 can adjust the parameters of target model 116 based on the L2 distance between the training encoding representation and the reference encoding representation.

[0064] In some embodiments, in order to support the local attention mechanism of the target model 116, the system 100 may also implement a mask-based attention mechanism during training.

[0065] Specifically, Figure 3C A schematic diagram of the training target model is shown. (For example...) Figure 3C As shown, during the training process, system 100 can similarly determine value parameter 352, key parameter 354, and query parameter 356 based on input 350.

[0066] Additionally, system 100 can determine query-key pair 358 based on key parameter 354 and query parameter 356. Additionally, system 100 can determine attention information 360 corresponding to the target block to be generated based on query-key pair 358 and mask 362.

[0067] For example, during the training phase, in the process of generating a 4- to 8-second encoded representation, mask 362 can instruct attention information 360 to be determined based on attention parameters of at least one block (e.g., 0 to 4 seconds) associated with the target block (4 to 8 seconds). Alternatively, mask 362 can also instruct attention information 360 to be determined solely based on the target block (4 to 8 seconds) itself.

[0068] Similarly, during the training phase, in the process of generating an 8- to 12-second encoded representation, mask 362 can instruct attention information 360 to be determined based on attention parameters of at least one block (e.g., 0 to 4 seconds and 4 to 8 seconds) associated with the target block (8 to 12 seconds). Alternatively, mask 362 can also instruct attention information 360 to be determined solely based on the target block (8 to 12 seconds) itself.

[0069] Therefore, during training, block masks ensure that the model can only access historical data within a certain time frame when computing the output of the current block, rather than the entire long audio sequence. This approach reduces the model's dependence on long audio data during training, because even short audio datasets can be used to train the model, as long as they provide sufficient local contextual information.

[0070] Based on the process described above, embodiments of this disclosure can generate music content in a streaming manner, thereby supporting partial playback of the music content during the generation process. Furthermore, embodiments of this disclosure can reduce reliance on long audio training data, reducing the cost of data acquisition and training resources.

[0071] Example devices and equipment

[0072] Embodiments of this disclosure also provide corresponding apparatus for implementing the above methods or processes. Figure 4A schematic structural block diagram of an apparatus 400 for generating music content according to certain embodiments of the present disclosure is shown. The apparatus 400 may be implemented as or included in system 100. Various modules / components in the apparatus 400 may be implemented by hardware, software, firmware, or any combination thereof.

[0073] like Figure 4 As shown, the device 400 includes: an acquisition module 410 configured to acquire a set of tokens generated based on input information; a providing module 420 configured to provide a set of tokens to a target model to generate multiple encoded representations corresponding to multiple blocks, wherein the target encoded representation corresponding to a first block is generated based on a first set of attention parameters associated with a second block, the second block being earlier than the first block; and a generation module 430 configured to generate target music content by decoding the multiple encoded representations.

[0074] In some embodiments, the acquisition module 410 is further configured to: process at least a portion of the input information using a language model to generate at least a portion of the tokens in a set of tokens; and / or process the audio content of the input information using an audio compressor to generate at least a portion of the tokens in a set of tokens.

[0075] In some embodiments, the apparatus 400 further includes an attention module configured to write a first set of attention parameters corresponding to the second block to a cache module during the generation of the encoded representation of the second block.

[0076] In some embodiments, the attention module is further configured to: obtain a first set of attention parameters corresponding to the second block from the cache module during the process of generating the target encoding representation of the first block; update a second set of attention parameters corresponding to the first block based on the first set of attention parameters; determine first attention information corresponding to the first block based on the updated second set of attention parameters; and generate a target encoding representation corresponding to the first block based on the first attention information.

[0077] In some embodiments, the attention module is further configured to concatenate the first set of attention parameters to the second set of attention parameters in order to update the second set of attention parameters.

[0078] In some embodiments, the attention module is further configured to write a second set of attention parameters to the caching module.

[0079] In some embodiments, the first set of attention parameters includes a set of key-value parameters corresponding to the second block.

[0080] In some embodiments, the target model is trained based on the following process: determining a reference coded representation of the training audio content; processing a set of training tokens of the training audio content using the target model to generate a training coded representation; and training the target model based on the difference between the reference coded representation and the training coded representation.

[0081] In some embodiments, processing a set of training tokens for training audio content using a target model includes: determining a target mask corresponding to a target block; determining second attention information corresponding to the target block based on the target mask; and generating a target training encoded representation corresponding to the target block based on the second attention information.

[0082] In some embodiments, the target mask instruction determines second attention information based on the attention parameters of at least one block associated with the target block, wherein the time of at least one block is before that of the target block.

[0083] In some embodiments, the target model includes a diffusion model.

[0084] The modules included in device 400 can be implemented in various ways, including software, hardware, firmware, or any combination thereof. In some embodiments, one or more units can be implemented using software and / or firmware, such as machine-executable instructions stored on a storage medium. In addition to or as an alternative to machine-executable instructions, some or all of the modules in device 400 can be implemented at least partially by one or more hardware logic components. By way of example, and not limitation, exemplary types of hardware logic components that can be used include field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.

[0085] Figure 5 A block diagram of an electronic device 500 in which one or more embodiments of the present disclosure may be implemented is shown. It should be understood that... Figure 5 The electronic device 500 shown is merely exemplary and should not be construed as limiting the functionality and scope of the embodiments described herein. Figure 5 The electronic device 500 shown can be used to achieve Figure 1 System 100.

[0086] like Figure 5As shown, electronic device 500 is in the form of a general-purpose electronic device. Components of electronic device 500 may include, but are not limited to, one or more processors or processing units 510, memory 520, storage device 530, one or more communication units 540, one or more input devices 550, and one or more output devices 560. Processing unit 510 may be a physical or virtual processor and is capable of performing various processes according to programs stored in memory 520. In a multiprocessor system, multiple processing units execute computer-executable instructions in parallel to improve the parallel processing capability of electronic device 500.

[0087] Electronic device 500 typically includes multiple computer storage media. Such media can be any accessible media that is accessible to electronic device 500, including but not limited to volatile and non-volatile media, removable and non-removable media. Memory 520 can be volatile memory (e.g., registers, cache, random access memory (RAM)), non-volatile memory (e.g., read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. Storage device 530 can be removable or non-removable media and can include machine-readable media, such as flash drives, disks, or any other media that can be used to store information and / or data and can be accessed within electronic device 500.

[0088] Electronic device 500 may further include additional removable / non-removable, volatile / non-volatile storage media. Although not explicitly stated... Figure 5 As shown, disk drives for reading from or writing to removable, non-volatile disks (e.g., "floppy disks") and optical disk drives for reading from or writing to removable, non-volatile optical disks can be provided. In these cases, each drive can be connected to a bus (not shown) via one or more data media interfaces. Memory 520 may include computer program product 525 having one or more program modules configured to perform various methods or actions of various embodiments of this disclosure.

[0089] Communication unit 540 enables communication with other electronic devices via a communication medium. Additionally, the functionality of components of electronic device 500 can be implemented using a single computing cluster or multiple computing machines capable of communicating via communication connections. Therefore, electronic device 500 can operate in a networked environment using logical connections to one or more other servers, network personal computers (PCs), or another network node.

[0090] Input device 550 can be one or more input devices, such as a mouse, keyboard, trackball, etc. Output device 560 can be one or more output devices, such as a monitor, speaker, printer, etc. Electronic device 500 can also communicate with one or more external devices (not shown) via communication unit 540 as needed. These external devices include storage devices, display devices, etc., and can communicate with one or more devices that enable user interaction with electronic device 500, or with any device that enables electronic device 500 to communicate with one or more other electronic devices (e.g., network card, modem, etc.). Such communication can be performed via input / output (I / O) interface (not shown).

[0091] According to an exemplary implementation of this disclosure, a computer-readable storage medium is provided that stores computer-executable instructions thereon, wherein the computer-executable instructions are executed by a processor to implement the methods described above. According to an exemplary implementation of this disclosure, a computer program product is also provided, which is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions, which are executed by a processor to implement the methods described above.

[0092] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatuses, devices, and computer program products implemented according to this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0093] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processing unit of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0094] Computer-readable program instructions can be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions that execute on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0095] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction, which contains one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0096] Various implementations of this disclosure have been described above. The foregoing description is exemplary and not exhaustive, nor is it limited to the disclosed implementations. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described implementations. The terminology used herein is chosen to best explain the principles, practical applications, or improvements to technology in the market, or to enable others skilled in the art to understand the various implementations disclosed herein.

Claims

1. A method of generating music content, comprising: obtaining a set of tokens generated based on input information; providing the set of tokens to a target model to generate a plurality of encoded representations corresponding to a plurality of chunks, wherein a target encoded representation corresponding to a first chunk is generated based on a first set of attention parameters associated with a second chunk, the second chunk being earlier in time than the first chunk; and generating target music content by decoding the plurality of encoded representations.

2. The method of claim 1, wherein obtaining a set of tokens generated based on input information comprises: processing at least part of the input information with a language model to generate at least part of the set of tokens; and / or processing audio content of the input information with an audio compressor to generate at least part of the set of tokens.

3. The method of claim 1, further comprising: writing, to a cache module, the first set of attention parameters corresponding to the second chunk in a process of generating an encoded representation of the second chunk.

4. The method of claim 3, further comprising: obtaining, from the cache module, the first set of attention parameters corresponding to the second chunk in a process of generating the target encoded representation of the first chunk; updating a second set of attention parameters corresponding to the first chunk based on the first set of attention parameters; determining the first attention information corresponding to the first chunk based on the updated second set of attention parameters; and generating the target encoded representation based on the first attention information.

5. The method of claim 4, wherein updating a second set of attention parameters corresponding to the first chunk based on the first set of attention parameters comprises: concatenating the first set of attention parameters to the second set of attention parameters to update the second set of attention parameters.

6. The method of claim 4, further comprising: writing the second set of attention parameters to a cache module.

7. The method of claim 1, wherein the first set of attention parameters comprises a set of key-value parameters corresponding to the second chunk.

8. The method of claim 1, wherein the target model is trained based on a process of: determining a reference encoded representation of training audio content; processing a set of training tokens of the training audio content with the target model to generate a training encoded representation; and training the target model based on a difference between the reference encoded representation and the training encoded representation.

9. The method of claim 8, wherein processing a set of training tokens of the training audio content with the target model comprises: determining a target mask corresponding to a target chunk; determining second attention information corresponding to the target chunk based on the target mask; and generating a target training encoded representation corresponding to the target chunk based on the second attention information. ​ ​ ​ ​ ​ 10. The method of claim 9, wherein the target mask indicates that the second attention information is determined based on an attention parameter of at least one patch associated with the target patch, wherein the at least one patch is earlier in time than the target patch.

11. The method of claim 1, wherein the target model comprises a diffusion model.

12. An apparatus for generating music content, comprising: an obtaining module configured to obtain a set of tokens generated based on input information; a providing module configured to provide the set of tokens to a target model to generate a plurality of encoded representations corresponding to a plurality of patches, wherein a target encoded representation corresponding to a first patch is generated based on a first set of attention parameters associated with a second patch, the second patch being earlier in time than the first patch; and a generating module configured to generate target music content by decoding the plurality of encoded representations.

13. An electronic device, comprising: at least one processing unit; and at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions, when executed by the at least one processing unit, cause the electronic device to perform the method of any of claims 1-11.

14. A computer-readable storage medium having stored thereon a computer program, the computer program being executable by a processor to implement the method of any of claims 1-11. ​ ​

Images