Music generation methods, devices and storage media

Abstract

This disclosure relates to a music generation method, apparatus, and storage medium. The method includes: determining chord information and rhythm pattern information corresponding to user-inputted music preference information; generating target MIDI data based on the chord information and rhythm pattern information; determining target rendering rules, the target rendering rules including timbre combinations and rendering order for MIDI data; and performing timbre rendering on the target MIDI data according to the target rendering rules to obtain target music. In this way, based on the chord information and rhythm pattern information corresponding to the user's music preferences, MIDI data that better matches the user's music preferences can be generated. Then, rendering according to the timbre combinations and order results in the final target music having richer musical information and a more complete musical structure. Furthermore, it can also meet the music requirements of DJing scenarios.

Description

Technical Field

[0001] This disclosure relates to the field of music synthesis, and more particularly to a music generation method, apparatus, and storage medium. Background Technology

[0002] Currently, in the field of music generation, the goal of synthesizing music is mainly achieved based on techniques such as rules, probabilistic models, and deep learning. Among these, due to the inherent limitations of rules and probabilistic models, music generation based on deep learning has become more popular in recent years. Currently, deep learning-based music generation primarily focuses on generating music such as piano pieces, single instrumental melodies, and pop arrangements. The generated music is relatively simple and cannot generate more complex music, such as electronic music used in DJ sets. Summary of the Invention

[0003] To overcome the problems existing in related technologies, this disclosure provides a music generation method, apparatus and storage medium.

[0004] According to a first aspect of the present disclosure, a music generation method is provided, the method comprising:

[0005] Based on the user's input music preference information, determine the chord information and rhythm pattern information corresponding to the music preference information;

[0006] Based on the chord information and the rhythm pattern information, generate target MIDI data;

[0007] Determine the target rendering rules, which include the timbre combination and rendering order for MIDI data timbre rendering;

[0008] According to the target rendering rules, the target MIDI data is rendered to obtain the target music.

[0009] Optionally, generating target MIDI data based on the chord information and the rhythm pattern information includes:

[0010] The chord information and rhythmic pattern information are input into the generation model to obtain the note distribution information of the preset track output by the generation model, wherein the generation model is obtained by training a generative adversarial network;

[0011] The target MIDI data is determined based on the note distribution information.

[0012] Optionally, the generation model includes at least one generation sub-model, and the generation sub-model corresponds one-to-one with the preset audio track;

[0013] The step of inputting the chord information and the rhythmic pattern information into the generation model to obtain the note distribution information of the preset track output by the generation model includes:

[0014] Obtain noise information;

[0015] The chord information, rhythmic information, and noise information are respectively input into each generation sub-model to obtain the note distribution information output by each generation sub-model, thereby obtaining the note distribution information of each preset track.

[0016] Optionally, the generative model is obtained in the following way:

[0017] Acquire first training data, which includes chord samples and rhythm pattern samples;

[0018] A training model is determined, the training model comprising a generation network and a discriminator corresponding to each of the preset audio tracks;

[0019] The target chord sample, the target rhythm pattern sample, and the acquired target noise are respectively input into each of the generator networks used in this training to obtain the first output result of each generator network. The target chord sample and the target rhythm pattern sample are taken from the chord sample and rhythm pattern sample in the first training data.

[0020] For each preset track, the first output result corresponding to the preset track, the target chord sample, and the target rhythm pattern sample are input to the discriminator corresponding to the preset track to obtain the second output result output by the discriminator;

[0021] If the training stop condition is not met, update each generator network used in this training based on the second output result to obtain the updated training model, and use the updated training model for the next training.

[0022] If the training stop condition is met, the generative networks used in this training are used as generative sub-models to obtain a generative model composed of the generative sub-models.

[0023] Optionally, the training model further includes a global discriminant network;

[0024] The step of updating each generator network used in this training based on the second output result includes:

[0025] Based on the second output result corresponding to each preset audio track, determine the first loss value of the generation network corresponding to each preset audio track;

[0026] Based on the first output result corresponding to each preset audio track, the target chord sample, and the target rhythm pattern sample, an input tensor is generated;

[0027] The input tensor is fed into the global discriminant network to obtain the third output result of the global discriminant network;

[0028] Based on the third output result, determine the second loss value;

[0029] Based on the first loss value and the second loss value, the network parameters of each generator network used in this training are updated.

[0030] Optionally, each of the generating sub-models includes a first generator for generating note start point distribution information and a second generator for generating note duration information.

[0031] Optionally, the note distribution information includes note starting point distribution information and sustained note position information;

[0032] Determining the target MIDI data based on the note distribution information includes:

[0033] The starting position information of the note and the position information of the sustained note are combined to obtain the combined result;

[0034] The merged result is then denoised to obtain the target MIDI data.

[0035] Optionally, determining the target rendering rules includes:

[0036] Based on the pre-defined correspondence between music preference information and rendering rules, the rendering rule corresponding to the music preference information input by the user is determined and used as the target rendering rule.

[0037] Optionally, determining the target rendering rules includes:

[0038] The music preference information is input into a pre-trained rule generation model to obtain the output of the rule generation model, which serves as the target rendering rule. The rule generation model is obtained by training a neural network model using second training data, which includes multiple sets of music preference information samples and rendering rule samples.

[0039] According to a second aspect of the present disclosure, a music generation apparatus is provided, the apparatus comprising:

[0040] The first determining module is configured to determine the chord information and rhythm pattern information corresponding to the music preference information input by the user.

[0041] The generation module is configured to generate target MIDI data based on the chord information and the rhythm pattern information;

[0042] The second determining module is configured to determine the target rendering rules, which include the timbre combination and rendering order for rendering MIDI data.

[0043] The rendering module is configured to perform timbre rendering on the target MIDI data according to the target rendering rules to obtain the target music.

[0044] According to a third aspect of the present disclosure, a music generation apparatus is provided, comprising:

[0045] processor;

[0046] Memory used to store processor-executable instructions;

[0047] The processor is configured as follows:

[0048] Based on the user's input music preference information, determine the chord information and rhythm pattern information corresponding to the music preference information;

[0049] Based on the chord information and the rhythm pattern information, generate target MIDI data;

[0050] Determine the target rendering rules, which include the timbre combination and rendering order for MIDI data timbre rendering;

[0051] According to the target rendering rules, the target MIDI data is rendered to obtain the target music.

[0052] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided that stores computer program instructions thereon, which, when executed by a processor, implement the steps of the music generation method provided in the first aspect of the present disclosure.

[0053] The technical solutions provided by the embodiments of this disclosure may include the following beneficial effects:

[0054] The above technical solution determines the chord and rhythm information corresponding to the user's music preferences. Then, based on the chord and rhythm information, target MIDI data is generated, and target rendering rules are determined. These rules include the timbre combinations and rendering order for the MIDI data, and the target MIDI data is rendered according to these rules to obtain the target music. Thus, based on the user's music preferences, corresponding chord and rhythm information is determined, and MIDI data matching these chord and rhythm information is generated. Then, based on the determined timbre combinations and rendering order, the MIDI data is rendered to generate the final target music. In this way, by generating MIDI data that better matches the user's music preferences based on the chord and rhythm information, and then rendering it according to the timbre combinations and order, the final target music has richer musical information and a more complete musical structure. Furthermore, it can also meet the musical needs of DJing scenarios.

[0055] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0056] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0057] Figure 1 This is a flowchart illustrating a music generation method according to an exemplary embodiment.

[0058] Figure 2 This is an exemplary flowchart of a music generation method provided according to an exemplary embodiment, which describes the step of generating target MIDI data based on chord information and rhythm pattern information.

[0059] Figure 3 This is an exemplary flowchart of a music generation method for generating a target model, provided according to an exemplary embodiment.

[0060] Figure 4 This is a block diagram of a music generation apparatus according to an exemplary embodiment.

[0061] Figure 5 This is a block diagram of a music generation apparatus according to an exemplary embodiment.

[0062] Figure 6 This is a block diagram of a music generation apparatus according to an exemplary embodiment. Detailed Implementation

[0063] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0064] Before introducing the methods provided in this disclosure, the application scenarios involved in the various embodiments of this disclosure will first be introduced. The application scenarios involved in this disclosure can be any scenario where music needs to be automatically generated for a user based on their requested needs (i.e., the user's music preferences). This application scenario can be that the electronic device generates corresponding music for the user based on the user's selection from provided music preference options, or it can be that the electronic device generates corresponding music for the user based on the user's actively input music preferences (e.g., text input, voice input, etc.). Taking the user actively inputting music preferences as an example, the user can activate the voice assistant function of the electronic device and voice input music preferences such as favorite music styles or music scenes. The electronic device generates music that matches the user's input music preferences, thus providing suitable music for the user. The electronic device can be a terminal or a backend server. For example, if the electronic device is a terminal, it can be a mobile terminal such as a smartphone, tablet, smartwatch, smart bracelet, or PDA (Personal Digital Assistant), or a fixed terminal such as a desktop computer.

[0065] Figure 1 This is a flowchart illustrating a music generation method according to an exemplary embodiment. As described above, the method provided in this disclosure can be applied to electronic devices, such as terminals or servers. Figure 1 As shown, the method may include steps 11 to 14.

[0066] In step 11, based on the music preference information input by the user, the chord information and rhythm pattern information corresponding to the music preference information are determined.

[0067] Music preference information can include music style and music context. Music style can be, for example, soothing or intense. Music context can be, for example, travel, lunch break, or exercise.

[0068] In one possible implementation, a correspondence between music preference information and chord information and rhythmic information can be pre-set, so that the corresponding chord information and rhythmic information can be determined based on the music preference information input by the user.

[0069] In another possible implementation, multiple chord and rhythm information labeled with music preference information can be obtained as training data to train a neural network model, resulting in a model capable of generating corresponding chord and rhythm information based on the input music preference information. In actual use, the user's input music preference information is fed into the model, and the model's output is the chord and rhythm information corresponding to the user's input music preference information.

[0070] Both chord information and rhythm pattern information can be matrices with the number of rows as the number of musical beats and the number of columns as the number of pitches. For example, both chord information and rhythm pattern information can be 32*128 matrices, where 32 represents the 32nd note and 128 represents the number of pitches.

[0071] In step 12, target MIDI data is generated based on chord information and rhythm pattern information.

[0072] In one possible implementation, step 12 may include steps 21 and 22, such as... Figure 2 As shown.

[0073] In step 21, chord information and rhythm pattern information are input into the generation model to obtain the note distribution information of the preset track output by the generation model.

[0074] The generative model is obtained by training a generative adversarial network (GAN). GANs are deep learning models and one of the most promising unsupervised learning methods on complex distributions in recent years. They generate better outputs through the game-like learning between (at least) two modules: a generative model and a discriminative model. In fact, chord information and rhythmic information are used as conditional information for training the GAN.

[0075] In one possible implementation, step 21 may include the following steps:

[0076] Obtain noise information;

[0077] Chord information, rhythmic information, and noise information are input into each generation sub-model to obtain the note distribution information output by each generation sub-model, thereby obtaining the note distribution information of each preset track.

[0078] The noise information can be random noise, such as noise randomly sampled from a standard normal distribution. The generation model can include at least one generation sub-model, and each generation sub-model corresponds one-to-one with a preset audio track. The MIDI data can be single-track MIDI data or multi-track MIDI data, where each track of the MIDI data corresponds to a different instrument.

[0079] For example, the generative model can be obtained through steps 31 to 36 as follows: Figure 3 As shown.

[0080] In step 31, the first training data is obtained.

[0081] The first training data may include chord samples and rhythm pattern samples. These samples can be manually annotated based on existing musical scores. For example, both chord and rhythm pattern samples can be matrices with the number of rows as the number of musical beats and the number of columns as the number of pitches. For example, both chord and rhythm pattern samples can be 32*128 matrices, where 32 represents a 32nd note and 128 represents the number of pitches. For chords, each note within the chord at each time step can be set to 1, while other positions are set to 0, to form a chord sample. For rhythm patterns, any pitch at the downbeat time step can be set to 1, while other positions in the matrix are set to 0, to form a rhythm pattern sample.

[0082] In step 32, the training model is determined.

[0083] The training model can include a generation network and a discriminator, each corresponding to a preset audio track. When determining the training model, its structure needs to be built based on the number of audio tracks in the final MIDI data to be generated. That is, each preset audio track corresponds to a set of generation networks and discriminators.

[0084] In step 33, the target chord sample, the target rhythm pattern sample, and the acquired target noise are respectively input into each generator network used in this training to obtain the first output result of each generator network.

[0085] The target chord samples are taken from the chord samples in the first training data, and the target rhythm pattern samples are taken from the rhythm pattern samples in the first training data. The target noise can be noise randomly sampled from a standard normal distribution.

[0086] Each generative network may include a first generator for generating note start point distribution information and a second generator for generating note duration information. Accordingly, the first output may be information characterizing the note distribution. The first output may be a matrix including note start point distribution information and sustained note position information, and the matrix contained in the first output may have the same matrix structure as the matrix corresponding to chord information and rhythmic pattern information. The note start point distribution information is used to identify the position where a note begins to appear, and the sustained note position information is used to indicate the positions where the note is sustained. For example, in the matrix corresponding to the note start point distribution information, the starting position of each note is 1, and other positions are 0; correspondingly, in the matrix corresponding to the sustained note position information, the position where the note is sustained is 1, and other positions are 0.

[0087] In step 34, for each preset track, the first output result, target chord sample, and target rhythm pattern sample corresponding to the preset track are input to the discriminator corresponding to the preset track to obtain the second output result output by the discriminator.

[0088] During the training of the generator network, the discriminator can use a discriminator with fixed internal parameters. The discriminator can be pre-trained using the first training data, that is, using the untrained generator network to generate output results, and feeding the output results and real MIDI data samples into the discriminator to train the discriminator's discrimination ability, and obtaining a trained discriminator with a certain discrimination ability. Its internal parameters are then fixed and used for training the generator network.

[0089] The first output consists of two matrices representing the note distribution, and the target chord sample and target rhythm pattern sample are also two matrices. Furthermore, all four matrices have the same matrix structure. Therefore, these four matrices can be combined to form a new matrix (or tensor) and input into the corresponding discriminator to obtain the discriminator's second output. The second output is the discriminator's score for the first output, reflecting whether the discriminator determines the first output to be real data. In this disclosure, the purpose of training the generative network is to enable it to generate data that is as realistic as possible, so realistic that the discriminator cannot distinguish whether the output of the generative network is real or fake. That is, the discriminator's score for the first output (i.e., the second output) is very close to the score of the real data.

[0090] In step 35, if the training stop condition is not met, the generator networks used in this training are updated according to the second output result to obtain the updated training model, and the updated training model is used for the next training.

[0091] For example, the conditions for stopping training could be, for instance, the training duration reaching a preset duration, the number of training iterations reaching a preset number, or the model loss value for this training iteration falling below a preset threshold.

[0092] In one possible implementation, if the training stop condition is not met, the first loss value of the generator network corresponding to each preset audio track can be determined based on the second output result corresponding to each preset audio track, and the generator network used this time can be updated using the first loss value.

[0093] Each second output corresponds to a score output by a discriminator, and the real data also has a corresponding score. For each second output, the first loss value can be determined using the score corresponding to the second output and the real data. For example, the cross-entropy loss function can be calculated using the scores corresponding to the second output and the real data to obtain the first loss value.

[0094] In another possible implementation, if the training model includes multiple generator networks, it may also include a global discriminator network for comprehensive evaluation based on the output of each generator network. Therefore, if the training stop condition is not met, the following steps may also be included:

[0095] Generate an input tensor based on the first output result, target chord sample, and target rhythm pattern sample corresponding to each preset audio track;

[0096] The input tensor is fed into the global discriminant network to obtain the third output result of the global discriminant network;

[0097] Based on the third output, determine the second loss value;

[0098] Based on the first and second loss values, the network parameters of each generator network used in this training are updated.

[0099] The global discriminant network is essentially a discriminator used to evaluate whether the outputs of the individual generative networks used in training can be considered as real data overall. Therefore, the third output is the score output by the global discriminant network for the input tensor. The second loss value is determined based on the third output of the global discriminant network and the score corresponding to the real data. For example, the cross-entropy loss function can be calculated using the third output and the score corresponding to the real data to obtain the second loss value.

[0100] In the above way, even if the training stop condition is not met, in addition to calculating the first loss value, the second loss value can also be calculated. Combining the two loss values ​​to update the generator network is beneficial to obtaining a high-performing generator network more quickly.

[0101] In step 36, if the training stop condition is met, the generative networks used in this training are used as generative sub-models to obtain a generative model composed of generative sub-models.

[0102] Referring to the structure of the generative network, each generative sub-model can also include a first generator for generating the distribution information of the starting point of the note and a second generator for generating the duration information of the note.

[0103] Back Figure 2 In step 22, the target MIDI data is determined based on the note distribution information.

[0104] In one possible implementation, step 22 may include the following steps:

[0105] The starting position information of the note and the position information of the sustained note are combined to obtain the merged result;

[0106] The merged results are then denoised to obtain the target MIDI data.

[0107] The note distribution information can include note start point distribution information and sustained note position information. As mentioned above, the note start position information and sustained note position information are two identical matrices, where the starting position of a note is 1, the sustained note position is 1, and the rest are 0. Therefore, by adding the two matrices, the start and duration of each note can be identified, i.e., the position of each note can be located. Thus, the note start position information and sustained note position information can be merged to obtain a combined result.

[0108] After obtaining the merged result, further denoising processing can be performed. The merged result contains three types of data: positions that are both the start and duration of a note are 2, positions that are only the duration of a note are 1, and positions that are neither are 0. Therefore, if a segment of data in the matrix is ​​211111, it means that this segment corresponds to a note. If a segment of data in the matrix is ​​01111, it is considered to be noise and can be denoised.

[0109] In step 13, the target rendering rules are determined.

[0110] The target rendering rules include the timbre combination and rendering order for MIDI data. For example, if MIDI data contains three tracks, corresponding to drums, chords, and bass, the target rendering rule could be to render the MIDI data twice in a loop, rendering only the chord track the first time, and rendering the chord, drum, and bass tracks the second time.

[0111] In one possible implementation, step 13 may include the following steps:

[0112] Based on the pre-defined correspondence between music preference information and rendering rules, the rendering rule corresponding to the user-input music preference information is determined and used as the target rendering rule.

[0113] In another possible implementation, step 13 may include the following steps:

[0114] Music preference information is input into a pre-trained rule generation model to obtain the output of the rule generation model, which is then used as the target rendering rule.

[0115] The rule generation model is obtained by training the neural network model using the second training data, which includes multiple sets of music preference information samples and rendering rule samples.

[0116] For example, a rule generation model can be generated as follows:

[0117] Build a neural network model;

[0118] Select a set of music preference information samples and rendering rule samples used in this training from the second training data, and use them as target music preference information samples and target rendering rule samples;

[0119] The target preference information sample is input into the neural network model used in this training to obtain the fourth output result of the neural network model;

[0120] If the conditions for stopping training are not met, the model loss value is calculated using the fourth output result and the target rendering rule sample, and the neural network model used in this training is updated according to the loss value. The updated neural network model is then used for the next training.

[0121] When the conditions for stopping training are met, the neural network model used in this training is determined to be a rule generation model.

[0122] In step 14, the target MIDI data is rendered with timbre according to the target rendering rules to obtain the target music.

[0123] The above technical solution determines the chord and rhythm information corresponding to the user's music preferences. Then, based on the chord and rhythm information, target MIDI data is generated, and target rendering rules are determined. These rules include the timbre combinations and rendering order for the MIDI data, and the target MIDI data is rendered according to these rules to obtain the target music. Thus, based on the user's music preferences, corresponding chord and rhythm information is determined, and MIDI data matching these chord and rhythm information is generated. Then, based on the determined timbre combinations and rendering order, the MIDI data is rendered to generate the final target music. In this way, by generating MIDI data that better matches the user's music preferences based on the chord and rhythm information, and then rendering it according to the timbre combinations and order, the final target music has richer musical information and a more complete musical structure. Furthermore, it can also meet the musical needs of DJing scenarios.

[0124] Figure 4 This is a block diagram of a music generation apparatus according to an exemplary embodiment. (Refer to...) Figure 4 The device 40 may include:

[0125] The first determining module 41 is configured to determine the chord information and rhythm pattern information corresponding to the music preference information input by the user.

[0126] Generation module 42 is configured to generate target MIDI data based on the chord information and the rhythm pattern information;

[0127] The second determining module 43 is configured to determine the target rendering rules, which include the timbre combination and rendering order for rendering MIDI data.

[0128] The rendering module 44 is configured to perform timbre rendering on the target MIDI data according to the target rendering rules to obtain the target music.

[0129] Optionally, the generation module 42 includes:

[0130] The first processing submodule is configured to input the chord information and the rhythm pattern information into the generation model to obtain the note distribution information of the preset track output by the generation model, wherein the generation model is obtained by training a generative adversarial network;

[0131] The first determining submodule is configured to determine the target MIDI data based on the note distribution information.

[0132] Optionally, the generation model includes at least one generation sub-model, and the generation sub-model corresponds one-to-one with the preset audio track;

[0133] The first processing submodule includes:

[0134] The acquisition submodule is configured to acquire noise information;

[0135] The second processing submodule is configured to input the chord information, the rhythmic information, and the noise information into each generation sub-model to obtain the note distribution information output by each generation sub-model, thereby obtaining the note distribution information of each preset track.

[0136] Optionally, the generative model is obtained through the following modules:

[0137] The acquisition module is configured to acquire first training data, which includes chord samples and rhythm pattern samples.

[0138] The third determining module is configured to determine the training model, which includes a generation network and a discriminator corresponding to each of the preset audio tracks.

[0139] The first processing module is configured to input the target chord sample, the target rhythm pattern sample, and the acquired target noise into each of the generator networks used in this training, and obtain the first output result of each generator network, wherein the target chord sample and the target rhythm pattern sample are taken from the chord sample and rhythm pattern sample in the first training data.

[0140] The second processing module is configured to input the first output result corresponding to the preset audio track, the target chord sample, and the target rhythm pattern sample to the discriminator corresponding to the preset audio track for each preset audio track, and obtain the second output result output by the discriminator. The second output result is the score of the discriminator on the first output result, and the score is used to reflect whether the discriminator determines that the first output result is real data.

[0141] The update module is configured to update each generator network used in the current training based on the second output result if the training stop condition is not met, so as to obtain an updated training model, and use the updated training model for the next training.

[0142] The fourth determining module is configured to, when the training stop condition is met, use the generative network used in this training as a generative sub-model to obtain a generative model composed of the generative sub-models.

[0143] Optionally, the training model further includes a global discriminant network;

[0144] The update module includes:

[0145] The second determining submodule is configured to determine the first loss value of the generator network corresponding to each preset audio track based on the second output result corresponding to each preset audio track.

[0146] The generation submodule is configured to generate an input tensor based on the first output result corresponding to each preset track, the target chord sample, and the target rhythm pattern sample;

[0147] The third processing submodule is configured to input the input tensor into the global discriminant network to obtain a third output result from the global discriminant network;

[0148] The third determination submodule is configured to determine the second loss value based on the third output result;

[0149] The update submodule is configured to update the network parameters of each generator network used in this training based on the first loss value and the second loss value.

[0150] Optionally, each of the generating sub-models includes a first generator for generating note start point distribution information and a second generator for generating note duration information.

[0151] Optionally, the note distribution information includes note starting point distribution information and sustained note position information;

[0152] The first determining submodule includes:

[0153] The merging submodule is configured to merge the note start position information and the sustained note position information to obtain a merging result;

[0154] A denoising submodule is configured to denoise the merged result to obtain the target MIDI data.

[0155] Optionally, the second determining module 43 includes:

[0156] The fourth determination submodule is configured to determine the rendering rule corresponding to the music preference information input by the user, based on the preset correspondence between music preference information and rendering rules, and use it as the target rendering rule.

[0157] Optionally, the second determining module 43 includes:

[0158] The fourth processing submodule is configured to input the music preference information into a pre-trained rule generation model to obtain the output of the rule generation model as the target rendering rule. The rule generation model is obtained by training a neural network model using second training data, which includes multiple sets of music preference information samples and rendering rule samples.

[0159] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0160] This disclosure also provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the steps of the music generation method provided in this disclosure.

[0161] Figure 5 This is a block diagram illustrating a music generation apparatus 800 according to an exemplary embodiment. For example, apparatus 800 may be a mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, medical device, fitness equipment, personal digital assistant, etc.

[0162] Reference Figure 5 The device 800 may include one or more of the following components: a processing component 802, a memory 804, a power component 806, a multimedia component 808, an audio component 810, an input / output (I / O) interface 812, a sensor component 814, and a communication component 816.

[0163] Processing component 802 typically controls the overall operation of device 800, such as operations associated with display, telephone calls, data communication, camera operation, and recording. Processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the music generation method described above. Furthermore, processing component 802 may include one or more modules to facilitate interaction between processing component 802 and other components. For example, processing component 802 may include a multimedia module to facilitate interaction between multimedia component 808 and processing component 802.

[0164] Memory 804 is configured to store various types of data to support the operation of device 800. Examples of such data include instructions for any application or method operating on device 800, contact data, phonebook data, messages, pictures, videos, etc. Memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0165] The power supply component 806 provides power to the various components of the device 800. The power supply component 806 may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power to the device 800.

[0166] Multimedia component 808 includes a screen that provides an output interface between the device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 808 includes a front-facing camera and / or a rear-facing camera. When the device 800 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0167] Audio component 810 is configured to output and / or input audio signals. For example, audio component 810 includes a microphone (MIC) configured to receive external audio signals when device 800 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 804 or transmitted via communication component 816. In some embodiments, audio component 810 also includes a speaker for outputting audio signals.

[0168] I / O interface 812 provides an interface between processing component 802 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0169] Sensor assembly 814 includes one or more sensors for providing status assessments of various aspects of device 800. For example, sensor assembly 814 may detect the on / off state of device 800, the relative positioning of components such as the display and keypad of device 800, changes in the position of device 800 or a component of device 800, the presence or absence of user contact with device 800, the orientation or acceleration / deceleration of device 800, and temperature changes of device 800. Sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 814 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 814 may also include an accelerometer, a gyroscope, a magnetometer, a pressure sensor, or a temperature sensor.

[0170] Communication component 816 is configured to facilitate wired or wireless communication between device 800 and other devices. Device 800 can access wireless networks based on communication standards, such as WiFi, 2G, or 3G, or combinations thereof. In one exemplary embodiment, communication component 816 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 816 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0171] In an exemplary embodiment, the apparatus 800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the music generation method described above.

[0172] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 804 including instructions, which can be executed by a processor 820 of the device 800 to complete the music generation method described above. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0173] In another exemplary embodiment, a computer program product is also provided, which includes a computer program executable by a programmable device, the computer program having a code portion for performing the music generation method described above when executed by the programmable device.

[0174] Figure 6 This is a block diagram illustrating a music generation apparatus 1900 according to an exemplary embodiment. For example, apparatus 1900 may be provided as a server. (Refer to...) Figure 6 The apparatus 1900 includes a processing component 1922, which further includes one or more processors, and memory resources represented by memory 1932 for storing instructions, such as application programs, that can be executed by the processing component 1922. The application programs stored in memory 1932 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 1922 is configured to execute instructions to perform the aforementioned music generation method.

[0175] Device 1900 may also include a power supply component 1926 configured to perform power management of device 1900, a wired or wireless network interface 1950 configured to connect device 1900 to a network, and an input / output (I / O) interface 1958. Device 1900 can operate on an operating system, such as Windows Server, stored in memory 1932. TM macOS X TM Unix TM Linux TM FreeBSD TM Or similar.

[0176] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of this disclosure. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0177] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A method for generating music, characterized in that, The method includes: Based on the user's input music preference information, determine the chord information and rhythm pattern information corresponding to the music preference information; Obtain noise information; input the chord information, rhythmic information, and noise information into each generation sub-model in the generation model to obtain the note distribution information output by each generation sub-model, thereby obtaining the note distribution information of each preset track. The generation model is obtained by training a generative adversarial network, and includes at least one generation sub-model, each corresponding to a preset track. Based on the note distribution information, determine the target MIDI data. Determine the target rendering rules, which include the timbre combination and rendering order for rendering the target MIDI data; According to the target rendering rules, the target MIDI data is rendered to obtain the target music.

2. The method according to claim 1, characterized in that, The generative model is obtained in the following way: Acquire first training data, which includes chord samples and rhythm pattern samples; A training model is determined, the training model comprising a generation network and a discriminator corresponding to each of the preset audio tracks; The target chord sample, the target rhythm pattern sample, and the acquired target noise are respectively input into each of the generator networks used in this training to obtain the first output result of each generator network. The target chord sample and the target rhythm pattern sample are taken from the chord sample and rhythm pattern sample in the first training data. For each preset track, the first output result corresponding to the preset track, the target chord sample, and the target rhythm pattern sample are input to the discriminator corresponding to the preset track to obtain the second output result output by the discriminator. The second output result is the score of the discriminator for the first output result. The score is used to reflect whether the discriminator determines that the first output result is real data. If the training stop condition is not met, update each generator network used in this training based on the second output result to obtain the updated training model, and use the updated training model for the next training. If the training stop condition is met, the generative networks used in this training are used as generative sub-models to obtain a generative model composed of the generative sub-models.

3. The method according to claim 2, characterized in that, The training model also includes a global discriminant network; The step of updating each generator network used in this training based on the second output result includes: Based on the second output result corresponding to each preset audio track, determine the first loss value of the generation network corresponding to each preset audio track; Based on the first output result corresponding to each preset audio track, the target chord sample, and the target rhythm pattern sample, an input tensor is generated; The input tensor is fed into the global discriminant network to obtain the third output result of the global discriminant network; Based on the third output result, determine the second loss value; Based on the first loss value and the second loss value, the network parameters of each generator network used in this training are updated.

4. The method according to claim 1, characterized in that, Each of the aforementioned sub-models includes a first generator for generating information on the distribution of note start points and a second generator for generating information on note duration.

5. The method according to claim 1, characterized in that, The note distribution information includes note starting point distribution information and sustained note position information; Determining the target MIDI data based on the note distribution information includes: The starting position information of the note and the position information of the sustained note are combined to obtain the combined result; The merged result is then denoised to obtain the target MIDI data.

6. The method according to claim 1, characterized in that, The rules for determining the target rendering include: Based on the pre-defined correspondence between music preference information and rendering rules, the rendering rule corresponding to the music preference information input by the user is determined and used as the target rendering rule.

7. The method according to claim 1, characterized in that, The rules for determining the target rendering include: The music preference information is input into a pre-trained rule generation model to obtain the output of the rule generation model, which serves as the target rendering rule. The rule generation model is obtained by training a neural network model using second training data, which includes multiple sets of music preference information samples and rendering rule samples.

8. A music generation device, characterized in that, The device includes: The first determining module is configured to determine the chord information and rhythm pattern information corresponding to the music preference information input by the user. A generation module is configured to acquire noise information; input the chord information, rhythmic information, and noise information into each generation sub-model in the generation model to obtain the note distribution information output by each generation sub-model, thereby obtaining the note distribution information of each preset track. The generation model is obtained by training a generative adversarial network, and includes at least one generation sub-model, each corresponding one-to-one with a preset track. Based on the note distribution information, target MIDI data is determined. The second determining module is configured to determine the target rendering rules, which include the timbre combination and rendering order for rendering the target MIDI data. The rendering module is configured to perform timbre rendering on the target MIDI data according to the target rendering rules to obtain the target music.

9. A music generation device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured as follows: Based on the user's input music preference information, determine the chord information and rhythm pattern information corresponding to the music preference information; Obtain noise information; input the chord information, rhythmic information, and noise information into each generation sub-model in the generation model to obtain the note distribution information output by each generation sub-model, thereby obtaining the note distribution information of each preset track. The generation model is obtained by training a generative adversarial network, and includes at least one generation sub-model, each corresponding to a preset track. Based on the note distribution information, determine the target MIDI data. Determine the target rendering rules, which include the timbre combination and rendering order for rendering the target MIDI data; According to the target rendering rules, the target MIDI data is rendered to obtain the target music.

10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When executed by a processor, the program instructions implement the steps of the method described in any one of claims 1 to 7.

Images