Music melody generation method, model training method, device and medium

Abstract

The present disclosure relates to a method for generating a music melody, a training method for a model, an apparatus and a medium. The method for generating a music melody comprises: obtaining a target rhythm sequence, the target rhythm sequence comprising a plurality of position symbols and a duration corresponding to each of the position symbols, each of the position symbols comprising a placeholder and a rest, the placeholder representing a pitch to be filled in a corresponding position, and the rest representing that a sound does not exist in the corresponding position; inputting the target rhythm sequence into a melody generation model to obtain a pitch sequence, the pitch sequence comprising pitches respectively corresponding to the positions of the placeholders in the target rhythm sequence; and generating a target music melody based on the target rhythm sequence and the pitch sequence.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and in particular to a method for generating a music melody, a model training method, a device and a medium. Background Art

[0002] At present, the development of music generation mainly includes methods based on rules, probability models, deep learning and other technologies. The limitations of rules and probability models themselves are too great. Therefore, in recent years, the research on music generation has tended to focus on deep learning. In related technologies, deep learning models can be used to generate music melodies. However, this model performs poorly in terms of rhythm in generation tasks, and there are a large number of invalid generation steps and generated content. Summary of the invention

[0003] In order to overcome the problems existing in the related art, the present disclosure provides a method for generating a music melody, a model training method, a device and a medium.

[0004] According to a first aspect of an embodiment of the present disclosure, there is provided a method for generating a music melody, comprising:

[0005] Acquire a target rhythm sequence, wherein the target rhythm sequence includes a plurality of position symbols and a duration corresponding to each position symbol, wherein each position symbol includes a placeholder and a rest, wherein the placeholder represents a pitch to be filled in a corresponding position, and the rest represents that a pitch does not exist in the corresponding position;

[0006] The target rhythm sequence is input into a melody generation model to obtain a pitch sequence, wherein the pitch sequence includes pitches corresponding to the positions of the placeholders in the target rhythm sequence; wherein the melody generation model is obtained by training first training data using machine learning, wherein the first training data includes: a plurality of first sample rhythm sequences and a first label corresponding to each of the first sample rhythm sequences, each of the first sample rhythm sequences includes a plurality of sample position symbols and a duration corresponding to each of the sample position symbols, each of the sample position symbols includes a sample placeholder and a sample rest, the sample placeholder represents a pitch to be filled in at a corresponding position, the sample rest represents that a sound does not exist at the corresponding position, and the first label represents the pitch sequence;

[0007] A target music melody is generated based on the target rhythm sequence and the pitch sequence.

[0008] In some embodiments, obtaining the target rhythm sequence includes:

[0009] Acquire an initial rhythm sequence, the initial rhythm sequence comprising a plurality of position symbols and a duration corresponding to each position symbol, each position symbol comprising a placeholder and a rest, the placeholder representing a pitch to be filled in a corresponding position, and the rest representing that a pitch does not exist in the corresponding position;

[0010] The initial rhythm sequence is input into the rhythm generation model to obtain the target rhythm sequence, and the length of the target rhythm sequence is greater than that of the initial rhythm sequence; wherein the rhythm generation model is obtained by training the second training data using machine learning, the second training data includes multiple second sample rhythm sequences and a second label corresponding to each of the second sample rhythm sequences, each of the second sample rhythm sequences includes multiple sample position symbols and a time value corresponding to each of the sample position symbols, each of the sample position symbols includes a sample placeholder and a sample rest, the sample placeholder represents the pitch to be filled in the corresponding position, the sample rest represents the absence of a tone at the corresponding position, and the second label represents a rhythm sequence whose length is greater than that of the second sample rhythm sequence.

[0011] In some embodiments, inputting the target rhythm sequence into a melody generation model to obtain a pitch sequence includes:

[0012] Iteratively executing the steps of determining a vector of a current sequence to be processed based on the target rhythm sequence, and inputting the vector of the current sequence to be processed into the melody generation model to obtain the pitch corresponding to the position of the corresponding placeholder, until the pitches corresponding to the positions of all placeholders in the target rhythm sequence are obtained;

[0013] Obtaining the pitch sequence based on the pitches corresponding to the positions of all placeholders in the target rhythm sequence;

[0014] Among them, when the iterative execution is the first round, the current sequence to be processed is obtained based on the target rhythm sequence and the filling start mark, and the filling start mark is located at the end of the target rhythm sequence; when the iterative execution is not the first round, the current sequence to be processed is obtained based on the target rhythm sequence, the filling start mark and the pitch obtained in the history round, and the pitch obtained in the history round is located after the filling start mark.

[0015] In some embodiments, when the iterative execution is not the first round, determining the vector of the current sequence to be processed based on the target rhythm sequence includes:

[0016] Based on the target rhythm sequence, the filling start mark and the pitch obtained by the historical round, the current sequence to be processed is determined, and the current sequence to be processed is encoded to obtain an initial vector of the current sequence to be processed;

[0017] The initial vector of the current sequence to be processed and a preset category vector are fused to obtain the vector of the current sequence to be processed.

[0018] In some embodiments, the melody generation model includes a Transformer model, and the rhythm generation model includes a Transformer model.

[0019] According to a second aspect of an embodiment of the present disclosure, a method for training a melody generation model is provided, comprising:

[0020] Acquire first training data, wherein the first training data includes a plurality of first sample rhythm sequences and a first label corresponding to each of the first sample rhythm sequences, each of the first sample rhythm sequences includes a plurality of sample position symbols and a duration corresponding to each of the sample position symbols, each of the sample position symbols includes a sample placeholder and a sample rest, the sample placeholder represents a pitch to be filled in a corresponding position, the sample rest represents that a sound does not exist in the corresponding position, and the first label represents a pitch sequence;

[0021] The melody generation model is obtained by performing model training on the initial melody generation model using the first training data.

[0022] According to a third aspect of an embodiment of the present disclosure, a method for training a rhythm generation model is provided, comprising:

[0023] Acquire second training data, wherein the second training data includes a plurality of second sample rhythm sequences and a second label corresponding to each of the second sample rhythm sequences, each of the second sample rhythm sequences includes a plurality of sample position symbols and a duration corresponding to each of the sample position symbols, each of the sample position symbols includes a sample placeholder and a sample rest, the sample placeholder represents a pitch to be filled in a corresponding position, the sample rest represents that a tone does not exist in a corresponding position, and the second label represents a rhythm sequence having a length greater than that of the second sample rhythm sequence;

[0024] The initial rhythm generation model is trained using the second training data to obtain the rhythm generation model.

[0025] According to a fourth aspect of an embodiment of the present disclosure, there is provided a device for generating a music melody, comprising:

[0026] A first acquisition module is configured to acquire a target rhythm sequence, wherein the target rhythm sequence includes a plurality of position symbols and a duration corresponding to each position symbol, wherein each position symbol includes a placeholder and a rest, wherein the placeholder represents a pitch to be filled in a corresponding position, and the rest represents that a pitch does not exist in the corresponding position;

[0027] An input module is configured to input the target rhythm sequence into a melody generation model to obtain a pitch sequence, wherein the pitch sequence includes pitches corresponding to the positions of the placeholders in the target rhythm sequence; wherein the melody generation model is obtained by training first training data using machine learning, wherein the first training data includes: a plurality of first sample rhythm sequences and a first label corresponding to each of the first sample rhythm sequences, each of the first sample rhythm sequences includes a plurality of sample position symbols and a duration corresponding to each of the sample position symbols, each of the sample position symbols includes a sample placeholder and a sample rest, the sample placeholder represents a pitch to be filled in at a corresponding position, the sample rest represents that a sound does not exist at a corresponding position, and the first label represents a pitch sequence;

[0028] The determination module is configured to generate a target music melody based on the target rhythm sequence and the pitch sequence.

[0029] In some embodiments, the first acquisition module is further configured to:

[0030] Acquire an initial rhythm sequence, the initial rhythm sequence comprising a plurality of position symbols and a duration corresponding to each position symbol, each position symbol comprising a placeholder and a rest, the placeholder representing a pitch to be filled in a corresponding position, and the rest representing that a pitch does not exist in the corresponding position;

[0031] The initial rhythm sequence is input into the rhythm generation model to obtain the target rhythm sequence, and the length of the target rhythm sequence is greater than that of the initial rhythm sequence; wherein the rhythm generation model is obtained by training the second training data using machine learning, the second training data includes multiple second sample rhythm sequences and a second label corresponding to each of the second sample rhythm sequences, each of the second sample rhythm sequences includes multiple sample position symbols and a time value corresponding to each of the sample position symbols, each of the sample position symbols includes a sample placeholder and a sample rest, the sample placeholder represents the pitch to be filled in the corresponding position, the sample rest represents the absence of a tone at the corresponding position, and the second label represents a rhythm sequence whose length is greater than that of the second sample rhythm sequence.

[0032] In some embodiments, the input module is further configured to:

[0033] Iteratively executing the steps of determining a vector of a current sequence to be processed based on the target rhythm sequence, and inputting the vector of the current sequence to be processed into the melody generation model to obtain the pitch corresponding to the position of the corresponding placeholder, until the pitches corresponding to the positions of all placeholders in the target rhythm sequence are obtained;

[0034] Obtaining the pitch sequence based on the pitches corresponding to the positions of all placeholders in the target rhythm sequence;

[0035] Among them, when the iterative execution is the first round, the current sequence to be processed is obtained based on the target rhythm sequence and the filling start mark, and the filling start mark is located at the end of the target rhythm sequence; when the iterative execution is not the first round, the current sequence to be processed is obtained based on the target rhythm sequence, the filling start mark and the pitch obtained in the history round, and the pitch obtained in the history round is located after the filling start mark.

[0036] In some embodiments, the input module is further configured to: when the iterative execution is not the first round, determine the current sequence to be processed based on the target rhythm sequence, the fill-in start mark and the pitch obtained from the historical round, and encode the current sequence to be processed to obtain the initial vector of the current sequence to be processed; fuse the initial vector of the current sequence to be processed and the preset category vector to obtain the vector of the current sequence to be processed.

[0037] In some embodiments, the melody generation model includes a Transformer model, and the rhythm generation model includes a Transformer model.

[0038] According to a fifth aspect of an embodiment of the present disclosure, there is provided a training device for a melody generation model, comprising:

[0039] A second acquisition module is configured to acquire first training data, wherein the first training data includes a plurality of first sample rhythm sequences and a first label corresponding to each of the first sample rhythm sequences, each of the first sample rhythm sequences includes a plurality of sample position symbols and a duration corresponding to each of the sample position symbols, each of the sample position symbols includes a sample placeholder and a sample rest, the sample placeholder represents a pitch to be filled in a corresponding position, the sample rest represents that a sound does not exist in the corresponding position, and the first label represents a pitch sequence;

[0040] The first training module is configured to perform model training on the initial melody generation model through the first training data to obtain the melody generation model.

[0041] According to a sixth aspect of an embodiment of the present disclosure, there is provided a training device for a rhythm generation model, comprising:

[0042] A third acquisition module is configured to acquire second training data, wherein the second training data includes a plurality of second sample rhythm sequences and a second label corresponding to each of the second sample rhythm sequences, each of the second sample rhythm sequences includes a plurality of sample position symbols and a duration corresponding to each of the sample position symbols, each of the sample position symbols includes a sample placeholder and a sample rest, the sample placeholder represents a pitch to be filled in a corresponding position, the sample rest represents that a sound does not exist in a corresponding position, and the second label represents a rhythm sequence having a length greater than that of the second sample rhythm sequence;

[0043] The second training module is configured to perform model training on the initial rhythm generation model through the second training data to obtain the rhythm generation model.

[0044] According to a seventh aspect of the embodiments of the present disclosure, there is provided a device for generating a music melody, comprising:

[0045] processor;

[0046] a memory for storing processor-executable instructions;

[0047] Wherein, the processor is configured to:

[0048] Acquire a target rhythm sequence, wherein the target rhythm sequence includes a plurality of position symbols and a duration corresponding to each position symbol, wherein each position symbol includes a placeholder and a rest, wherein the placeholder represents a pitch to be filled in a corresponding position, and the rest represents that a pitch does not exist in the corresponding position;

[0049] The target rhythm sequence is input into a melody generation model to obtain a pitch sequence, wherein the pitch sequence includes pitches corresponding to the positions of the placeholders in the target rhythm sequence; wherein the melody generation model is obtained by training first training data using machine learning, wherein the first training data includes: a plurality of first sample rhythm sequences and a first label corresponding to each of the first sample rhythm sequences, each of the first sample rhythm sequences includes a plurality of sample position symbols and a duration corresponding to each of the sample position symbols, each of the sample position symbols includes a sample placeholder and a sample rest, the sample placeholder represents a pitch to be filled in at a corresponding position, the sample rest represents that a sound does not exist at the corresponding position, and the first label represents the pitch sequence;

[0050] A target music melody is generated based on the target rhythm sequence and the pitch sequence.

[0051] According to an eighth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, on which computer program instructions are stored. When the program instructions are executed by a processor, the steps of the method provided in the first to third aspects of the present disclosure are implemented.

[0052] The technical solution provided by the embodiments of the present disclosure may include the following beneficial effects: characterizing the note length or rest length by time value, that is, characterizing the note length or rest length by one mark, compared with using two marks of the note start time and the note end time to represent the note length, it can avoid generating invalid information in the process of obtaining the pitch sequence; at the same time, the time value in the target rhythm sequence can reflect the rhythm information of the entire piece of music, so that the existing rhythm distribution of the entire piece of music will be taken into account in the process of generating the pitch sequence, thereby making the generated pitch sequence more in line with the overall structure of the music.

[0053] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0055] Figure 1 It is a schematic diagram showing an implementation environment according to an exemplary embodiment.

[0056] Figure 2 The figure is a flow chart of a method for generating a music melody according to an exemplary embodiment.

[0057] Figure 3 is a flowchart of obtaining a target rhythm sequence according to an exemplary embodiment.

[0058] Figure 4 The figure is a flowchart of obtaining a pitch sequence according to an exemplary embodiment.

[0059] Figure 5 The invention is a block diagram of a device for generating a music melody according to an exemplary embodiment.

[0060] Figure 6 It is a block diagram of a training device for a melody generation model according to an exemplary embodiment.

[0061] Figure 7 The invention is a block diagram of a training device for a rhythm generation model according to an exemplary embodiment.

[0062] Figure 8 The invention is a block diagram of a device for generating a music melody according to an exemplary embodiment.

[0063] Fig. 9 The invention is a block diagram of a device for generating a music melody according to an exemplary embodiment. DETAILED DESCRIPTION

[0064] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0065] In some embodiments, the deep learning model used in the related art may include but is not limited to a convolutional neural network (CNN), a recurrent neural network (RNN), a long short-term memory network (LSTM), a Transformer model, etc. In some embodiments, the model can be used to process a sequence represented by markers such as note start, note end, and time lapse information to generate a sequence using the same data representation method, and then the sequence is converted into a generated music melody through post-processing. However, this method has the following characteristics: (1) Due to the use of time lapse information, the rhythm errors existing in the generation process will accumulate. For example, if a certain piece of music has a rhythm error, as time goes by, the rhythm error will also exist in the subsequent music, which will lead to a poor rhythm of the entire generated music. (2) Since two markers, note start and note end, are used to represent a note, this marking method needs to exist in pairs. However, the corresponding markers generated in the generation process may not be paired. Therefore, many markers that cannot be matched one by one will appear in the generation process, and these markers themselves are invalid and will affect the generation of other markers, resulting in the inability to generate music melodies.

[0066] Figure 1 FIG. 1 is a schematic diagram of an implementation environment according to an exemplary embodiment. Figure 1 As shown, the implementation environment may include: a model training device 110 and a model using device 120. In some embodiments, the model training device 110 may be a computer device such as a computer, a server, etc., for training the rhythm generation model and the melody generation model. The model training device 110 may use machine learning to train the rhythm generation model and the melody generation model. The training process of the rhythm generation model and the melody generation model may be referred to as follows Figure 3 and Figure 4 And the related description will not be repeated here.

[0067] The trained rhythm generation model and melody generation model can be deployed in the model using device 120 for use. The model using device 120 can be a terminal device such as a mobile phone, a tablet computer, a personal computer, a multimedia player, or a server. The model using device 120 can generate the target music melody through the melody generation model, or through the rhythm generation model and the melody generation model. For the specific details of generating the target music melody, please refer to the following step S23 and its related description, which will not be repeated here.

[0068] Figure 2 FIG. 1 is a flow chart of a method for generating a music melody according to an exemplary embodiment. Figure 2 As shown, the method includes the following steps.

[0069] In step S21, a target rhythm sequence is obtained, which includes multiple position symbols and the duration corresponding to each position symbol. Each position symbol includes a placeholder and a rest. The placeholder represents the pitch to be filled in the corresponding position, and the rest represents that there is no pitch at the corresponding position.

[0070] In some embodiments, the duration may be used to characterize the relative duration of a note or rest. In some embodiments, the duration may include, but is not limited to, a whole note, a half note, a quarter note, an eighth note, a sixteenth note, and a thirty-second note. In some embodiments, the duration corresponding to a placeholder may be a note duration, which is used to characterize the relative duration of a note, i.e., characterize the duration of a note. The duration corresponding to a rest may be used to characterize the relative duration of a rest, i.e., characterize the duration of the absence of a note at the corresponding position, i.e., the duration of the note pause.

[0071] In some embodiments, the target rhythm sequence may be a sequence of rhythm information of music of a target length, and the target length may be specifically set according to actual conditions, and the embodiments of the present disclosure do not impose any restrictions on this. The note length and the note pause length, that is, the music rhythm, may be reflected by the note duration and the rest duration, and therefore, the target rhythm sequence may reflect the rhythm information of the entire music (that is, the music of the target length).

[0072] In some embodiments, the target rhythm sequence may be obtained by calling a preset target rhythm sequence in a database, obtaining a preset target rhythm sequence selected by a user at a terminal, or obtaining a target rhythm sequence input by a user at a terminal.

[0073] In some embodiments, the target rhythm sequence can also be generated by a pre-trained rhythm generation model. For details on generating the target rhythm sequence, see Figure 3 And the related description will not be repeated here.

[0074] In step S22, the target rhythm sequence is input into the melody generation model to obtain a pitch sequence, which includes pitches corresponding to the positions of the placeholders in the target rhythm sequence.

[0075] In some embodiments, the target rhythm sequence can be processed according to a pre-trained melody generation model to obtain a pitch sequence. In some embodiments, the melody generation model can be obtained by training the first training data using machine learning, the first training data including: multiple first sample rhythm sequences and a first label corresponding to each first sample rhythm sequence, each first sample rhythm sequence including multiple sample position symbols and a time value corresponding to each of the sample position symbols, each sample position symbol including a sample placeholder and a sample rest, the sample placeholder represents the pitch to be filled in the corresponding position, the sample rest represents the absence of the sound in the corresponding position, and the first label represents the pitch sequence. For specific details on obtaining the pitch sequence and training the melody generation model, please refer to Figure 4 And the related description will not be repeated here.

[0076] In step S23, a target music melody is generated based on the target rhythm sequence and the pitch sequence.

[0077] In certain embodiments, the music melody can represent a complete music form, and the music melody can be composed of pitch and rhythm. In certain embodiments, the pitch included in the pitch sequence can be filled in the position of the corresponding placeholder in the target rhythm sequence to generate the target music melody.

[0078] The disclosed embodiment uses duration to characterize the note length or rest length, that is, uses one mark to characterize the note length. Compared with using two marks, namely the note start time and the note end time, to represent the note length, this can avoid generating invalid information in the process of obtaining the pitch sequence. At the same time, the duration in the target rhythm sequence can reflect the rhythm information of the entire piece of music, so that the existing rhythm distribution of the entire piece of music can be taken into account in the process of generating the pitch sequence, thereby making the generated pitch sequence more consistent with the overall structure of the music.

[0079] Figure 3 is a flowchart of obtaining a target rhythm sequence according to an exemplary embodiment. Figure 3 As shown, the following steps are included.

[0080] In step S31, an initial rhythm sequence is obtained. The initial rhythm sequence includes multiple position symbols and the duration corresponding to each position symbol. Each position symbol includes a placeholder and a rest. The placeholder represents the pitch to be filled in the corresponding position, and the rest represents that there is no pitch at the corresponding position.

[0081] In step S32, the initial rhythm sequence is input into a rhythm generation model to obtain the target rhythm sequence, wherein the length of the target rhythm sequence is greater than that of the initial rhythm sequence.

[0082] In some embodiments, the initial rhythm sequence may be a sequence of rhythm information of music reflecting an initial length. The initial length may be specifically set according to actual conditions, the target length is greater than the initial length, and correspondingly, the target rhythm sequence is greater than the length of the initial rhythm sequence.

[0083] In some embodiments, the data representation of the initial rhythm sequence is the same as that of the target rhythm sequence. For details about the initial rhythm sequence, please refer to the description about the target rhythm sequence in step S21 above, which will not be repeated here.

[0084] In some embodiments, the initial rhythm sequence can be processed according to a pre-trained rhythm generation model to obtain a target rhythm sequence. The pre-trained rhythm generation model can continue to write the initial rhythm sequence to obtain the target rhythm sequence.

[0085] In some embodiments, the rhythm generation model can be obtained by training the second training data using machine learning, the second training data including multiple second sample rhythm sequences and second labels corresponding to each second sample rhythm sequence, each second sample rhythm sequence including multiple sample position symbols and time values ​​corresponding to each sample position symbol, each sample position symbol including a sample placeholder and a sample rest, the sample placeholder represents the pitch to be filled in the corresponding position, the sample rest represents the absence of the pitch at the corresponding position, and the second label represents a rhythm sequence whose length is greater than the second sample rhythm sequence.

[0086] In some embodiments, the rhythm generation model can be trained based on the following method: obtaining second training data; training the initial rhythm generation model with the second training data to obtain the rhythm generation model. For the specific details of the second training data, please refer to the above description, which will not be repeated here.

[0087] In some embodiments, the plurality of second sample rhythm sequences included in the second training data may be data input into the initial rhythm generation model for training the rhythm generation model. The data representation of the second sample rhythm sequence is the same as the target rhythm sequence. For the specific details of the second sample rhythm sequence, please refer to the above step S21 and its related description, which will not be repeated here.

[0088] In some embodiments, the second label corresponding to the second sample rhythm sequence can be used to characterize some real information of the second sample rhythm sequence. The second label can characterize a rhythm sequence whose length is greater than the second sample rhythm sequence, that is, the second label can characterize the real rhythm information corresponding to the continued sequence obtained after the second sample rhythm sequence is continued. In some embodiments, multiple second sample rhythm sequences can be obtained according to a preset song database. For example, the melody part of a song in a song database can be extracted, and the melody part can be marked according to the data representation of the initial rhythm sequence to obtain multiple second sample rhythm sequences.

[0089] In some embodiments, the parameters of the initial rhythm generation model can be iteratively updated based on multiple second sample rhythm sequences to reduce the loss function value corresponding to each second sample rhythm sequence, thereby obtaining a trained rhythm generation model. In some embodiments, the loss function value corresponding to each second sample rhythm sequence can be determined by the following process: inputting the second sample rhythm sequence into the rhythm generation model to obtain a predicted rhythm sequence; and determining the loss function value based on the difference between the predicted rhythm sequence and the rhythm sequence corresponding to the second label. The predicted rhythm sequence can be the rhythm sequence obtained after the rhythm generation model continues to write the second sample rhythm sequence.

[0090] In some embodiments, the parameters of the initial rhythm generation model can be continuously updated based on multiple second sample rhythm sequences to reduce the loss function value corresponding to each second sample rhythm sequence so that the loss function value meets a preset condition. For example, the loss function value converges, or the loss function value is less than a preset value. When the loss function value meets the preset condition, the model training is completed and a trained rhythm generation model is obtained.

[0091] In the disclosed embodiment, since the sample rhythm sequence only includes duration and rest information, the influence of the larger dimension of the pitch mark on the rhythm generation is avoided, so that the rhythm generation model can more easily learn the patterns in the data.

[0092] In some embodiments, the vector of the initial rhythm sequence can be input into the rhythm generation model to obtain the target rhythm sequence. The vector of the initial rhythm sequence can be an encoding vector, for example, a single hot encoding vector. In some embodiments, the information representation of the placeholder, duration and rest in the vector of the initial rhythm sequence can be 1-dimensional, 32-dimensional and 1-dimensional. Among them, the 32-dimensional duration can include the duration corresponding to the multiple of the 32-note duration, that is, the duration corresponding to 1 to 32 times of the 32-note duration. The 32-note duration is 1 / 32 of the whole note duration, that is, the duration of 32 times the 32-note duration corresponds to the whole note, and so on, 2 times the 32-note duration corresponds to the sixteenth note, 4 times the 32-note duration corresponds to the eighth note, and 8 times the 32-note duration corresponds to the quarter note. It can be understood that other multiples of the 32-note duration also correspond to other note durations, which will not be repeated here. It is worth noting that the dimension of the time value is not limited to 32 dimensions. For example, it can also be 64 dimensions. Correspondingly, the 64-dimensional time value can include the time value corresponding to the multiple of the 64-note time value.

[0093] For example, taking the vector of the initial rhythm sequence as a one-hot encoding vector (i.e., a vector formed by a value of 0 or 1), if the placeholder at the corresponding position is 1, it represents the pitch to be filled in at that position; if the position of 8 times the value of the 32-note (i.e., a quarter note) in the 32-dimensional time value corresponding to the placeholder is 1, and the positions of other multiples of the 32-note value are 0, then the note value representing the pitch to be filled in is a quarter note; if the rest is 1, it represents that there is no sound at the corresponding position; if the position of 4 times the value of the 32-note (i.e., an eighth note) in the 32-dimensional time value corresponding to the rest is 1, and the positions of other multiples of the 32-note value are 0, then the rest value is represented as an eighth note. The disclosed embodiment represents each time value information by multiples of the note value, so that the note durations are in a multiple relationship, so that the rhythm distribution in the target rhythm sequence generated subsequently is stable.

[0094] In some embodiments, the pre-trained rhythm generation model can also generate a target rhythm sequence in response to a trigger event. In some embodiments, the trigger event can include a user input rhythm beginning or a user's start operation. Generating the target rhythm sequence through the user's start operation can enable the pre-trained rhythm generation model to automatically generate the target rhythm sequence without input.

[0095] The disclosed embodiment generates a target rhythm sequence by inputting an initial rhythm sequence into a rhythm generation model, that is, the initial rhythm sequence is processed by the rhythm generation model to generate a target rhythm sequence. Since the duration is used to represent the note length or rest length in the initial rhythm sequence, that is, the note length or rest length is represented by a mark, compared with using two marks of the note start time and the note end time to represent the note length, it is possible to avoid generating invalid information in the process of generating the target rhythm sequence, and representing the note length by duration can make the note duration have a stable multiple relationship, so that the rhythm distribution in the generated target rhythm sequence is more stable and closer to human composition. At the same time, in the process of generating the target rhythm sequence (i.e., continuing the initial rhythm sequence), only the existence or rest of the note is considered, and there is no need to consider the pitch information, which avoids the influence of the larger dimension of the pitch mark on the rhythm generation.

[0096] Figure 4 is a flowchart of obtaining a pitch sequence according to an exemplary embodiment. Figure 4 As shown, the following steps are included.

[0097] In step S41, iterative execution is performed to determine the vector of the current sequence to be processed based on the target rhythm sequence, and input the vector of the current sequence to be processed into the melody generation model to obtain the pitch corresponding to the position of the corresponding placeholder, until the pitch corresponding to the position of all placeholders in the target rhythm sequence is obtained.

[0098] In some embodiments, pitch may refer to the height of a sound. Different types of music data have different types of pitches. For example, if the music data is a MIDI file, the corresponding pitches may include 128 types. MIDI (Musical Instrument Digital Interface) is an industrial standard electronic communication protocol that defines various notes or playing codes for playing devices such as electronic musical instruments.

[0099] In some embodiments, the vector of the current sequence to be processed can be input into a pre-trained melody generation model to obtain the pitch corresponding to the position of the corresponding placeholder. In some embodiments, when the iterative execution is the first round, the current sequence to be processed is obtained based on the target rhythm sequence and the filling start mark, and the filling start mark is located at the end of the target rhythm sequence.

[0100] In some embodiments, the melody generation model adds a fill-in start mark to the end of the input sequence (i.e., the target rhythm sequence) during the process of generating the pitch, and then generates a pitch sequence after the mark to obtain an output sequence. It is understandable that the output sequence may include the target rhythm sequence, the fill-in start mark, and the pitch sequence. In some embodiments, the fill-in start mark may be used to characterize that the front of the fill-in start mark in the output sequence is the target rhythm sequence, and the back of the fill-in mark is the pitch sequence.

[0101] In some embodiments, when the iterative execution is not the first round, the current sequence to be processed is obtained based on the target rhythm sequence, the start mark is filled in, and the pitch obtained by the historical round is located behind the start mark. For example, the iterative execution is the third round, and the current sequence to be processed is obtained based on the target rhythm sequence, the start mark is filled in, and the pitch obtained by the first two rounds (i.e., the historical round). It can be seen that the melody generation model generates the next round of sequence to be processed according to the existing sequence in each round of iterations of generating pitch, that is, the sequence to be processed of the next round is obtained according to the existing sequence obtained by the target rhythm sequence, the start mark is filled in, and the pitch predicted by the historical round. Therefore, in the generation process of the melody generation model, each type of information (e.g., position symbol, time value, fill in the start mark, and pitch) already present in the existing sequence will be noticed by the model, that is, the melody generation model predicts the next pitch according to the rhythm distribution of the whole music and the existing pitch, so that the predicted pitch is more in line with the overall music structure and its rhythm distribution.

[0102] In some embodiments, when the iterative execution is the first round, the vector of the current sequence to be processed is determined based on the target rhythm sequence, including: determining the current sequence to be processed based on the target rhythm sequence and filling in the start mark, and encoding the current sequence to be processed to obtain the vector of the current sequence to be processed. In some embodiments, when the iterative execution is not the first round, the vector of the current sequence to be processed is determined based on the target rhythm sequence, including: determining the current sequence to be processed based on the target rhythm sequence, filling in the start mark and the pitch obtained in the historical round, and encoding the current sequence to be processed to obtain the initial vector of the current sequence to be processed; fusing the initial vector of the current sequence to be processed and the preset category vector to obtain the vector of the current sequence to be processed.

[0103] In some embodiments, the encoding may be a one-hot encoding, and correspondingly, the vector obtained by the encoding method is a one-hot encoding vector. In some embodiments, the dimension corresponding to the information representation of the placeholder, time value, rest, start mark and pitch in the vector of the current sequence to be processed may be 1 dimension, 32 dimensions, 1 dimension, 1 dimension and 128 dimensions. In some embodiments, 128-dimensional pitches correspond to 128 pitches respectively. It is worth noting that the dimension corresponding to the pitch information is not limited to 128 dimensions. For example, if the pitch includes 200 kinds, the dimension corresponding to the pitch information is 200 dimensions. The representation form of the vector of the current sequence to be processed is similar to that of the target rhythm sequence. For details, please refer to the above-mentioned step S32 and its related description, which will not be repeated here.

[0104] In some embodiments, the initial vector of the current sequence to be processed (for example, a one-hot encoding vector) can be embedded to obtain an embedded vector of the current sequence to be processed, and the embedded vector and the preset category vector are fused to obtain a vector of the current sequence to be processed. Since the dimension of the one-hot encoding is high and the data is sparse, after the embedding process, the discrete sequence can be mapped to a continuous vector, so that the data dimension of the embedded vector is low, thereby improving the processing efficiency of the melody generation model.

[0105] In some embodiments, the preset category vector can be determined based on the information category included in the current sequence to be processed, and the information category can include the placeholders, time values, rests, fill-in start marks, and pitches as described above, and each information category can correspond to a preset category probability value. For example, if the current sequence to be processed includes a target rhythm sequence, a fill-in start mark, and a pitch, the preset category vector is determined by the category probability values ​​corresponding to the placeholders, time values, rests, fill-in start marks, and pitches. The disclosed embodiment fuses the initial vector of the current sequence to be processed and the preset category vector, so that each information belonging to the same category in the current sequence to be processed is closer in a certain direction, and the difference between each information of different categories is greater, so that the melody generation model pays more attention to the information of the same category in the current sequence to be processed during the generation process, and more accurately generates a pitch that matches the structure of the entire music.

[0106] In some embodiments, the embedding vector and the preset category vector belong to the same spatial dimension. For example, the element values ​​of both belong to (0,1). In some embodiments, the embedding vector and the preset category vector of the current sequence to be processed can be fused. For example, the element values ​​corresponding to the two are added. Correspondingly, in some embodiments, the rhythm generation model can process the embedding vector of the target rhythm sequence, and the melody generation model can process the embedding vector of the current sequence to be processed.

[0107] In some embodiments, the melody generation model can be trained based on the following method: obtain first training data; perform model training on the initial melody generation model through the first training data to obtain the melody generation model. In some embodiments, the first training data includes multiple first sample rhythm sequences and a first label corresponding to each first sample rhythm sequence, each first sample rhythm sequence includes multiple sample position symbols and a time value corresponding to each sample position symbol, each sample position symbol includes a sample placeholder and a sample rest, the sample placeholder represents the pitch to be filled in the corresponding position, the sample rest represents the non-existence of the sound in the corresponding position, and the first label represents the pitch sequence. The first sample rhythm sequence is similar to the second sample rhythm sequence, and the specific details can be found in Figure 3 And the related description will not be repeated here.

[0108] In some embodiments, the parameters of the initial melody generation model can be iteratively updated based on multiple first sample rhythm sequences to reduce the loss function value corresponding to each first sample rhythm sequence, thereby obtaining a trained melody generation model. In some embodiments, the loss function value corresponding to each first sample rhythm sequence is determined by the following process: the first sample rhythm sequence is input into the melody generation model to obtain a predicted pitch sequence, the predicted pitch sequence includes pitches corresponding to the positions of sample placeholders in the first sample rhythm sequence; based on the difference between the predicted pitch sequence and the pitch sequence in the first label, the loss function value is determined. The training method of the melody generation model is similar to that of the rhythm generation model. For details, please refer to the above Figure 3 And the related description will not be repeated here.

[0109] In some embodiments, the rhythm generation model may include a Transformer model. In some embodiments, the melody generation model may include a Transformer model. For specific details of the Transformer model, please refer to the relevant technology and will not be repeated here.

[0110] In step S42, a pitch sequence is obtained based on the pitches corresponding to the positions of all placeholders in the target rhythm sequence.

[0111] It can be understood that the pitch sequence can be obtained by arranging the pitches corresponding to the positions of all the placeholders in the target rhythm sequence in order. The length of the pitch sequence is the same as the length of the placeholder sequence in the target rhythm sequence.

[0112] The disclosed embodiment generates rhythm through a rhythm generation model and generates pitch through a melody generation model, that is, the rhythm and pitch are generated step by step, thereby avoiding the influence of the larger dimension of pitch information on rhythm generation.

[0113] Figure 5is a block diagram of a device for generating a music melody according to an exemplary embodiment. Figure 5 The device 500 includes a first acquisition module 510, an input module 520, and a determination module 530.

[0114] The first acquisition module 510 is configured to acquire a target rhythm sequence, wherein the target rhythm sequence includes a plurality of position symbols and a duration corresponding to each position symbol, wherein each position symbol includes a placeholder and a rest, wherein the placeholder represents a pitch to be filled in a corresponding position, and the rest represents that a pitch does not exist in the corresponding position;

[0115] The input module 520 is configured to input the target rhythm sequence into a melody generation model to obtain a pitch sequence, wherein the pitch sequence includes pitches corresponding to the positions of the placeholders in the target rhythm sequence; wherein the melody generation model is obtained by training first training data using machine learning, wherein the first training data includes: a plurality of first sample rhythm sequences and a first label corresponding to each of the first sample rhythm sequences, each of the first sample rhythm sequences includes a plurality of sample position symbols and a duration corresponding to each of the sample position symbols, each of the sample position symbols includes a sample placeholder and a sample rest, the sample placeholder represents a pitch to be filled in at a corresponding position, the sample rest represents that a sound does not exist at the corresponding position, and the first label represents a pitch sequence;

[0116] The determination module 530 is configured to generate a target music melody based on the target rhythm sequence and the pitch sequence.

[0117] In some embodiments, the first acquisition module 510 is further configured to:

[0118] Acquire an initial rhythm sequence, the initial rhythm sequence comprising a plurality of position symbols and a duration corresponding to each position symbol, each position symbol comprising a placeholder and a rest, the placeholder representing a pitch to be filled in a corresponding position, and the rest representing that a pitch does not exist in the corresponding position;

[0119] The initial rhythm sequence is input into the rhythm generation model to obtain the target rhythm sequence, and the length of the target rhythm sequence is greater than that of the initial rhythm sequence; wherein the rhythm generation model is obtained by training the second training data using machine learning, the second training data includes multiple second sample rhythm sequences and a second label corresponding to each of the second sample rhythm sequences, each of the second sample rhythm sequences includes multiple sample position symbols and a time value corresponding to each of the sample position symbols, each of the sample position symbols includes a sample placeholder and a sample rest, the sample placeholder represents the pitch to be filled in the corresponding position, the sample rest represents the absence of a tone at the corresponding position, and the second label represents a rhythm sequence whose length is greater than that of the second sample rhythm sequence.

[0120] In some embodiments, the input module 520 is further configured to:

[0121] Iteratively executing the steps of determining a vector of a current sequence to be processed based on the target rhythm sequence, and inputting the vector of the current sequence to be processed into the melody generation model to obtain the pitch corresponding to the position of the corresponding placeholder, until the pitches corresponding to the positions of all placeholders in the target rhythm sequence are obtained;

[0122] Obtaining the pitch sequence based on the pitches corresponding to the positions of all placeholders in the target rhythm sequence;

[0123] Among them, when the iterative execution is the first round, the current sequence to be processed is obtained based on the target rhythm sequence and the filling start mark, and the filling start mark is located at the end of the target rhythm sequence; when the iterative execution is not the first round, the current sequence to be processed is obtained based on the target rhythm sequence, the filling start mark and the pitch obtained in the history round, and the pitch obtained in the history round is located after the filling start mark.

[0124] In some embodiments, the input module 520 is further configured to: when the iterative execution is not the first round, determine the current sequence to be processed based on the target rhythm sequence, the fill-in start mark and the pitch obtained from the historical round, and encode the current sequence to be processed to obtain the initial vector of the current sequence to be processed; fuse the initial vector of the current sequence to be processed and the preset category vector to obtain the vector of the current sequence to be processed.

[0125] In some embodiments, the melody generation model includes a Transformer model, and the rhythm generation model includes a Transformer model.

[0126] Figure 6FIG. 1 is a block diagram of a training device for a melody generation model according to an exemplary embodiment. Figure 6 , the device 600 includes a second acquisition module 610 and a first training module 620.

[0127] The second acquisition module 610 is configured to: acquire first training data, wherein the first training data includes a plurality of first sample rhythm sequences and a first label corresponding to each of the first sample rhythm sequences, each of the first sample rhythm sequences includes a plurality of sample position symbols and a duration corresponding to each of the sample position symbols, each of the sample position symbols includes a sample placeholder and a sample rest, the sample placeholder represents a pitch to be filled in a corresponding position, the sample rest represents that a sound does not exist in the corresponding position, and the first label represents a pitch sequence;

[0128] The first training module 620 is configured to: perform model training on the initial melody generation model using the first training data to obtain the melody generation model.

[0129] Figure 7 is a block diagram of a training device for a rhythm generation model according to an exemplary embodiment. Figure 7 The device 700 includes a third acquisition module 710 and a second training module 720.

[0130] The third acquisition module 710 is configured to acquire second training data, wherein the second training data includes a plurality of second sample rhythm sequences and a second label corresponding to each of the second sample rhythm sequences, each of the second sample rhythm sequences includes a plurality of sample position symbols and a duration corresponding to each of the sample position symbols, each of the sample position symbols includes a sample placeholder and a sample rest, the sample placeholder represents a pitch to be filled in a corresponding position, the sample rest represents that a sound does not exist in a corresponding position, and the second label represents a rhythm sequence whose length is greater than that of the second sample rhythm sequence;

[0131] The second training module 720 is configured to perform model training on the initial rhythm generation model using the second training data to obtain the rhythm generation model.

[0132] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0133] The present disclosure also provides a computer-readable storage medium having computer program instructions stored thereon, which implement the steps of the data generating method provided by the present disclosure when the program instructions are executed by a processor.

[0134] Figure 88 is a block diagram of a device 800 for generating a music melody according to an exemplary embodiment. For example, the device 800 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0135] Reference Figure 8 , the device 800 may include one or more of the following components: a processing component 802 , a memory 804 , a power component 806 , a multimedia component 808 , an audio component 810 , an input / output (I / O) interface 812 , a sensor component 814 , and a communication component 816 .

[0136] The processing component 802 generally controls the overall operation of the device 800, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the above-mentioned data generation method. In addition, the processing component 802 may include one or more modules to facilitate the interaction between the processing component 802 and other components. For example, the processing component 802 may include a multimedia module to facilitate the interaction between the multimedia component 808 and the processing component 802.

[0137] The memory 804 is configured to store various types of data to support operations on the device 800. Examples of such data include instructions for any application or method operating on the device 800, contact data, phone book data, messages, pictures, videos, etc. The memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0138] The power component 806 provides power to the various components of the device 800. The power component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device 800.

[0139] The multimedia component 808 includes a screen that provides an output interface between the device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. When the device 800 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0140] The audio component 810 is configured to output and / or input audio signals. For example, the audio component 810 includes a microphone (MIC), and when the device 800 is in an operating mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 804 or sent via the communication component 816. In some embodiments, the audio component 810 also includes a speaker for outputting audio signals.

[0141] I / O interface 812 provides an interface between processing component 802 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: home button, volume button, start button, and lock button.

[0142] The sensor assembly 814 includes one or more sensors for providing various aspects of the status assessment of the device 800. For example, the sensor assembly 814 can detect the open / closed state of the device 800, the relative positioning of components, such as the display and keypad of the device 800, and the sensor assembly 814 can also detect the position change of the device 800 or a component of the device 800, the presence or absence of user contact with the device 800, the orientation or acceleration / deceleration of the device 800, and the temperature change of the device 800. The sensor assembly 814 may include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 814 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0143] The communication component 816 is configured to facilitate wired or wireless communication between the device 800 and other devices. The device 800 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0144] In an exemplary embodiment, the apparatus 800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to perform the above-mentioned data generation method.

[0145] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 804 including instructions, and the instructions can be executed by the processor 820 of the device 800 to complete the above data generation method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0146] In another exemplary embodiment, a computer program product is also provided, which includes a computer program that can be executed by a programmable device, and the computer program has a code portion for executing the above-mentioned method for generating music melody, training method for melody generation model, and training method for rhythm generation model when executed by the programmable device.

[0147] Fig. 9 is a block diagram of a device 900 for generating a music melody according to an exemplary embodiment. For example, the device 900 may be provided as a server. Fig. 9 The apparatus 900 includes a processing component 922, which further includes one or more processors, and a memory resource represented by a memory 932 for storing instructions, such as an application, that can be executed by the processing component 922. The application stored in the memory 932 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 922 is configured to execute instructions to perform the above-mentioned data generation method.

[0148] The device 900 may also include a power supply component 926 configured to perform power management of the device 900, a wired or wireless network interface 950 configured to connect the device 900 to a network, and an input / output (I / O) interface 958. The device 900 may operate based on an operating system stored in the memory 932, such as Windows Server 2000. TM , Mac OS X TM , Unix TM , Linux TM , FreeBSD TM or similar.

[0149] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the present disclosure. This application is intended to cover any variations, uses or adaptations of the present disclosure, which follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure. The specification and examples are to be considered as exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0150] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A method for generating a music melody, characterized in that: include: Acquire a target rhythm sequence, wherein the target rhythm sequence includes a plurality of position symbols and a duration corresponding to each position symbol, wherein each position symbol includes a placeholder and a rest, wherein the placeholder represents a pitch to be filled in a corresponding position, and the rest represents that a pitch does not exist in the corresponding position; The target rhythm sequence is input into a melody generation model to obtain a pitch sequence, wherein the pitch sequence includes pitches corresponding to the positions of the placeholders in the target rhythm sequence; wherein the melody generation model is obtained by training first training data using machine learning, wherein the first training data includes: a plurality of first sample rhythm sequences and a first label corresponding to each of the first sample rhythm sequences, each of the first sample rhythm sequences includes a plurality of sample position symbols and a duration corresponding to each of the sample position symbols, each of the sample position symbols includes a sample placeholder and a sample rest, the sample placeholder represents a pitch to be filled in at a corresponding position, the sample rest represents that a sound does not exist at the corresponding position, and the first label represents the pitch sequence; A target music melody is generated based on the target rhythm sequence and the pitch sequence.

2. The method according to claim 1, characterized in that The step of obtaining the target rhythm sequence comprises: Acquire an initial rhythm sequence, the initial rhythm sequence comprising a plurality of position symbols and a duration corresponding to each position symbol, each position symbol comprising a placeholder and a rest, the placeholder representing a pitch to be filled in a corresponding position, and the rest representing that a pitch does not exist in the corresponding position; The initial rhythm sequence is input into the rhythm generation model to obtain the target rhythm sequence, and the length of the target rhythm sequence is greater than that of the initial rhythm sequence; wherein the rhythm generation model is obtained by training the second training data using machine learning, the second training data includes multiple second sample rhythm sequences and a second label corresponding to each of the second sample rhythm sequences, each of the second sample rhythm sequences includes multiple sample position symbols and a time value corresponding to each of the sample position symbols, each of the sample position symbols includes a sample placeholder and a sample rest, the sample placeholder represents the pitch to be filled in the corresponding position, the sample rest represents the absence of a tone at the corresponding position, and the second label represents a rhythm sequence whose length is greater than that of the second sample rhythm sequence.

3. The method according to claim 1, characterized in that The step of inputting the target rhythm sequence into a melody generation model to obtain a pitch sequence comprises: Iteratively executing the steps of determining a vector of a current sequence to be processed based on the target rhythm sequence, and inputting the vector of the current sequence to be processed into the melody generation model to obtain the pitch corresponding to the position of the corresponding placeholder, until the pitches corresponding to the positions of all placeholders in the target rhythm sequence are obtained; Obtaining the pitch sequence based on the pitches corresponding to the positions of all placeholders in the target rhythm sequence; Among them, when the iterative execution is the first round, the current sequence to be processed is obtained based on the target rhythm sequence and the filling start mark, and the filling start mark is located at the end of the target rhythm sequence; when the iterative execution is not the first round, the current sequence to be processed is obtained based on the target rhythm sequence, the filling start mark and the pitch obtained in the history round, and the pitch obtained in the history round is located after the filling start mark.

4. The method according to claim 3, characterized in that In a case where the iterative execution is not the first round, determining a vector of a current sequence to be processed based on the target rhythm sequence includes: Based on the target rhythm sequence, the filling start mark and the pitch obtained by the historical round, the current sequence to be processed is determined, and the current sequence to be processed is encoded to obtain an initial vector of the current sequence to be processed; The initial vector of the current sequence to be processed and a preset category vector are fused to obtain the vector of the current sequence to be processed.

5. The method according to claim 2, characterized in that: The melody generation model includes a Transformer model, and the rhythm generation model includes a Transformer model.

6. A method for training a melody generation model, characterized in that: include: Acquire first training data, wherein the first training data includes a plurality of first sample rhythm sequences and a first label corresponding to each of the first sample rhythm sequences, each of the first sample rhythm sequences includes a plurality of sample position symbols and a duration corresponding to each of the sample position symbols, each of the sample position symbols includes a sample placeholder and a sample rest, the sample placeholder represents a pitch to be filled in a corresponding position, the sample rest represents that a sound does not exist in the corresponding position, and the first label represents a pitch sequence; The melody generation model is obtained by performing model training on the initial melody generation model using the first training data.

7. A training method for a rhythm generation model, characterized in that: include: Acquire second training data, wherein the second training data includes a plurality of second sample rhythm sequences and a second label corresponding to each of the second sample rhythm sequences, each of the second sample rhythm sequences includes a plurality of sample position symbols and a duration corresponding to each of the sample position symbols, each of the sample position symbols includes a sample placeholder and a sample rest, the sample placeholder represents a pitch to be filled in a corresponding position, the sample rest represents that a tone does not exist in a corresponding position, and the second label represents a rhythm sequence having a length greater than that of the second sample rhythm sequence; The initial rhythm generation model is trained using the second training data to obtain the rhythm generation model.

8. A device for generating a music melody, characterized in that: include: A first acquisition module is configured to acquire a target rhythm sequence, wherein the target rhythm sequence includes a plurality of position symbols and a duration corresponding to each position symbol, wherein each position symbol includes a placeholder and a rest, wherein the placeholder represents a pitch to be filled in a corresponding position, and the rest represents that a pitch does not exist in the corresponding position; An input module is configured to input the target rhythm sequence into a melody generation model to obtain a pitch sequence, wherein the pitch sequence includes pitches corresponding to the positions of the placeholders in the target rhythm sequence; wherein the melody generation model is obtained by training first training data using machine learning, wherein the first training data includes: a plurality of first sample rhythm sequences and a first label corresponding to each of the first sample rhythm sequences, each of the first sample rhythm sequences includes a plurality of sample position symbols and a duration corresponding to each of the sample position symbols, each of the sample position symbols includes a sample placeholder and a sample rest, the sample placeholder represents a pitch to be filled in at a corresponding position, the sample rest represents that a sound does not exist at a corresponding position, and the first label represents a pitch sequence; The determination module is configured to generate a target music melody based on the target rhythm sequence and the pitch sequence.

9. A device for generating a music melody, characterized in that: include: processor; a memory for storing processor-executable instructions; Wherein, the processor is configured to: Acquire a target rhythm sequence, wherein the target rhythm sequence includes a plurality of position symbols and a duration corresponding to each position symbol, wherein each position symbol includes a placeholder and a rest, wherein the placeholder represents a pitch to be filled in a corresponding position, and the rest represents that a pitch does not exist in the corresponding position; The target rhythm sequence is input into a melody generation model to obtain a pitch sequence, wherein the pitch sequence includes pitches corresponding to the positions of the placeholders in the target rhythm sequence; wherein the melody generation model is obtained by training first training data using machine learning, wherein the first training data includes: a plurality of first sample rhythm sequences and a first label corresponding to each of the first sample rhythm sequences, each of the first sample rhythm sequences includes a plurality of sample position symbols and a duration corresponding to each of the sample position symbols, each of the sample position symbols includes a sample placeholder and a sample rest, the sample placeholder represents a pitch to be filled in at a corresponding position, the sample rest represents that a sound does not exist at the corresponding position, and the first label represents the pitch sequence; A target music melody is generated based on the target rhythm sequence and the pitch sequence.

10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the program instructions are executed by a processor, the steps of the method described in any one of claims 1 to 7 are implemented.

Images