Music generation method and device, electronic equipment and storage medium

Abstract

The invention discloses a music generation method and device, electronic equipment and a storage medium. The method comprises the steps of obtaining reference music and target lyrics in response to triggering of a music generation event; generating at least one reference coding vector based on the reference music; wherein the reference coding vector is a coding vector of at least one layer; performing word segmentation processing on the target lyrics, and converting each word segmentation unit in the target lyrics into a corresponding word segmentation unit vector; and inputting the at least one reference coding vector and the word segmentation unit vector into a pre-constructed music generation large model, and generating target music according to an output result of the music generation large model. According to the scheme, the large model is generated based on the music, the target music of any length, any content and any type can be generated at a time according to the obtained reference music and the target lyrics, the generation is free, the flexibility is high, and the diversity of the generated target music is greatly improved.

Description

Technical Field

[0001] The present invention relates to the field of audio technology, and in particular to a music generating method, device, electronic device and storage medium. Background Art

[0002] The existing song generation technology generally processes the vocals and accompaniment separately, wherein the accompaniment is generally extracted from the accompaniment in the reference song or template song, and the vocals are generally derived from voice input or text input and refer to the sound spectrum of the original singer to generate the corresponding spectrum, and then generate the vocal a cappella. Finally, the accompaniment or the vocal a cappella can be output separately, or the two tracks of accompaniment and vocal a cappella can be merged into one output song. That is, the existing music generation technology requires two systems to process the accompaniment and the vocals respectively, and generally can only achieve vocal optimization by adjusting the vocal spectrum, or generate the vocal spectrum through text input to achieve simple functions such as filling in lyrics, etc. The output accompaniment is part or all of the reference music, and the vocals refer to the vocal rhythm in the reference music. The system is relatively complex, and it is impossible to create music out of nothing and generate it freely. The flexibility is low, and the generated music lacks diversity. Summary of the invention

[0003] The present invention provides a music generation method, device, electronic device and storage medium, which can generate target music of any length, any content and any type at one time based on the acquired reference music and target lyrics. It can create music out of nothing, generate it freely and with high flexibility, greatly improving the diversity of the generated target music.

[0004] According to one aspect of the present invention, there is provided a music generation method, comprising:

[0005] In response to a music generation event being triggered, obtaining reference music and target lyrics;

[0006] Generate at least one reference encoding vector based on the reference music; wherein the reference encoding vector is an encoding vector of at least one layer;

[0007] Performing word segmentation processing on the target lyrics, and converting each word segmentation unit in the target lyrics into a corresponding word segmentation unit vector; wherein the word segmentation unit vector is an encoding vector of at least one layer;

[0008] The at least one reference coding vector and the word segmentation unit vector are input into a pre-constructed music generation model, and the target music is generated according to the output result of the music generation model.

[0009] According to another aspect of the present invention, there is provided a music generating device, comprising:

[0010] A reference music acquisition module, for acquiring reference music and target lyrics in response to a music generation event being triggered;

[0011] A reference coding vector generation module, used to generate at least one reference coding vector based on the reference music; wherein the reference coding vector is a coding vector of at least one layer;

[0012] A word segmentation unit vector determination module is used to perform word segmentation processing on the target lyrics and convert each word segmentation unit in the target lyrics into a corresponding word segmentation unit vector; wherein the word segmentation unit vector is an encoding vector of at least one layer;

[0013] The target music generation module is used to input the at least one reference coding vector and the word segmentation unit vector into a pre-constructed music generation model, and generate target music according to the output result of the music generation model.

[0014] According to another aspect of the present invention, there is provided an electronic device, the electronic device comprising:

[0015] at least one processor; and

[0016] a memory communicatively connected to the at least one processor; wherein,

[0017] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the music generation method described in any embodiment of the present invention.

[0018] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to implement the music generation method described in any embodiment of the present invention when executed by a processor.

[0019] The music generation scheme of the embodiment of the present invention, in response to the triggering of a music generation event, obtains reference music and target lyrics; generates at least one reference encoding vector based on the reference music; wherein the reference encoding vector is an encoding vector of at least one layer; performs word segmentation processing on the target lyrics, and converts each word segmentation unit in the target lyrics into a corresponding word segmentation unit vector; wherein the word segmentation unit vector is an encoding vector of at least one layer; inputs the at least one reference encoding vector and the word segmentation unit vector into a pre-constructed music generation model, and generates the target music according to the output result of the music generation model. Through the technical scheme provided by the embodiment of the present invention, based on the music generation model, it is possible to generate target music of any length, any content, and any type at one time according to the obtained reference music and target lyrics, creating something out of nothing, generating freely, and with high flexibility, which greatly improves the diversity of the generated target music.

[0020] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0022] Figure 1 A flowchart of a music generation method provided in Embodiment 1 of the present invention;

[0023] Figure 2 A flowchart of a music generation method provided in Embodiment 2 of the present invention;

[0024] Figure 3 A flowchart of a music generation method provided in Embodiment 3 of the present invention;

[0025] Figure 4 A flowchart of a music generation method provided in Embodiment 4 of the present invention;

[0026] Figure 5 A flowchart of a music generation method provided in Embodiment 5 of the present invention;

[0027] Figure 6 A structural schematic diagram of a music generating device provided in Embodiment 6 of the present invention;

[0028] Figure 7 It is a schematic diagram of the structure of an electronic device for implementing the music generating method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0029] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0030] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0031] Embodiment 1

[0032] Figure 1 This is a flowchart of a music generation method provided in Embodiment 1 of the present invention. This embodiment is applicable to the case of generating music. The method can be executed by a music generation device. The music generation device can be implemented in the form of hardware and / or software. The music generation device can be configured in an electronic device. Figure 1 As shown, the method includes:

[0033] S110: In response to a music generation event being triggered, obtaining reference music and target lyrics.

[0034] In an embodiment of the present invention, when a music generation request input by a user is received, it is determined that a music generation event is triggered. In response to the music generation event being triggered, reference music and target lyrics are obtained. Exemplarily, a certain music input by a user can be used as reference music, a certain music selected from a music database can be used as reference music, or a certain music selected by default can be used as reference music. Among them, the reference music can be a song containing vocals and accompaniment, or it can be light music containing only accompaniment, or it can be a cappella music containing only vocals. The length of the reference music can be a preset length, or it can be greater than a preset length (such as 5 seconds or 10 seconds), and the reference music contains a certain amount of style information, for example, the reference music is a piece of music that can reflect the type of music or all playing instruments or all singing styles. Optionally, the target lyrics can be a piece of lyrics input by the user in text, or a piece of lyrics input by the user in voice, or lyrics composed of subtitle information or voice information in the input video. The target lyrics can be any lyrics content, and the target lyrics can be content of any length. It should be noted that the embodiment of the present invention does not limit the content and length of the target lyrics.

[0035] S120. Generate at least one reference coding vector based on the reference music; wherein the reference coding vector is a coding vector of at least one layer.

[0036] In an embodiment of the present invention, at least one reference coding vector of at least one layer is generated based on the reference music. Exemplarily, the reference music as a whole can be encoded into at least one global vector of at least one layer by an encoder, and the global vector is used as a reference coding vector. It is also possible to intercept music of a preset length (such as 5 seconds or 10 seconds, etc.) from the reference music, and encode the intercepted music of the preset length into at least one reference coding vector of at least one layer by an encoder. Another exemplary method is to split the reference music into music parts of at least one dimension, and encode each music part of each dimension into at least one reference coding vector of at least one layer by an encoder. For example, the reference music is split into an accompaniment part and a vocal part, and the accompaniment part is encoded by a MERT encoder to generate a corresponding reference coding vector, and the vocal part is encoded by a Mel encoder to generate a corresponding reference coding vector.

[0037] S130, performing word segmentation processing on the target lyrics, and converting each word segmentation unit in the target lyrics into a corresponding word segmentation unit vector; wherein the word segmentation unit vector is an encoding vector of at least one layer.

[0038] In an embodiment of the present invention, the target lyrics are segmented based on a preset segmentation algorithm, and the target lyrics are divided into a plurality of segmentation units, wherein the segmentation unit can be each character, each word, or each sentence. Each segmentation unit in the target lyrics is encoded based on an encoder, and a corresponding segmentation unit vector of at least one layer is generated. The more layers of the segmentation unit vector, the richer the information in the target lyrics, such as emotional information, can be represented.

[0039] S140, inputting the at least one reference coding vector and the word segmentation unit vector into a pre-constructed music generation model, and generating target music according to the output result of the music generation model.

[0040] In an embodiment of the present invention, at least one reference coding vector and a word segmentation unit vector corresponding to each word segmentation unit are input into a pre-constructed music generation model, so that the music generation model outputs a feature vector corresponding to the music according to the input information, and converts the feature vector into target music based on the music generation module. The music generation model is a pre-trained machine learning model, for example, the music generation model is any open source model, such as GPT.

[0041] The music generation method of the embodiment of the present invention, in response to a music generation event being triggered, obtains reference music and target lyrics; generates at least one reference encoding vector based on the reference music; wherein the reference encoding vector is an encoding vector of at least one layer; performs word segmentation processing on the target lyrics, and converts each word segmentation unit in the target lyrics into a corresponding word segmentation unit vector; wherein the word segmentation unit vector is an encoding vector of at least one layer; inputs the at least one reference encoding vector and the word segmentation unit vector into a pre-constructed music generation model, and generates the target music according to the output result of the music generation model. Through the technical solution provided by the embodiment of the present invention, based on the music generation model, it is possible to generate target music of any length, any content, and any type at one time according to the obtained reference music and target lyrics, creating something out of nothing, generating freely, and with high flexibility, which greatly improves the diversity of the generated target music.

[0042] In some embodiments, the acquisition of reference music includes: acquiring at least 2 reference music containing at least one different element, feature, style, and dimension; correspondingly, generating at least one reference encoding vector based on the reference music includes: generating at least 2 reference encoding vectors based on at least 2 reference music containing at least one different element, feature, style, and dimension; inputting the at least one reference encoding vector and the word segmentation unit vector into a pre-constructed music generation model, and generating target music according to the output result of the music generation model, including: inputting the at least 2 reference encoding vectors and the word segmentation unit vector into a pre-constructed music generation model, and generating target music that integrates at least one different element, feature, style, and dimension of at least 2 reference music according to the output result of the music generation model. The advantage of such a setting is that target music containing at least one different element, feature, style, and dimension of multiple reference music can be generated based on the music generation model, so that the generated target music has diversity.

[0043] In an embodiment of the present invention, in response to a music generation event being triggered, at least two reference musics are obtained, wherein the two reference musics are musics containing at least one different element, feature, style, and dimension. For each reference music, at least one corresponding reference coding vector of at least one layer is generated based on the reference music. It can be understood that at least two reference coding vectors can be generated based on at least two reference musics containing at least one different element, feature, style, and dimension, and each reference coding vector is a coding vector of at least one layer. The at least two reference coding vectors and the word segmentation unit vectors corresponding to the target lyrics are input into the music generation model, and the target music is generated according to the output result of the music generation model, wherein the target music is music that integrates at least one different element, feature, style, and dimension of at least two reference musics. Exemplarily, two reference musics are obtained, wherein one reference music is piano accompaniment music and the other reference music is violin accompaniment music, and then the music generation model can generate a target music mixed with two reference music elements (piano accompaniment and violin accompaniment). In another exemplary embodiment, one reference music is a male solo version of the song, and the other reference music is a female solo version of the song. Then, the music generation model can generate a target music that is a mixture of male and female characteristics of mixed singing and chorus. In another exemplary embodiment, one reference music is a rock style song, and the other reference music is a pop style song. Then, the music generation model can generate a target music that is a mixture of two styles (rock style and pop style). In another exemplary embodiment, one reference music is accompaniment music, and the other reference music is vocal music. Then, the music generation model can generate a target music that is a mixture of two reference music dimensions (accompaniment music and vocal music). It can be understood that the music generation model can generate target music (song) based on the user's specified reference to a certain element, feature, style, or dimension of one of the reference music (e.g., accompaniment), and a certain element, feature, style, or dimension of another reference music (e.g., vocals).

[0044] Embodiment 2

[0045] Figure 2 A flowchart of a music generation method provided in Embodiment 2 of the present invention is shown in FIG. Figure 2 As shown, the method includes:

[0046] S210: When a general music generation request input by a user is received, it is determined that a music generation event is triggered, and reference music and target lyrics of a preset duration are obtained.

[0047] In an embodiment of the present invention, when a user wants to generate a certain type of music including an accompaniment part and / or a vocal part, a general music generation request is input. Among them, the general music generation request may include any generation request of an accompaniment music generation request, a cappella music generation request, and a song music generation request. When the general music generation request input by the user is received, it is determined that a music generation event is triggered. In response to the music generation event being triggered, reference music and target lyrics of a preset duration (such as 5 seconds or 10 seconds) matching the general music generation request are obtained. Exemplarily, when the general music generation request is a cappella music generation request, the reference music may be a cappella music containing only vocals; when the general music generation request is an accompaniment music generation request, the reference music may be accompaniment music containing only accompaniment; when the general music generation request is a song music generation request, the reference music may be song music containing vocals and accompaniment.

[0048] S220. Encode the reference music through an encoder to generate at least one reference coding vector; wherein the reference coding vector is a coding vector of at least one layer.

[0049] In an embodiment of the present invention, reference music of a preset duration is taken as a whole and encoded by an encoder to generate at least one reference coding vector of at least one layer, wherein the encoder may be an open source encoder, such as a MERT encoder or a Mel encoder.

[0050] S230, performing word segmentation processing on the target lyrics, and converting each word segmentation unit in the target lyrics into a corresponding word segmentation unit vector; wherein the word segmentation unit vector is an encoding vector of at least one layer.

[0051] S240. Input the at least one reference coding vector and the word segmentation unit vector into a pre-constructed music generation model, and generate target music according to the output result of the music generation model.

[0052] In an embodiment of the present invention, when the general music generation request is a song music generation request, the music generation model is a song generation model, that is, at least one reference coding vector generated by the reference music and the word segmentation unit vector corresponding to the target lyrics are input into the song generation model, and the target song music containing the target lyrics is generated according to the song generation model, wherein the target song music has the same or similar accompaniment style and vocal singing style as the reference music. When the general music generation request is a cappella music generation request, the music generation model is a cappella generation model, that is, at least one reference coding vector generated by the reference music and the word segmentation unit vector corresponding to the target lyrics are input into the cappella generation model, and the target cappella music containing the target lyrics is generated according to the cappella generation model, wherein the target cappella music has the same or similar vocal singing style as the reference music. When the general music generation request is an accompaniment music generation request, the music generation big model is an accompaniment generation big model, that is, at least one reference coding vector generated by the reference music and the word segmentation unit vector corresponding to the target lyrics are input into the accompaniment generation big model, and the target accompaniment music is generated according to the accompaniment generation big model, wherein the accompaniment style of the target accompaniment music is the same or similar to that of the reference music.

[0053] Through the technical solution provided by the embodiment of the present invention, based on the music generation large model, it is possible to generate target music with the same style as the reference music according to the acquired reference music and target lyrics, which is free and highly flexible.

[0054] Embodiment 3

[0055] Figure 3 A flowchart of a music generation method provided in Embodiment 3 of the present invention is shown in FIG. Figure 3 As shown, the method includes:

[0056] S310: When a general music generation request input by a user is received, it is determined that a music generation event is triggered, and reference music and target lyrics are obtained.

[0057] In an embodiment of the present invention, when a user wants to generate song music including an accompaniment part and a vocal part, a general music generation request is input. When the general music generation request input by the user is received, it is determined that a music generation event is triggered. In response to the music generation event being triggered, reference music and target lyrics are obtained. The reference music can be a song including vocals and accompaniment, or accompaniment music including only accompaniment, or a cappella music including only vocals.

[0058] S320: Discretize the reference music to generate at least two frames of first reference audio data.

[0059] In an embodiment of the present invention, the reference music is discretized to generate at least two frames of first reference audio data. Exemplarily, the reference music can be discretized into 10 frames of reference audio data per second. For example, if the reference music is a 10-second piece of music, the reference music can be discretized into 100 frames of reference audio data. Another exemplary embodiment, the reference music can also be discretized as a whole, for example, the reference music is discretized into 500 frames of reference audio data, wherein the number of frames of discrete reference audio data can be determined according to the length of the reference music, and the larger the length of the reference music, the more frames of discrete reference audio data. In an embodiment of the present invention, for the convenience of description, the reference audio data generated by the discretization of the reference music is referred to as the first reference audio data.

[0060] S330. Encode each frame of the first reference audio data through an encoder to generate at least one reference coding vector; wherein the reference coding vector is a coding vector of at least one layer.

[0061] In an embodiment of the present invention, each frame of the first reference audio data is encoded by an encoder to generate a corresponding reference encoding vector of at least one layer, wherein the number of reference encoding vectors is the same as the number of frames of the first reference audio data. Optionally, the same encoder may be used to encode each frame of the first reference audio data, or different encoders may be used to encode each frame of the first reference audio data. The encoder may be an open source encoder, such as a MERT encoder or a Mel encoder.

[0062] S340, performing word segmentation processing on the target lyrics, and converting each word segmentation unit in the target lyrics into a corresponding word segmentation unit vector; wherein the word segmentation unit vector is an encoding vector of at least one layer.

[0063] S350, inputting the at least one reference coding vector and the word segmentation unit vector into a pre-constructed music generation model, and generating target music according to the output result of the music generation model.

[0064] Through the technical solution provided by the embodiment of the present invention, based on the music generation large model, it is possible to generate target music with the same style as the reference music according to the acquired reference music and target lyrics, which is free and highly flexible.

[0065] Embodiment 4

[0066] Figure 4 A flowchart of a music generation method provided in Embodiment 4 of the present invention is shown in FIG. Figure 4 As shown, the method includes:

[0067] S410: When a personalized music generation request input by a user is received, it is determined that a music generation event is triggered, and reference music and target lyrics are obtained.

[0068] Exemplarily, the personalized music generation request may include any one or more of a cappella music generation request or an accompaniment music generation request or a song music generation request. Optionally, the personalized music generation request may also be a more detailed and personalized music generation request specified by the user, for example, a cappella music generation request for male (bass, baritone, tenor), a cappella music generation request for female (alt, mezzo-soprano, soprano), a cappella music generation request for one or more roles, a cappella music generation request in various languages ​​(such as Chinese, dialects, English, etc.), an accompaniment music generation request for one or more musical instruments (such as piano), a solo solo music generation request, and a chorus multi-instrument ensemble music generation request. When the personalized music generation request input by the user is received, it is determined that the music generation event is triggered, and reference music and target lyrics are obtained.

[0069] S420: Analyze the reference music, and extract from the reference music a reference music portion of at least one dimension that matches the personalized music generation request.

[0070] Exemplarily, the reference music is analyzed, and the reference music parts of two dimensions, the accompaniment part and the vocal part, are extracted from the reference music. Among them, a variety of methods can be used to extract the audio data of the vocal part and the audio data of the accompaniment part from the audio of the reference music. For example, the reference music can be Fourier transformed to obtain the mixed amplitude spectrum and mixed phase spectrum of the accompaniment signal and the vocal signal, and then the amplitude spectrum of the vocal and the accompaniment sound is separated using a trained separation model (which can be a model built based on a deep neural network). The amplitude spectrum and the mixed phase spectrum of the separated vocal and accompaniment sound are subjected to inverse Fourier transform to obtain the audio data of the vocal part and the audio data of the accompaniment part. Exemplarily, the audio data of the vocal part and the audio data of the accompaniment part in the reference music can be extracted by audio filtering. Exemplarily again, the audio of the reference music can be copied to two tracks, and the high-frequency vocal signal and the low-frequency accompaniment signal are filtered out respectively to achieve the separation of the vocal signal and the accompaniment signal.

[0071] In an embodiment of the present invention, if the personalized music generation request is a male a cappella music generation request, the reference music is analyzed and the male voice part is extracted from the reference music; if the personalized music generation request is a request for accompaniment music of multiple instruments, the reference music is analyzed and the accompaniment part of each instrument is extracted from the reference music. It should be noted that the embodiment of the present invention does not limit the number of dimensions of the reference music part that matches the personalized music generation request.

[0072] S430. For the reference music part of each dimension, discretize the reference music part to generate at least two frames of second reference audio data.

[0073] In an embodiment of the present invention, the reference music part of each dimension is discretized to generate at least two frames of second reference audio data corresponding to the reference music part of each dimension. Exemplarily, the reference music part of each dimension can be discretized into 20 frames of reference audio data per second. For example, if the reference music part of a certain dimension is a 4-second music, the reference music part of the dimension can be discretized into 80 frames of reference audio data. Another exemplary embodiment, the reference music part of each dimension can also be discretized as a whole. For example, the reference music part of a certain dimension is discretized into 100 frames of reference audio data, wherein the number of frames of discrete reference audio data can be determined according to the length of the reference music part of each dimension. The larger the length of the reference music part, the more frames of discrete reference audio data. In an embodiment of the present invention, for the convenience of description, the reference audio data generated by the discretization of the reference music part of each dimension is referred to as the second reference audio data. It should be noted that the number of frames of the second reference audio data generated by the discretization of the reference music part of each dimension can be the same or different.

[0074] S440: Encode each frame of the second reference audio data through an encoder to generate at least one reference encoding vector.

[0075] In an embodiment of the present invention, each frame of the second reference audio data is encoded by an encoder to generate a corresponding reference coding vector of at least one layer, wherein the number of reference coding vectors is the same as the number of frames of the second reference audio data. Optionally, the same encoder may be used to encode each frame of the second reference audio data corresponding to the reference music parts of each dimension, or different encoders may be used to encode each frame of the second reference audio data corresponding to the reference music parts of different dimensions. The encoder may be an open source encoder, such as a MERT encoder or a Mel encoder. Exemplarily, after extracting the accompaniment part and the vocal part from the reference music, and discretizing the accompaniment part and the vocal part respectively, the MERT encoder may be used to encode the second reference audio data discretely generated by the accompaniment part to generate the corresponding reference coding vector, and the Mel encoder may be used to encode the second reference audio data discretely generated by the vocal part to generate the corresponding reference coding vector.

[0076] S450, performing word segmentation processing on the target lyrics, and converting each word segmentation unit in the target lyrics into a corresponding word segmentation unit vector; wherein the word segmentation unit vector is a coding vector of at least one layer.

[0077] S460, inputting the at least one reference coding vector and the word segmentation unit vector into a pre-constructed music generation model, and generating target music according to the output result of the music generation model.

[0078] In an embodiment of the invention, if the music generated by the user's request is a song, the reference coding vectors of all dimensions of the reference music and the word segmentation unit vectors of the target lyrics are input into the music generation model, and the music generation model outputs the coding vector of the corresponding song according to the input information, and converts the coding vector into the corresponding target music (song). If the music generated by the user's request is an accompaniment, the reference coding vector of the accompaniment part of the reference music is retained, and the reference coding vectors of other dimensions (such as the coding vector corresponding to the vocal part) are set as special coding vectors (such as 0 or 1), and the reference coding vector of the accompaniment part of the reference music, the special coding vectors of other dimensions (such as the coding vector corresponding to the vocal part) and the word segmentation unit vector corresponding to the target lyrics are input into the music generation model, and the music generation model outputs the coding vector of the corresponding accompaniment according to the input information, and converts the coding vector into the corresponding target music (accompaniment).

[0079] Through the technical solution provided by the embodiment of the present invention, based on the music generation large model, it is possible to generate target music of any length, any content, and any type at one time according to the acquired reference music and target lyrics. It is possible to create music out of nothing, generate it freely, and have high flexibility, which greatly improves the diversity of the generated target music and meets the user's demand for personalized music generation.

[0080] Embodiment 5

[0081] Figure 5 A flowchart of a music generation method provided in Embodiment 5 of the present invention is shown in FIG. Figure 5 As shown, the method includes:

[0082] S510: When a personalized music generation request input by a user is received, it is determined that a music generation event is triggered, and reference music and target lyrics of a preset duration are obtained.

[0083] In the embodiment of the present invention, when a personalized music generation request input by a user is received, reference music and target lyrics of a preset duration (such as 5 seconds or 10 seconds) are obtained.

[0084] S520: Analyze the reference music, and extract from the reference music a reference music portion of at least one dimension that matches the personalized music generation request.

[0085] S530. For the reference music part of each dimension, encode the reference music part through an encoder to generate at least one reference encoding vector.

[0086] In an embodiment of the present invention, the reference music part of each dimension is taken as a whole, and the reference music part of each dimension is encoded by an encoder to generate a corresponding reference coding vector of at least one layer. Optionally, the reference music parts of each dimension can be encoded by the same encoder, or different encoders can be used to encode the reference music parts of each dimension. Among them, the encoder can be an open source encoder, such as a MERT encoder or a Mel encoder. Exemplarily, the accompaniment part and the vocal part are extracted from the reference music, and the accompaniment part can be encoded by the MERT encoder to generate a corresponding reference coding vector, and the vocals can be encoded by the Mel encoder to generate a corresponding reference coding vector.

[0087] S540, performing word segmentation processing on the target lyrics, and converting each word segmentation unit in the target lyrics into a corresponding word segmentation unit vector; wherein the word segmentation unit vector is an encoding vector of at least one layer.

[0088] S550, input the at least one reference coding vector and the word segmentation unit vector into a pre-constructed music generation model, and generate target music according to the output result of the music generation model.

[0089] Through the technical solution provided by the embodiment of the present invention, based on the music generation large model, it is possible to generate target music of any length, any content, and any type at one time according to the acquired reference music and target lyrics. It is possible to create music out of nothing, generate it freely, and have high flexibility, which greatly improves the diversity of the generated target music and meets the user's demand for personalized music generation.

[0090] In some embodiments, before inputting the at least one reference coding vector and the word segmentation unit vector into a pre-constructed music generation large model, it also includes: obtaining a music training sample set; wherein the music training sample set contains at least two groups of music training samples, each group of music training samples includes fitting music, at least one sample music and at least one sample lyrics; respectively encoding the fitting music, the sample music and the sample lyrics in each group of the music training samples to generate at least one corresponding sample coding vector; wherein the sample coding vector is a coding vector of at least one layer; training the preset large model based on the sample coding vector to generate the music generation large model.

[0091] In an embodiment of the present invention, a music training sample set is obtained, wherein the music training sample set includes at least two groups of music training samples, and each group of music training samples includes fitting music, at least one sample music, and at least one sample lyrics. Exemplarily, a certain mature music can be used as the fitting music in a certain group of music training samples, at least one section of music can be intercepted from the mature music as the sample music in the group of music training samples, and any music lyrics in the mature music can be used as the sample lyrics in the group of music training samples. Based on the fitting music, sample music, and sample lyrics in each group of music training samples, at least one corresponding sample coding vector is generated, that is, at least one corresponding sample coding vector is generated based on the fitting music in each group of music training samples, at least one corresponding sample coding vector is generated based on the sample music in each group of music training samples, and at least one corresponding sample coding vector is generated based on the sample lyrics in each group of music training samples. Wherein, the sample coding vector is a coding vector of at least one layer.

[0092] Exemplarily, the fitting music and the sample music can be respectively encoded into at least one corresponding sample encoding vector by an encoder. Optionally, encoding the fitting music in each group of the music training samples to generate at least one corresponding sample encoding vector includes: discretizing the fitting music in each group of the music training samples to generate at least two frames of first fitting audio data; encoding each frame of the first fitting audio data by an encoder to generate at least one fitting encoding vector. For example, discretizing the fitting music to generate at least two frames of first fitting audio data, and encoding each frame of first fitting audio data into a corresponding sample encoding vector by an encoder. The sample music is uniformly encoded as a whole to generate a corresponding sample encoding vector. Or similarly, the sample music is discretized to generate at least two frames of first sample audio data, and each frame of first sample audio data is encoded into a corresponding sample encoding vector by an encoder. Exemplarily, the sample lyrics are segmented based on a preset segmentation algorithm, and each segmentation unit in the sample lyrics is encoded based on the encoder to generate at least one layer of corresponding sample encoding vectors.

[0093] Optionally, respectively encoding the fitting music, the sample music and the sample lyrics in each group of the music training samples to generate at least one corresponding sample encoding vector includes:

[0094] For each group of the music training samples, a fitting music part of at least one dimension is extracted from the fitting music in the music training samples, and a sample music part of at least one dimension is extracted from the sample music in the music training samples; the fitting music part of each dimension, the sample music part of each dimension and the sample lyrics in the music training samples are respectively encoded to generate at least one corresponding sample encoding vector.

[0095] Optionally, the fitted music part of each dimension is encoded to generate at least one corresponding sample encoding vector, including: for the fitted music part of each dimension, discretizing the fitted music part to generate at least two frames of second fitted audio data; encoding each frame of the second fitted audio data through an encoder to generate at least one corresponding reference encoding vector.

[0096] Exemplarily, for each group of music training samples, the fitting music in the music training samples is split into a preset number of dimensions, such as splitting a fitting music part of at least one dimension from the fitting music (such as a fitting music part of two dimensions of the accompaniment part and the vocal part), discretizing the fitting music part of each dimension, generating at least two frames of second fitting audio data, encoding each frame of the second fitting audio data through an encoder, and generating a corresponding sample encoding vector. Similarly, for each group of music training samples, the sample music in the music training samples is split into a preset number of dimensions, such as splitting a sample music part of at least one dimension from the sample music (such as a sample music part of two dimensions of the accompaniment part and the vocal part), uniformly encoding the sample music part of at least one dimension split as a whole, and generating at least one corresponding sample encoding vector. Or the sample music part of at least one dimension split is discretized to generate at least two frames of second sample audio data, and each frame of the second sample audio data is encoded through an encoder to generate a corresponding sample encoding vector. For each group of music training samples, the sample lyrics in the music training samples are encoded to generate at least one corresponding sample encoding vector.

[0097] In an embodiment of the present invention, a preset large model is trained based on a sample coding vector to generate a music generation large model, wherein the preset large model may be any open source large model, such as GPT, etc. Optionally, the preset large model is trained based on the sample coding vector to generate a music generation large model, including: inputting at least one sample coding vector corresponding to the fitted music part of all or part of the dimensions, at least one sample coding vector corresponding to the sample music part of all or part of the dimensions, and at least one sample coding vector corresponding to the sample lyrics into the preset large model, training the preset large model, and generating a music generation large model.

[0098] Exemplarily, the encoding vectors corresponding to the fitting music part of all dimensions generated by splitting the fitting music, the encoding vectors corresponding to the sample music part of all dimensions generated by splitting the sample music, and the encoding vectors corresponding to the sample lyrics are respectively input into the preset large model, so that each frame token output by the preset large model is fitted with at least one layer of encoding vectors of each frame of the fitting music, the loss function is calculated, and the optimization is continuously performed to complete the model training. For example, when the encoding vectors of all dimensions corresponding to the sample music are input into the preset large model, the complete encoding of the corresponding mature music (i.e., fitting music) is output, such as when the encoding vectors of the entire unsplit sample music are input into the preset large model, or when the encoding vectors of the accompaniment part and the encoding vectors of the vocal part of the sample music are input into the preset large model, the complete version of the fitting mature music (i.e., the complete song) is output. Another exemplary embodiment is that the encoding vectors corresponding to the fitting music part of some dimensions generated by splitting the fitting music, the encoding vectors corresponding to the sample music part of some dimensions generated by splitting the sample music, and the encoding vectors corresponding to the sample lyrics are respectively input into the preset large model, and the preset large model is trained. Exemplarily, when the coding vectors of some dimensions obtained after the sample music is split are input into the preset large model, the coding vectors of some dimensions obtained after the mature music is split are matched. For example, only the coding vectors of the accompaniment part of the sample music are input into the preset large model. At this time, the coding vectors of the vocal part of the sample music can be set to the preset code (such as 0, 1 or other special codes), and the preset codes are input into the preset large model at the same time, and the accompaniment part split from the mature music is output. The advantage of such a setting is that the trained large model can generate different music types (accompaniment, a cappella, song or other types or subdivided types of music) according to the music generation request input by the user.

[0099] In some embodiments, generating target music according to the output result of the music generation model includes: obtaining the output vector of the music generation model; converting the output vector into target music through a music generation module. In an embodiment of the present invention, the encoding vector output by the music generation model is similar to at least one layer of encoding vector structure of each frame discretized and encoded by the mature music fitted during the training of the music generation model. For example, if the mature music fitted during the model training is 25 frames per second, and each frame is represented by a 1-layer encoding vector, then the encoding vector output by the model is also a 1-layer encoding vector, and then the output vector of the music generation model is obtained through the music generation module, and the output vector is converted into the data of each frame of the corresponding target music; if the mature music fitted during the model training is 25 frames per second, and each frame is represented by a 4-layer encoding vector, then the encoding vector output by the model is also a 4-layer encoding vector, that is, 4 encoding vectors represent the data information of one frame, and then the output vector of the music output model is obtained through the music generation module, and the vector is converted into the data of each frame of the corresponding target music. Among them, the music generation module can be an open source speech synthesis model, a music generation model or other multimodal generation model in the prior art, such as a DIT model.

[0100] Embodiment 6

[0101] Figure 6 This is a schematic diagram of the structure of a music generating device provided by Embodiment 6 of the present invention. Figure 6 As shown, the device comprises:

[0102] The reference music acquisition module 610 is used to acquire reference music and target lyrics in response to a music generation event being triggered;

[0103] A reference coding vector generation module 620, configured to generate at least one reference coding vector based on the reference music; wherein the reference coding vector is a coding vector of at least one layer;

[0104] The word segmentation unit vector determination module 630 is used to perform word segmentation processing on the target lyrics and convert each word segmentation unit in the target lyrics into a corresponding word segmentation unit vector; wherein the word segmentation unit vector is an encoding vector of at least one layer;

[0105] The target music generation module 640 is used to input the at least one reference coding vector and the word segmentation unit vector into a pre-constructed music generation model, and generate target music according to the output result of the music generation model.

[0106] Optionally, the reference music acquisition module is used to:

[0107] When receiving a general music generation request input by a user, determining that a music generation event is triggered;

[0108] Accordingly, the reference coding vector generation module includes:

[0109] The reference coding vector generating unit is used to encode the reference music through an encoder to generate at least one reference coding vector.

[0110] Optionally, the reference coding vector generating unit is used to:

[0111] Discretize the reference music to generate at least two frames of first reference audio data;

[0112] The first reference audio data of each frame is encoded by an encoder to generate at least one reference encoding vector.

[0113] Optionally, the reference music acquisition module is used to:

[0114] When receiving a personalized music generation request input by a user, determining that a music generation event is triggered;

[0115] Accordingly, the reference coding vector generation module includes:

[0116] A reference music part extraction unit, configured to analyze the reference music and extract from the reference music a reference music part of at least one dimension matching the personalized music generation request;

[0117] The reference coding vector generating unit is used to encode the reference music part of each dimension through an encoder to generate at least one reference coding vector.

[0118] Optionally, a reference encoding vector generating unit is used to:

[0119] For each dimension of the reference music portion, discretize the reference music portion to generate at least two frames of second reference audio data;

[0120] The encoder encodes each frame of the second reference audio data to generate at least one reference encoding vector.

[0121] Optionally, also include:

[0122] A music training sample set acquisition module, used for acquiring a music training sample set before inputting the at least one reference coding vector and the word segmentation unit vector into a pre-built music generation model; wherein the music training sample set includes at least two groups of music training samples, each group of music training samples includes fitting music, at least one sample music and at least one sample lyrics;

[0123] A sample coding vector generation module, used to encode the fitting music, the sample music and the sample lyrics in each group of the music training samples, and generate at least one corresponding sample coding vector; wherein the sample coding vector is a coding vector of at least one layer;

[0124] The music generation big model generation module is used to train the preset big model based on the sample coding vector to generate the music generation big model.

[0125] Optionally, the sample coding vector generation module is used to:

[0126] Discretize the fitting music in each group of the music training samples to generate at least two frames of first fitting audio data;

[0127] Each frame of the first fitting audio data is encoded by an encoder to generate at least one fitting encoding vector.

[0128] Optionally, the sample coding vector generation module includes:

[0129] A sample music part extraction unit, configured to extract, for each group of the music training samples, a fitting music part of at least one dimension from the fitting music in the music training samples, and to extract a sample music part of at least one dimension from the sample music in the music training samples;

[0130] The sample coding vector generating unit is used to respectively encode the fitting music part of each dimension, the sample music part of each dimension and the sample lyrics in the music training sample to generate at least one corresponding sample coding vector.

[0131] Optionally, the sample coding vector generating unit is used to:

[0132] For each dimension of the fitted music part, discretize the fitted music part to generate at least two frames of second fitted audio data;

[0133] Each frame of the second fitting audio data is encoded by an encoder to generate at least one corresponding reference encoding vector.

[0134] Optionally, the music generation model generation module is used to:

[0135] Input at least one sample coding vector corresponding to the fitted music part of all dimensions or part of the dimensions, at least one sample coding vector corresponding to the sample music part of all dimensions or part of the dimensions, and at least one sample coding vector corresponding to the sample lyrics into a preset large model, train the preset large model, and generate a music generation large model.

[0136] Optionally, the target music generation module is used to:

[0137] Obtaining an output vector of the music generation model;

[0138] The output vector is converted into target music through a music generation module.

[0139] Optionally, the reference music acquisition module is used to:

[0140] Obtain at least 2 reference music containing at least one different element, feature, style, or dimension;

[0141] Accordingly, the reference coding vector generation module is used to:

[0142] Generate at least two reference encoding vectors based on at least two reference musics containing at least one different element, feature, style, or dimension;

[0143] The target music generation module is used to:

[0144] The at least two reference coding vectors and the word segmentation unit vector are input into a pre-constructed music generation model, and target music that incorporates at least one different element, feature, style, and dimension of at least two reference music is generated based on the output result of the music generation model.

[0145] The music generating device provided in the embodiment of the present invention can execute the music generating method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0146] Embodiment 7

[0147] Figure 7 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0148] like Figure 7As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0149] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0150] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 performs the various methods and processes described above, such as a music generation method.

[0151] In some embodiments, the music generation method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the music generation method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the music generation method in any other appropriate manner (e.g., by means of firmware).

[0152] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0153] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0154] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0155] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0156] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0157] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0158] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0159] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A music generation method, It is characterized in that include: In response to a music generation event being triggered, obtaining reference music and target lyrics; Generate at least one reference encoding vector based on the reference music; wherein the reference encoding vector is an encoding vector of at least one layer; Performing word segmentation processing on the target lyrics, and converting each word segmentation unit in the target lyrics into a corresponding word segmentation unit vector; wherein the word segmentation unit vector is an encoding vector of at least one layer; The at least one reference coding vector and the word segmentation unit vector are input into a pre-constructed music generation model, and the target music is generated according to the output result of the music generation model.

2. The method according to claim 1, It is characterized in that In response to a music generation event being triggered, including: When receiving a general music generation request input by a user, determining that a music generation event is triggered; Accordingly, generating at least one reference encoding vector based on the reference music includes: The reference music is encoded by an encoder to generate at least one reference encoding vector.

3. The method according to claim 1, It is characterized in that In response to a music generation event being triggered, including: When receiving a personalized music generation request input by a user, determining that a music generation event is triggered; Accordingly, generating at least one reference encoding vector based on the reference music includes: Analyze the reference music, and extract from the reference music a reference music portion of at least one dimension matching the personalized music generation request; For the reference music portion of each dimension, the reference music portion is encoded by an encoder to generate at least one reference encoding vector.

4. The method according to claim 1, It is characterized in that Before inputting the at least one reference encoding vector and the word segmentation unit vector into a pre-built music generation model, the method further includes: Obtain a music training sample set; wherein the music training sample set includes at least two groups of music training samples, each group of music training samples includes fitting music, at least one sample music and at least one sample lyrics; Encode the fitting music, the sample music and the sample lyrics in each group of the music training samples respectively to generate at least one corresponding sample encoding vector; wherein the sample encoding vector is an encoding vector of at least one layer; The preset large model is trained based on the sample coding vector to generate a music generation large model.

5. The method according to claim 4, It is characterized in that Encoding the fitting music in each group of the music training samples to generate at least one corresponding sample encoding vector includes: Discretize the fitting music in each group of the music training samples to generate at least two frames of first fitting audio data; Each frame of the first fitting audio data is encoded by an encoder to generate at least one fitting encoding vector.

6. The method according to claim 4, It is characterized in that Encoding the fitting music, the sample music and the sample lyrics in each group of the music training samples respectively to generate at least one corresponding sample encoding vector includes: For each group of the music training samples, extracting a fitting music part of at least one dimension from the fitting music in the music training samples, and extracting a sample music part of at least one dimension from the sample music in the music training samples; The fitting music part of each dimension, the sample music part of each dimension and the sample lyrics in the music training sample are encoded respectively to generate at least one corresponding sample encoding vector.

7. The method according to claim 6, It is characterized in that Encoding the fitted music part of each dimension to generate at least one corresponding sample encoding vector includes: For each dimension of the fitted music part, discretize the fitted music part to generate at least two frames of second fitted audio data; Each frame of the second fitting audio data is encoded by an encoder to generate at least one corresponding reference encoding vector.

8. The method according to claim 7, It is characterized in that The preset large model is trained based on the sample coding vector to generate a music generation large model, including: Input at least one sample coding vector corresponding to the fitted music part of all dimensions or part of the dimensions, at least one sample coding vector corresponding to the sample music part of all dimensions or part of the dimensions, and at least one sample coding vector corresponding to the sample lyrics into a preset large model, train the preset large model, and generate a music generation large model.

9. The method according to claim 1, It is characterized in that Generating target music according to the output result of the music generation model includes: Obtaining an output vector of the music generation model; The output vector is converted into target music through a music generation module.

10. The method according to any one of claims 1 to 9, It is characterized in that The obtaining of reference music comprises: Obtain at least 2 reference music containing at least one different element, feature, style, or dimension; Accordingly, generating at least one reference encoding vector based on the reference music includes: Generate at least two reference encoding vectors based on at least two reference musics containing at least one different element, feature, style, dimension; Inputting the at least one reference encoding vector and the word segmentation unit vector into a pre-constructed music generation model, and generating target music according to an output result of the music generation model, including: The at least two reference coding vectors and the word segmentation unit vector are input into a pre-constructed music generation model, and target music that incorporates at least one different element, feature, style, and dimension of at least two reference music is generated based on the output result of the music generation model.

11. A music generating device, It is characterized in that include: A reference music acquisition module, for acquiring reference music and target lyrics in response to a music generation event being triggered; A reference coding vector generation module, used to generate at least one reference coding vector based on the reference music; wherein the reference coding vector is a coding vector of at least one layer; A word segmentation unit vector determination module is used to perform word segmentation processing on the target lyrics and convert each word segmentation unit in the target lyrics into a corresponding word segmentation unit vector; wherein the word segmentation unit vector is an encoding vector of at least one layer; The target music generation module is used to input the at least one reference coding vector and the word segmentation unit vector into a pre-constructed music generation model, and generate target music according to the output result of the music generation model.

12. An electronic device, It is characterized in that The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the music generating method according to any one of claims 1 to 10.

13. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to implement the music generation method according to any one of claims 1 to 10 when executed by a processor.

Images