Song creation method and system based on large model and storage medium

Abstract

The invention provides a song creation method and system based on a large model and a storage medium, and the method comprises the steps: carrying out the audio collection of a user, obtaining an input audio, carrying out the feature extraction of the input audio, and obtaining an audio feature; singing collection is carried out on the user to obtain a singing audio, and music style analysis is carried out on the singing audio to obtain a music style analysis result; determining a target music style according to the music style analysis result, and obtaining a song description of the user; and inputting the song description, the audio feature and the target music style into a pre-trained large model for song creation, and obtaining an output song. According to the embodiment of the invention, the song description, the audio feature and the target music style are input into the pre-trained large model for song creation, song creation can be effectively carried out based on the audio feature and the proper music style of the user, songs with personal characteristics of the user are generated, and the use experience of the user is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of song creation, and in particular to a song creation method, system and storage medium based on a large model. Background Art

[0002] Songwriting has always been a field for a small number of people. For those who have not received music education, it is difficult to create good songs even if they have talent or inspiration. To solve this problem, song generation software for song creation has emerged. You only need to input some song information into the song generation software, and the song generation software can automatically create songs.

[0003] In the existing song creation process, the song styles generated by the song generation software are relatively random and cannot match the user's own suitable style, which reduces the user's experience. Summary of the invention

[0004] The purpose of the embodiments of the present invention is to provide a song creation method, system and storage medium based on a large model to solve the problem of poor user experience in the song creation process of the prior art.

[0005] The embodiment of the present invention is implemented as follows: a song creation method based on a large model, the method comprising:

[0006] Collecting audio from the user to obtain recorded audio, and extracting features from the recorded audio to obtain audio features;

[0007] Collecting singing of the user to obtain singing audio, and performing music style analysis on the singing audio to obtain music style analysis results;

[0008] Determine a target music genre according to the music genre analysis result, and obtain a song description of the user;

[0009] The song description, the audio features and the target music style are input into the pre-trained large model to create the song and obtain an output song.

[0010] Preferably, feature extraction is performed on the recorded audio to obtain audio features, including:

[0011] Convert the input audio into text to obtain an audio text, and obtain the initial consonant and final vowel of each text character in the audio text respectively;

[0012] Acquire the pitch of the recorded audio, and determine the pitch of each text character according to the pitch of the recorded audio;

[0013] A character triplet feature is generated according to the pitch, initial consonant and final vowel of each text character, and feature mapping is performed on the character triplet feature to obtain the audio feature.

[0014] Preferably, performing a music style analysis on the singing audio to obtain a music style analysis result includes:

[0015] Performing audio segmentation on the singing audio to obtain segmented audios, and determining the original singing audio according to the time stamps of the segmented audios;

[0016] Calculating the audio similarity between the original singing audio and the corresponding segmented audio, and determining the singing score of the segmented audio according to the audio similarity;

[0017] The genre type of the segmented audio is obtained, and the genre score of each genre type is determined according to the singing score to obtain the genre analysis result.

[0018] Preferably, calculating the audio similarity between the original audio and the corresponding segmented audio includes:

[0019] Acquiring pitch information of the segmented audio, and generating a user audio track according to the pitch information;

[0020] The similarity between the user audio track and the standard audio track of the original audio is calculated to obtain the audio similarity.

[0021] Preferably, the song description, the audio features and the target music style are input into the pre-trained large model to create the song, and before the output song is obtained, the method further includes:

[0022] Obtain a model training sample, and input the sample description, audio sample features, and sample music style in the model training sample into the large model for convolution processing to obtain description convolution features, audio convolution features, and music style convolution features;

[0023] Performing feature decoding on the description convolution feature, the audio convolution feature, and the music style convolution feature to obtain a prediction feature;

[0024] The model loss is determined according to the prediction features and the sample labels of the model training samples, and the parameters of the large model are updated according to the model loss until the large model converges to obtain the pre-trained large model.

[0025] Preferably, feature decoding is performed on the description convolution feature, the audio convolution feature and the music style convolution feature to obtain the prediction feature, including:

[0026] Performing feature mapping on the description convolution feature, the audio convolution feature and the music style convolution feature to obtain description mapping features, audio mapping features and music style mapping features;

[0027] Performing residual processing on the description mapping feature, the audio mapping feature and the music style mapping feature to obtain description residual features, audio residual features and music style residual features;

[0028] The description residual feature, the audio residual feature and the music style residual feature are processed by a self-attention mechanism to obtain a fusion feature, and the fusion feature is deconvolved to obtain the prediction feature.

[0029] Preferably, the song description, the audio features and the target music style are input into the pre-trained large model to create the song, and after the output song is obtained, the method further comprises:

[0030] Acquire song modification information of the user, and modify the output song according to the song modification information;

[0031] Setting the modified output song and the corresponding song description, the audio features and the target music genre as a model fine-tuning sample;

[0032] When the sample quantity of the model fine-tuning samples is greater than the quantity threshold, the parameters of the pre-trained large model are fine-tuned according to the model fine-tuning samples.

[0033] Another object of an embodiment of the present invention is to provide a song creation system based on a large model, the system comprising:

[0034] The feature extraction module is used to collect audio from the user to obtain recorded audio, and to extract features from the recorded audio to obtain audio features;

[0035] A music style analysis module, used to collect singing data of the user to obtain singing audio, and to perform music style analysis on the singing audio to obtain music style analysis results;

[0036] Determine a target music genre according to the music genre analysis result, and obtain a song description of the user;

[0037] The song creation module is used to input the song description, the audio features and the target style into the pre-trained large model to create the song and obtain an output song.

[0038] Preferably, the feature extraction module is also used for:

[0039] Convert the input audio into text to obtain an audio text, and obtain the initial consonant and final vowel of each text character in the audio text respectively;

[0040] Acquire the pitch of the recorded audio, and determine the pitch of each text character according to the pitch of the recorded audio;

[0041] A character triplet feature is generated according to the pitch, initial consonant and final vowel of each text character, and feature mapping is performed on the character triplet feature to obtain the audio feature.

[0042] The embodiment of the present invention can effectively create songs based on the user's audio features and suitable styles by inputting song descriptions, audio features and target genres into a pre-trained large model for song creation, thereby generating songs with personal characteristics of the user and improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 is a flow chart of a song creation method based on a large model provided by a first embodiment of the present invention;

[0044] Figure 2 is a structural diagram of a song creation system based on a large model provided by a second embodiment of the present invention;

[0045] Figure 3 is a specific implementation step diagram of a song creation system based on a large model provided by a second embodiment of the present invention;

[0046] Figure 4 It is a schematic diagram of the structure of a terminal device provided in the third embodiment of the present invention. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0048] In order to illustrate the technical solution of the present invention, a specific embodiment is provided below for illustration.

[0049] Embodiment 1

[0050] See also Figure 1 , is a flow chart of a song creation method based on a large model provided in a first embodiment of the present invention. The song creation method based on a large model can be applied to any device or system. The song creation method based on a large model includes the following steps:

[0051] Step S10, collecting audio from the user to obtain recorded audio, and extracting features from the recorded audio to obtain audio features;

[0052] The recorded audio is obtained by prompting the user to output voice, and the audio features representing the user's personal timbre characteristics are obtained by extracting features from the recorded audio.

[0053] Optionally, feature extraction is performed on the recorded audio to obtain audio features, including:

[0054] The recorded audio is converted into text to obtain an audio text, and the initial consonants and final vowels of each text character in the audio text are respectively obtained; wherein, the recorded audio is recognized to achieve the effect of converting the recorded audio into text to obtain the audio text, the audio text is converted into letters to obtain a letter string, and the letters corresponding to each text character in the letter string are segmented into initial consonants and final vowels to obtain the initial consonants and final vowels of each text character;

[0055] Obtaining the pitch of the recorded audio, and determining the pitch of each text character according to the pitch of the recorded audio; wherein the pitch of the recorded audio is obtained by performing spectrum analysis on the recorded audio, the audio time of each text character in the recorded audio is obtained, and based on the audio time, the information corresponding to the pitch of each text character in the recorded audio is determined respectively to obtain the pitch of each text character;

[0056] Generate character triple features according to the pitch, initial consonant and final of each text character, and perform feature mapping on the character triple features to obtain the audio features; wherein, matrix transformation is performed on the pitch, initial consonant and final of each text character to obtain character triple features, and feature mapping is performed on the character triple features according to a preset mapping relationship to obtain the audio features.

[0057] Step S20, collecting singing of the user to obtain singing audio, and performing music style analysis on the singing audio to obtain music style analysis results;

[0058] Among them, multiple song accompaniments are played randomly to the user, and the user's singing information with the song accompaniment is collected to obtain the singing audio. The singing audio is analyzed by genre to analyze the suitability between the user and multiple songs of different genres.

[0059] Optionally, performing a music style analysis on the singing audio to obtain a music style analysis result includes:

[0060] The singing audio is segmented to obtain segmented audio, and the original singing audio is determined according to the time mark of the segmented audio; wherein the singing audio is segmented according to a preset time interval to obtain the segmented audio, and the preset time interval can be set according to demand. By performing audio segmentation on the singing audio, the audio information corresponding to each song accompaniment in the singing audio can be effectively obtained, and the time mark is used to indicate the audio starting time corresponding to each segmented audio, and the song accompaniment corresponding to the segmented audio is determined based on the audio starting time. Based on the corresponding relationship between the segmented audio and the song accompaniment, the original singing song of the song accompaniment is determined as the original singing audio corresponding to the segmented audio;

[0061] Calculating the audio similarity between the original singing audio and the corresponding segmented audio, and determining the singing score of the segmented audio according to the audio similarity; wherein the audio similarity is matched with a score query table to obtain the singing score, and the score query table stores the corresponding relationship between different audio similarities and corresponding singing scores;

[0062] The genre type of the segmented audio is obtained, and the genre score of each genre type is determined according to the singing score to obtain the genre analysis result; wherein, the singing scores corresponding to the same genre type are summed to obtain the genre score of each genre type, and the higher the genre score is, the more suitable the genre type corresponding to the genre score is for the user, that is, the user is better at singing songs of the corresponding genre.

[0063] Furthermore, the audio similarity between the original audio and the corresponding segmented audio is calculated, including: obtaining the pitch information of the segmented audio and generating a user audio track according to the pitch information; calculating the similarity between the user audio track and the standard audio track of the original audio to obtain the audio similarity.

[0064] Step S30, determining a target music genre according to the music genre analysis result, and obtaining a song description of the user;

[0065] Among them, the genre type corresponding to the maximum genre score is determined as the target genre, and the user's song description of the song to be generated is obtained. The song description can be filled in according to user needs, and the song description may include information such as lyrics desired by the user, song meaning or song scene description.

[0066] Step S40, inputting the song description, the audio features and the target music style into the pre-trained large model to create a song, and obtaining an output song;

[0067] Among them, song creation is carried out by inputting song description, audio features and target style into a pre-trained large model, so that users can participate in song creation without professional music knowledge. By extracting the user's audio features, music works with personal characteristics are generated. The songs generated by users have their own timbre, which improves the user experience and enhances the user's desire to share. The automated song generation process reduces the time and cost of professional music production.

[0068] Optionally, before inputting the song description, the audio features and the target music style into the pre-trained large model to create the song and obtaining the output song, the method further includes:

[0069] Obtain a model training sample, and input the sample description, audio sample features, and sample music style in the model training sample into the large model for convolution processing to obtain description convolution features, audio convolution features, and music style convolution features;

[0070] Performing feature decoding on the description convolution feature, the audio convolution feature, and the music style convolution feature to obtain a prediction feature;

[0071] Determine the model loss according to the prediction features and the sample labels of the model training samples, and update the parameters of the large model according to the model loss until the large model converges to obtain the pre-trained large model;

[0072] Among them, the sample description, audio sample features and sample style in the model training samples are input into the large model for convolution processing to achieve the feature encoding effect of the sample description, audio sample features and sample style, and the parameters of the large model are updated through model loss, so that the converged large model can effectively create output songs that are suitable for the user's style and have the same timbre as the user based on the song description, audio features and target style, thereby improving the user experience.

[0073] Furthermore, feature decoding is performed on the description convolution feature, the audio convolution feature, and the music style convolution feature to obtain prediction features, including:

[0074] Performing feature mapping on the description convolution feature, the audio convolution feature and the music style convolution feature to obtain description mapping feature, audio mapping feature and music style mapping feature; wherein, by performing vector conversion on the description convolution feature, the audio convolution feature and the music style convolution feature, the feature mapping effect of the description convolution feature, the audio convolution feature and the music style convolution feature is achieved;

[0075] The description mapping feature, the audio mapping feature and the style mapping feature are subjected to residual processing to obtain description residual features, audio residual features and style residual features; wherein, by performing residual processing on the description mapping feature, the audio mapping feature and the style mapping feature, it is effectively facilitated for the large model to learn the distinguishing features between the description mapping feature, the audio mapping feature and the style mapping feature and the corresponding input information;

[0076] The description residual features, the audio residual features and the style residual features are processed by a self-attention mechanism to obtain fused features, and the fused features are deconvolved to obtain the prediction features; wherein by processing the description residual features, the audio residual features and the style residual features by a self-attention mechanism, the description residual features, the audio residual features and the style residual features can be effectively fused to obtain fused features.

[0077] Furthermore, after the song description, the audio features and the target music style are input into the pre-trained large model to create the song and the output song is obtained, the method further includes:

[0078] Acquire song modification information of the user, and modify the output song according to the song modification information;

[0079] Setting the modified output song and the corresponding song description, the audio features and the target music genre as a model fine-tuning sample;

[0080] When the number of the model fine-tuning samples is greater than a quantity threshold, fine-tuning parameters of the pre-trained large model according to the model fine-tuning samples;

[0081] Among them, the output song is modified through song modification information, which further improves the suitability between the user and the output song. By setting the modified output song and the corresponding song description, audio features and target style as model fine-tuning samples, the parameter fine-tuning of the pre-trained large model is effectively guaranteed, further improving the accuracy of the pre-trained large model.

[0082] In this embodiment, by inputting song description, audio features and target genre into the pre-trained large model for song creation, song creation can be effectively performed based on the user's audio features and suitable genre, generating songs with the user's personal characteristics and improving the user experience.

[0083] Embodiment 2

[0084] See also Figure 2 , is a structural diagram of a song creation system 100 based on a large model provided in a second embodiment of the present invention, including:

[0085] The feature extraction module 10 is used to collect audio from the user to obtain recorded audio, and to extract features from the recorded audio to obtain audio features.

[0086] Optionally, the feature extraction module 10 is further used to: convert the input audio into text to obtain an audio text, and respectively obtain the initial consonant and final vowel of each text character in the audio text;

[0087] Acquire the pitch of the recorded audio, and determine the pitch of each text character according to the pitch of the recorded audio;

[0088] A character triplet feature is generated according to the pitch, initial consonant and final vowel of each text character, and feature mapping is performed on the character triplet feature to obtain the audio feature.

[0089] A music style analysis module 11 is used to collect singing data of the user to obtain singing audio, and to perform music style analysis on the singing audio to obtain a music style analysis result;

[0090] The target genre is determined according to the genre analysis result, and the song description of the user is obtained.

[0091] Optionally, the music style analysis module 11 is further used to: perform audio segmentation on the singing audio to obtain segmented audio, and determine the original singing audio according to the time stamp of the segmented audio;

[0092] Calculating the audio similarity between the original singing audio and the corresponding segmented audio, and determining the singing score of the segmented audio according to the audio similarity;

[0093] The genre type of the segmented audio is obtained, and the genre score of each genre type is determined according to the singing score to obtain the genre analysis result.

[0094] Furthermore, the music style analysis module 11 is also used to: obtain the pitch information of the segmented audio, and generate a user audio track according to the pitch information;

[0095] The similarity between the user audio track and the standard audio track of the original audio is calculated to obtain the audio similarity.

[0096] The song creation module 12 is used to input the song description, the audio features and the target style into the pre-trained large model to create the song and obtain an output song.

[0097] Optionally, the song creation module 12 is further used to: obtain a model training sample, and input the sample description, audio sample features and sample music style in the model training sample into the large model for convolution processing to obtain description convolution features, audio convolution features and music style convolution features;

[0098] Performing feature decoding on the description convolution feature, the audio convolution feature, and the music style convolution feature to obtain a prediction feature;

[0099] The model loss is determined according to the prediction features and the sample labels of the model training samples, and the parameters of the large model are updated according to the model loss until the large model converges to obtain the pre-trained large model.

[0100] Furthermore, the song creation module 12 is also used to: perform feature mapping on the description convolution feature, the audio convolution feature and the music style convolution feature to obtain description mapping features, audio mapping features and music style mapping features;

[0101] Performing residual processing on the description mapping feature, the audio mapping feature and the music style mapping feature to obtain description residual features, audio residual features and music style residual features;

[0102] The description residual feature, the audio residual feature and the music style residual feature are processed by a self-attention mechanism to obtain a fusion feature, and the fusion feature is deconvolved to obtain the prediction feature.

[0103] Furthermore, the song creation module 12 is also used to: obtain the user's song modification information, and modify the output song according to the song modification information;

[0104] Setting the modified output song and the corresponding song description, the audio features and the target music genre as a model fine-tuning sample;

[0105] When the sample quantity of the model fine-tuning samples is greater than the quantity threshold, the parameters of the pre-trained large model are fine-tuned according to the model fine-tuning samples.

[0106] See also Figure 3 , which is a specific implementation step diagram of the song creation system 100 based on the big model, including:

[0107] The user records dozens of sentences according to the system's instructions to obtain recorded audio. The system analyzes the timbre, pitch and other characteristics of the recorded audio to obtain audio features.

[0108] The user inputs dozens of lines of songs according to the system's guidance and obtains the singing audio. The system analyzes the style and tune suitable for the user and obtains the target style.

[0109] The user enters a description of the music, or enters some tunes to get a song description;

[0110] The music model combines and analyzes the user's timbre (audio features), suitable music genre (target music genre) and song description;

[0111] The music model uses the user's timbre to generate music that fits the user's style and description, and obtains the output song;

[0112] Based on user usage, we collect more data and regularly optimize the music model to enhance the model's ability to generate songs using personalized timbres.

[0113] In this embodiment, song creation is performed by inputting song description, audio features and target style into a pre-trained large model, so that users can participate in song creation without professional music knowledge. By extracting the user's audio features, music works with personal characteristics are generated. The songs generated by the user have their own timbre, which improves the user experience and enhances the user's desire to share. The automated song generation process reduces the time and cost of professional music production. Users can freely edit and adjust music works to achieve personalized expression.

[0114] Embodiment 3

[0115] Figure 4 2 is a block diagram of a terminal device 2 provided in the third embodiment of the present application. Figure 4 As shown, the terminal device 2 of this embodiment includes: a processor 20, a memory 21, and a computer program 22 stored in the memory 21 and executable on the processor 20, such as a program of a song creation method based on a large model. When the processor 20 executes the computer program 22, the steps in each embodiment of the song creation method based on a large model are implemented.

[0116] Exemplarily, the computer program 22 may be divided into one or more modules, which are stored in the memory 21 and executed by the processor 20 to complete the present application. The one or more modules may be a series of computer program instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program 22 in the terminal device 2. The terminal device may include, but is not limited to, a processor 20 and a memory 21.

[0117] The processor 20 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0118] The memory 21 may be an internal storage unit of the terminal device 2, such as a hard disk or memory of the terminal device 2. The memory 21 may also be an external storage device of the terminal device 2, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the terminal device 2. Further, the memory 21 may also include both an internal storage unit and an external storage device of the terminal device 2. The memory 21 is used to store the computer program and other programs and data required by the terminal device. The memory 21 may also be used to temporarily store data that has been output or is to be output.

[0119] In addition, each functional module in each embodiment of the present application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of software functional unit.

[0120] If the integrated module is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Among them, the computer-readable storage medium can be non-volatile or volatile. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable storage medium may include: any entity or device capable of carrying computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in computer-readable storage media can be appropriately increased or decreased according to the requirements of legislation and patent practices in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practices, computer-readable storage media do not include electrical carrier signals and telecommunication signals.

[0121] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A song creation method based on a large model, characterized in that: The method comprises: Collecting audio from the user to obtain recorded audio, and extracting features from the recorded audio to obtain audio features; Collecting singing of the user to obtain singing audio, and performing music style analysis on the singing audio to obtain music style analysis results; Determine a target music genre according to the music genre analysis result, and obtain a song description of the user; The song description, the audio features and the target music style are input into the pre-trained large model to create the song and obtain an output song.

2. The song creation method based on a large model as described in claim 1 is characterized in that: Performing feature extraction on the recorded audio to obtain audio features includes: Convert the input audio into text to obtain an audio text, and obtain the initial consonant and final vowel of each text character in the audio text respectively; Acquire the pitch of the recorded audio, and determine the pitch of each text character according to the pitch of the recorded audio; A character triplet feature is generated according to the pitch, initial consonant and final vowel of each text character, and feature mapping is performed on the character triplet feature to obtain the audio feature.

3. The song creation method based on a large model as described in claim 1 is characterized in that: Performing a music style analysis on the singing audio to obtain a music style analysis result includes: Performing audio segmentation on the singing audio to obtain segmented audios, and determining the original singing audio according to the time stamps of the segmented audios; Calculating the audio similarity between the original singing audio and the corresponding segmented audio, and determining the singing score of the segmented audio according to the audio similarity; The genre type of the segmented audio is obtained, and the genre score of each genre type is determined according to the singing score to obtain the genre analysis result.

4. The song creation method based on a large model as described in claim 3 is characterized in that: Calculating the audio similarity between the original audio and the corresponding segmented audio, including: Acquiring pitch information of the segmented audio, and generating a user audio track according to the pitch information; The similarity between the user audio track and the standard audio track of the original audio is calculated to obtain the audio similarity.

5. The song creation method based on a large model as claimed in claim 1, characterized in that: Inputting the song description, the audio features and the target music style into the pre-trained large model to create the song, and obtaining the output song also includes: Obtain a model training sample, and input the sample description, audio sample features, and sample music style in the model training sample into the large model for convolution processing to obtain description convolution features, audio convolution features, and music style convolution features; Performing feature decoding on the description convolution feature, the audio convolution feature, and the music style convolution feature to obtain a prediction feature; The model loss is determined according to the prediction features and the sample labels of the model training samples, and the parameters of the large model are updated according to the model loss until the large model converges to obtain the pre-trained large model.

6. The song creation method based on a large model as described in claim 5 is characterized in that: Feature decoding is performed on the description convolution feature, the audio convolution feature, and the music style convolution feature to obtain prediction features, including: Performing feature mapping on the description convolution feature, the audio convolution feature and the music style convolution feature to obtain description mapping features, audio mapping features and music style mapping features; Performing residual processing on the description mapping feature, the audio mapping feature and the music style mapping feature to obtain description residual features, audio residual features and music style residual features; The description residual feature, the audio residual feature and the music style residual feature are processed by a self-attention mechanism to obtain a fusion feature, and the fusion feature is deconvolved to obtain the prediction feature.

7. The song creation method based on a large model as claimed in claim 1, characterized in that: After inputting the song description, the audio features and the target music style into the pre-trained large model to create the song and obtaining the output song, the method further includes: Acquire song modification information of the user, and modify the output song according to the song modification information; Setting the modified output song and the corresponding song description, the audio features and the target music genre as a model fine-tuning sample; When the sample quantity of the model fine-tuning samples is greater than the quantity threshold, the parameters of the pre-trained large model are fine-tuned according to the model fine-tuning samples.

8. A song creation system based on a large model, characterized in that: The system comprises: The feature extraction module is used to collect audio from the user to obtain recorded audio, and to extract features from the recorded audio to obtain audio features; A music style analysis module, used to collect singing data of the user to obtain singing audio, and to perform music style analysis on the singing audio to obtain music style analysis results; Determine a target music genre according to the music genre analysis result, and obtain a song description of the user; The song creation module is used to input the song description, the audio features and the target style into the pre-trained large model to create the song and obtain an output song.

9. The song creation system based on a large model as claimed in claim 8, characterized in that: The feature extraction module is also used for: Convert the input audio into text to obtain an audio text, and obtain the initial consonant and final vowel of each text character in the audio text respectively; Acquire the pitch of the recorded audio, and determine the pitch of each text character according to the pitch of the recorded audio; A character triplet feature is generated according to the pitch, initial consonant and final vowel of each text character, and feature mapping is performed on the character triplet feature to obtain the audio feature.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images