Song generation method based on artificial intelligence

Abstract

The invention provides a song generation method based on artificial intelligence, belongs to the technical field of computers, and aims to solve the problems that in the related technology, efficiency is low when songs are generated, and complete music meeting user requirements cannot be accurately generated. The method comprises the following steps: calling an artificial intelligence music large model to obtain an initial music according to a song type and a song theme input by a user; the initial music comprises lyrics and an initial melody; determining a modification fragment in the initial melody; receiving a humming voice of a user for the modified segment; calling an artificial intelligence accompaniment large model to generate an editing melody according to the humming voice; replacing the modified segment of the initial melody with the edited melody to obtain a target melody; and according to the target melody and the lyrics, calling an artificial intelligence human voice loud model to generate singing human voices, and obtaining a complete music.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a song generation method based on artificial intelligence. Background Art

[0002] Artificial intelligence has made significant inroads in the music industry, enabling the generation of music and lyrics thanks to increases in computing power and data storage, as well as advances in machine learning and natural language processing.

[0003] Tools and applications that use AI to assist in songwriting have also been developed. Some tools can automatically generate music but cannot interact with users; some tools can convert humming into accompaniment but cannot generate complete music.

[0004] Therefore, the song generation tools provided in the related art are inefficient in generating songs and cannot accurately generate complete music that meets user needs. Summary of the invention

[0005] In order to overcome the problems existing in the related art, the present disclosure provides a song generation method based on artificial intelligence. The technical solution of the present disclosure is as follows: According to a first aspect of an embodiment of the present disclosure, there is provided a song generation method, comprising: According to the song type and song theme input by the user, the artificial intelligence music model is called to obtain an initial music; the initial music includes lyrics and an initial melody; determining a modified segment of the initial melody; receiving a humming voice of a user for the modified segment; According to the humming voice, calling the artificial intelligence accompaniment model to generate an edited melody; Using the edited melody to replace the modified segment of the initial melody to obtain a target melody; According to the target melody and the lyrics, an artificial intelligence vocal model is called to generate singing vocals to obtain a complete piece of music.

[0006] Optionally, using the edited melody to replace the modified segment of the initial melody to obtain a target melody includes: Determine a portion of the initial melody located before the modified segment in the initial music as a first melody, and determine a portion of the initial melody located after the modified segment in the initial music as a second melody; Get the humming segment adjustment algorithm; According to the first melody and the second melody, the rhythm of the edited melody is adjusted using the part of the humming segment adjustment algorithm used for rhythm adjustment to obtain an edited melody with adjusted rhythm; According to the first melody and the second melody, the pitch of the edited melody after the rhythm is adjusted is adjusted using the part used for pitch adjustment in the humming segment adjustment algorithm to obtain the edited melody after the rhythm and pitch are adjusted; The edited melody after the rhythm and pitch are adjusted is used to replace the modified segment of the initial melody to obtain the target melody.

[0007] Optionally, after obtaining the edited melody after the rhythm and pitch are adjusted, the method further includes: Fine-tuning the edited melody after the rhythm and pitch are adjusted to obtain the fine-tuned edited melody; The step of replacing the modified segment of the initial melody with the edited melody after the rhythm and pitch are adjusted to obtain the target melody comprises: The modified segment of the initial melody is replaced by the edited melody after fine-tuning to obtain the target melody.

[0008] Optionally, it also includes: If the target melody does not meet the user's needs, return to the steps of: receiving the user's humming voice for the modified segment; calling the artificial intelligence accompaniment model to generate an edited melody based on the humming voice; using the edited melody to replace the modified segment of the initial melody to obtain the target melody.

[0009] Optionally, the initial melody includes a main melody and an accompaniment, and determining the modified segment in the initial melody includes: Determine the main melody corresponding to the modified segment, and / or determine the accompaniment corresponding to the modified segment; According to the humming voice, the artificial intelligence accompaniment model is called to generate an edited melody, including: Generate a humming melody according to the humming voice; the humming melody is used to: replace the main melody corresponding to the modified segment, and / or replace the accompaniment corresponding to the modified segment; Determining a retained portion of the modified segment that is not replaced by the humming melody; The notes and beats of the humming melody are adjusted and combined with the retained parts to obtain an edited melody.

[0010] Optionally, according to the first melody and the second melody, the rhythm of the edited melody is adjusted using a portion of the humming segment adjustment algorithm for rhythm adjustment to obtain an edited melody after the rhythm is adjusted, including: respectively determining original beat points corresponding to the first melody and the second melody; Determining the beat points of the edited melody by identifying the beat pattern of the edited melody; The beat points of the edited melody are adjusted to be aligned with the original beat points to obtain the edited melody with adjusted rhythm.

[0011] Optionally, according to the first melody and the second melody, the pitch of the edited melody after the rhythm is adjusted is adjusted using the part used for pitch adjustment in the humming segment adjustment algorithm to obtain the edited melody after the rhythm and pitch are adjusted, comprising: Determining key notes corresponding to each beat point in the edited melody after the rhythm adjustment; determining any of the key notes as a target key note; Determining a first initial note corresponding to the target key note from the first melody and determining a second initial note corresponding to the target key note from the second melody; Determine the target key note, the first initial note, and the second initial note as a triplet mapping corresponding to the target key note; Determine the target key note and a subsequent key note adjacent to the target key note as a binary mapping corresponding to the target key note; The pitch of the edited melody is adjusted by using the triplet mapping and the binary mapping corresponding to each key note to obtain the edited melody with adjusted rhythm and pitch; Wherein, each of the triplet mappings represents: the pitch difference between the edited melody after the rhythm adjustment and the initial melody; each of the binary mappings represents: the pitch difference between each of the key notes within the edited melody after the rhythm adjustment.

[0012] Optionally, after obtaining the target melody, the method further includes: By adjusting the generation parameters in the artificial intelligence chord model, the chords of the target melody are adjusted as a whole; the generation parameters include rhythm, pitch, coordination and sound quality.

[0013] Optionally, according to the target melody and the lyrics, the artificial intelligence vocal model is called to generate singing vocals to obtain a complete piece of music, including: Deleting or adding each segment in the target melody to obtain an adjusted target melody; Modifying the lyrics to obtain adjusted lyrics; Obtaining a target music piece according to the adjusted target melody and the adjusted lyrics; According to the target music, the artificial intelligence vocal model is called to generate the singing vocals, and the singing vocals are generated to obtain the complete music.

[0014] Optionally, according to the target music, the artificial intelligence vocal model is called to generate singing vocals, and the singing vocals are generated to obtain a complete music, including: According to the singing effect of the vocals, the music section, lyrics, rhythm and pitch of the target music are readjusted to obtain an adjusted target music; According to the adjusted target music, the artificial intelligence vocal model is called to generate the singing vocals, and the singing vocals are regenerated to obtain the complete music.

[0015] According to a second aspect of an embodiment of the present disclosure, there is provided a song generation device, comprising: An acquisition module, used to call the artificial intelligence music model to obtain an initial music according to the song type and song theme input by the user; the initial music includes lyrics and an initial melody; A determination module, used for determining a modified segment in the initial melody; A receiving module, used for receiving a user's humming voice for the modified segment; An editing module, used to call an artificial intelligence accompaniment model to generate an edited melody according to the humming voice; A replacement module, used to replace the modified segment of the initial melody with the edited melody to obtain a target melody; The completion module is used to call the artificial intelligence vocal model to generate singing vocals according to the target melody and the lyrics to obtain a complete piece of music.

[0016] According to a third aspect of an embodiment of the present disclosure, there is provided an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the computer program is executed by the processor, the steps of the song generation method based on artificial intelligence as described in the first aspect are implemented.

[0017] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the artificial intelligence-based song generation method as described in the first aspect are implemented.

[0018] According to a fifth aspect of an embodiment of the present disclosure, a computer program product is provided, including a computer program, which, when executed by a processor, implements the steps of the artificial intelligence-based song generation method described in the first aspect.

[0019] The present disclosure allows users to select song types and themes according to their preferences, and preliminarily generates an initial music that meets the basic needs of the user, which can provide users with a relatively complete starting point. By determining the modified segments in the initial melody, the method can accurately find the parts that the user may want to modify, avoiding blind adjustments to the entire melody, thereby improving the efficiency of song generation. By allowing users to modify unsatisfactory segments by humming, the interactivity between users and the song generation process is enhanced. By replacing the melody and lyrics after the segment, the singing voice can be generated to obtain a complete music, which can improve the efficiency of song creation. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings required for use in the description of the embodiments of the present disclosure will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0021] Figure 1 is a schematic diagram of the steps of a song generation method based on artificial intelligence shown in an embodiment of the present disclosure; Figure 2 is a song generation flow chart shown in an embodiment of the present disclosure; Figure 3 is a block diagram of a song generation method based on artificial intelligence shown in an embodiment of the present disclosure; Figure 4 It is a schematic diagram of an electronic device shown in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0022] The following will be combined with the drawings in the embodiments of the present disclosure to clearly and completely describe the technical solutions in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, rather than all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0023] The terms "first", "second", etc. in the specification and claims of the present disclosure are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable when appropriate, so that the embodiments of the present disclosure can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second", etc. are generally of the same type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0024] Artificial intelligence and related technologies have made significant progress in the past few decades, and the music industry is no exception. Based on artificial intelligence, some music creation tools have been proposed in related technologies. Among them, some tools can automatically generate music, specifically based on a neural network model, through which various voices can be generated naturally, fluently, and with low latency, and can imitate intonation and accents. However, this tool is completely automatic in the music generation stage and cannot interact with the creator. There are also some tools that can capture the user's humming, generate accompaniment through the user's humming, and allow the user to adjust the pitch, speed and volume of the accompaniment. However, this tool cannot generate complete songs and singing.

[0025] In order to solve the above technical problems, the present disclosure proposes a song generation method, which can not only help creators to interactively generate songs, but also obtain complete songs by utilizing users' humming.

[0026] Figure 1 FIG. 1 is a schematic diagram of a method for generating songs based on artificial intelligence according to an embodiment of the present disclosure. Figure 1 As shown, the song generation method may specifically include the following steps: Step S11: Based on the song type and song theme input by the user, the artificial intelligence music model is called to obtain an initial music; the initial music includes lyrics and an initial melody.

[0027] The artificial intelligence music model can automatically generate a series of music that may meet the user's needs based on the input provided by the user.

[0028] In the process of obtaining the initial music using the artificial intelligence music model, the song type and song theme input by the user are first received. The song type may include pop, rock, classical, etc. The song theme may include love, inspiration, festival, etc.

[0029] Based on the song type and song theme input by the user, the artificial intelligence music model is called to generate a series of possible initial music. These initial music can be displayed through a candidate list. The user can view the candidate list and determine the initial music that meets the needs from the candidate list.

[0030] Any initial music piece in the candidate list has corresponding lyrics and initial melody.

[0031] Step S12: Determine the modified segment in the initial melody.

[0032] For the initial music selected by the user in the previous step, the user can determine whether to modify the initial melody of the initial music. The initial melody is composed of various structures, which may include an introduction, a chorus, a bridge, etc. The user can determine the segment that needs to be modified in the initial melody according to the structure of the initial melody and his own needs, and determine the segment that needs to be modified as the modified segment.

[0033] Step S13: receiving the user's humming voice for the modified segment.

[0034] The user's humming voice can be received through a microphone or other device. The received humming voice can be subjected to audio preprocessing, such as noise reduction, filtering, etc., to obtain the humming voice with higher audio quality.

[0035] Step S14: Based on the humming voice, call the artificial intelligence accompaniment model to generate an edited melody.

[0036] After receiving the user's humming voice, audio analysis is performed on the received humming voice to extract melody features, such as pitch, rhythm, etc. Based on the extracted melody features, the artificial intelligence accompaniment model is called to generate an edited melody corresponding to the humming voice according to the melody features.

[0037] Step S15: using the edited melody to replace the modified segment of the initial melody to obtain a target melody.

[0038] The modification of the modified segment is achieved by replacement. After the edited melody is obtained through the user's humming voice, the edited melody replaces the modified segment, and the edited melody replaces the position of the modified segment in the original initial music to obtain the target melody.

[0039] Step S16: Based on the target melody and the lyrics, call the artificial intelligence vocal model to generate singing vocals to obtain a complete piece of music.

[0040] According to the target melody and the lyrics, the artificial intelligence vocal model can be called to generate the corresponding singing vocals, which can imitate the intonation and accent, and finally obtain a complete song that meets the user's needs.

[0041] By adopting the embodiments of the present disclosure, an initial music that meets the user's needs is generated based on the song type and theme input by the user, providing the user with a personalized starting point for music creation. By modifying a specific segment of the melody by humming, the generated song is closer to the user's personal style and preferences. By guiding the modification of the melody by humming, the interactivity between the user and the music creation tool is enhanced. By automatically processing the user's humming voice and generating the edited melody, the workload of manually editing the melody is reduced, and the creation efficiency is improved.

[0042] The user's operation can be received through an interactive interface to implement each step of the song generation method proposed in the present disclosure. The user can perform a confirmation operation of modifying a segment in the interactive interface and start the operation of humming voice recording. After receiving each operation of the user, the corresponding steps of the subsequent song generation method are executed.

[0043] Among them, in an optional embodiment, it also includes: when the target melody does not meet the user's needs, return to the step of: receiving the user's humming voice for the modified segment; based on the humming voice, calling the artificial intelligence accompaniment model to generate an edited melody; using the edited melody, replacing the modified segment of the initial melody to obtain the target melody.

[0044] After obtaining the target melody, the user can play the target melody to determine whether it meets the user's expectations. If it meets the user's expectations, the target melody can be directly adopted and the next stage of operation can be performed. If it does not meet the user's expectations, the edited melody can be generated from scratch, specifically by re-receiving the user's humming voice for the modified segment, and then calling the artificial intelligence accompaniment model to generate the edited melody according to the humming voice, and according to the edited melody, replace the modified segment of the initial melody to obtain a new target melody.

[0045] The user can judge whether the target melody obtained each time meets the expectations. If it does not meet the expectations, the user's humming voice is received again until the target melody obtained according to the user's humming voice meets the user's expectations.

[0046] By adopting the embodiments of the present disclosure, users can directly hear and evaluate the generated target melody, and thus make judgments based on their preferences and expectations. If it does not meet expectations, through continuous iteration and adjustment, users can participate in the creation process of the melody, thereby improving the satisfaction and participation of the creation. Through multiple iterations, the details of the melody can also be gradually optimized to ensure that the quality of the melody finally generated meets the user's requirements. Users can interrupt the melody generation process at any time according to their needs, and make modifications and adjustments to make the melody generation process more controllable. Users can try different melody trends and styles during the iteration process to inspire creative inspiration.

[0047] Among them, in an optional embodiment, after obtaining the target melody, it also includes: adjusting the chords of the target melody as a whole by adjusting the generation parameters in the artificial intelligence chord model; the generation parameters include rhythm, pitch, coordination and sound quality.

[0048] After obtaining the target melody, the chords used in the target melody can be identified. If the currently used chords do not meet the user's needs, the chords used in the target melody can be adjusted as a whole. According to the style of the music and the needs of the user, a suitable large model is selected, and the chords of the target melody are adjusted by adjusting the generation parameters of the chords in the artificial intelligence chord large model, such as the rhythm of the chords, the tone of the chords, and the sound quality of the chords.

[0049] By using the embodiments of the present disclosure, creators can adjust generation parameters such as rhythm, pitch, and sound quality in real time, quickly obtain different versions of chords and accompaniment, and improve creation efficiency. According to the needs of the creator, the generation parameters of the chords are adjusted to achieve personalized music creation. By quickly generating chords and accompaniment, creators can more easily find inspiration and explore different musical directions and styles. Through automated chord generation and instrument coordination sound generation, the technical threshold for music creation is lowered, allowing more non-professional creators to participate in music creation.

[0050] Among them, in an optional embodiment, according to the target melody and the lyrics, an artificial intelligence vocal model is called to generate singing vocals to obtain a complete piece of music, including: deleting or adding each segment in the target melody to obtain the adjusted target melody; modifying the lyrics to obtain the adjusted lyrics; according to the adjusted target melody and the adjusted lyrics, a target piece of music is obtained; according to the target piece of music, the artificial intelligence vocal model is called to generate singing vocals, and singing vocals are generated to obtain a complete piece of music.

[0051] The structure of the target melody may be analyzed, and the structure of the target melody includes a chorus, a bridge, etc. The target melody segment may be deleted or added according to the structure of the target melody, for example, a certain chorus segment may be deleted. The added segment may be an existing segment of the target melody, or a new segment generated by the user humming. After the segment of the target melody is adjusted, the adjusted target melody is obtained.

[0052] The lyrics corresponding to the initial music can be modified, and new lyrics input by the user can be received to obtain adjusted lyrics.

[0053] According to the adjusted target melody and the adjusted lyrics, when calling the artificial intelligence vocal model to generate singing vocals, the generation parameters of the vocals in the artificial intelligence vocal model, such as pitch, timbre, and singing style, can be adjusted so that the model generates vocals that meet expectations.

[0054] The generated vocals are synthesized with the accompaniment to form a complete sound track and obtain a complete piece of music.

[0055] By using the embodiments of the present disclosure, creators can modify the generated songs instantly, add or delete lyrics, music segments, etc., which improves the flexibility and freedom of creation. It supports batch addition, deletion and modification of multiple lyrics paragraphs or music segments, further improving editing efficiency. By adding or deleting specific lyrics and music segments, creators can personalize the song content according to their own style and needs, and enhance the uniqueness of the work.

[0056] Among them, in an optional embodiment, according to the target music, the artificial intelligence vocal model is called to generate singing vocals, and the singing vocals are generated to obtain a complete music, including: according to the singing effect of the singing vocals, the music section, lyrics, rhythm and pitch of the target music are re-adjusted to obtain the adjusted target music; according to the adjusted target music, the artificial intelligence vocal model is called to generate singing vocals, and the singing vocals are regenerated to obtain a complete music.

[0057] The user can listen to the generated vocals and accompaniment in real time to determine whether it meets expectations. If not, the music section and lyrics can be readjusted, and the singing vocals can be generated from the readjusted music section and lyrics to obtain a complete music piece.

[0058] With the embodiments of the present disclosure, during the editing process, the creator can hear the modified effect in real time, quickly determine whether the modification meets expectations, and make further adjustments. During the editing process, the creator can use different combinations of lyrics and music segments to inspire new creative inspiration and explore more musical possibilities.

[0059] Among them, in an optional embodiment, the initial melody includes a main melody and an accompaniment, and determining a modified segment in the initial melody includes: determining the main melody corresponding to the modified segment, and / or determining the accompaniment corresponding to the modified segment; calling an artificial intelligence accompaniment large model to generate an edited melody according to the humming voice, including: generating a humming melody according to the humming voice; the humming melody is used to: replace the main melody corresponding to the modified segment, and / or replace the accompaniment corresponding to the modified segment; determine the retained part of the modified segment that is not replaced by the humming melody; adjust the notes and beats of the humming melody, and combine with the retained part to obtain the edited melody.

[0060] The melody in the original music includes the main melody and the accompaniment. The main melody is the melody sung by the lead singer or played by an instrument, and the accompaniment is the music part that supports the main melody, usually composed of harmony, rhythm instruments and other instruments.

[0061] Audio processing technology can be used to separate the main melody and accompaniment in the initial melody. The modified segment in the initial melody that needs to be modified can be determined according to user needs. The modified segment also includes the main melody and accompaniment. The user determines which component of the modified segment to modify, or the entire modified segment to modify. Specifically, the user can modify only the main melody or the accompaniment, or modify the main melody and the accompaniment at the same time.

[0062] In the process of calling the artificial intelligence accompaniment model to generate and edit the melody according to the user's humming voice, the retained part is first determined based on the modification content previously determined by the user. Specifically, if the user only modifies the main melody, then the retained part is the accompaniment. If both the main melody and the accompaniment need to be modified, then there is no retained part.

[0063] Based on the content to be modified in the modified segment, the user's humming voice is received to form a humming melody. Specifically, when the main melody needs to be modified, the generated humming melody is the new main melody. The notes and beats of the humming melody are adjusted to meet the user's needs, and the adjusted humming melody is combined with the retained part to obtain the edited melody.

[0064] By using the embodiments of the present disclosure, the main melody and the accompaniment part in the initial melody are separated, so that the user can more accurately determine the modification object, whether it is the main melody or the accompaniment part. The user provides new melody materials through humming, which can provide the user with an intuitive and personalized means of creation, and lower the threshold of music creation for the user. The generated humming melody can be used to combine with the retained part of the modified segment to obtain the edited melody, so as to realize personalized adjustment of the initial melody according to the user's humming.

[0065] In an optional embodiment, the use of the edited melody to replace the modified segment of the initial melody to obtain a target melody includes: determining a portion of the initial melody located before the modified segment in the initial music as a first melody, and determining a portion of the initial melody located after the modified segment in the initial music as a second melody; obtaining a humming segment adjustment algorithm; adjusting the rhythm of the edited melody according to the first melody and the second melody using a portion of the humming segment adjustment algorithm used for rhythm adjustment to obtain an edited melody after the rhythm is adjusted; adjusting the pitch of the edited melody after the rhythm is adjusted according to the first melody and the second melody using a portion of the humming segment adjustment algorithm used for pitch adjustment to obtain an edited melody after the rhythm and pitch are adjusted; replacing the modified segment of the initial melody with the edited melody after the rhythm and pitch are adjusted to obtain a target melody.

[0066] The modified segment is located by finding the start and end positions of the modified segment in the initial melody. The initial melody corresponding to the initial music piece is segmented, and the initial melody located before the start position of the modified segment is determined as the first melody, and the melody located after the end position of the modified segment is determined as the second melody.

[0067] The humming segment adjustment algorithm can adjust the rhythm and pitch of the edited melody based on the first melody and the second melody. First, the rhythm of the edited melody is adjusted by the humming segment adjustment algorithm, and then the pitch of the edited melody is adjusted based on the rhythm adjustment.

[0068] Based on the first melody and the second melody, the rhythm of the edited melody is automatically adjusted so that the rhythm of the edited melody matches the rhythm of the first melody and the second melody. Specifically, the rhythmic continuity can be ensured by adjusting the beat position, speed change, or adding / deleting rhythm elements of the edited melody. After the rhythm of the edited melody is adjusted, the edited melody with adjusted rhythm is obtained.

[0069] After obtaining the edited melody after the rhythm adjustment, the pitch is further automatically adjusted based on the first melody and the second melody. The pitch trend of the first melody and the second melody can be analyzed, such as the scale, interval relationship, the ups and downs of the melody line, etc. The pitch of the edited melody after the rhythm adjustment is adjusted so that the edited melody after the rhythm adjustment matches the pitch of the first melody and the second melody. Specifically, the pitch of the notes in the edited melody can be adjusted to maintain the smooth transition of the melody line and the rationality of the interval relationship.

[0070] The target melody is obtained by replacing the modified segment in the initial music with the edited melody after adjusting the rhythm and pitch. The length of the edited melody and the modified segment may not match, and the corresponding part can be cut out from the edited melody according to the length and rhythm of the modified segment, so that the edited melody and the modified segment are completely matched in length; according to the length and rhythm of the modified segment, the trimmed edited melody replaces the position of the modified segment in the initial melody.

[0071] By adopting the embodiments of the present disclosure, by respectively determining the partial melodies before and after the modified segment in the initial music as the first melody and the second melody, a clear boundary is provided for the subsequent editing work, making the editing process more orderly and controllable. When adjusting the edited melody, the first melody and the second melody are referred to ensure that the edited melody is consistent with the initial melody, thereby maintaining the overall coherence and fluency of the melody. After completing the adjustment of the rhythm and pitch, the edited melody is seamlessly replaced with the modified segment of the initial melody, avoiding abrupt transitions or a sense of discontinuity. The modification process of the modified segment provides great flexibility and diversity for the creator, and by adjusting the edited melody to varying degrees, a rich variety of musical works can be created.

[0072] Among them, in an optional embodiment, according to the first melody and the second melody, the rhythm of the edited melody is adjusted using the part of the humming segment adjustment algorithm used for rhythm adjustment to obtain the edited melody with adjusted rhythm, including: respectively determining the original beat points corresponding to the first melody and the second melody; determining the beat points of the edited melody by identifying the beat pattern of the edited melody; and adjusting the beat points of the edited melody to align with the original beat points to obtain the edited melody with adjusted rhythm.

[0073] When the rhythm of the edited melody is matched with the first melody and the second melody, the rhythm of the edited melody needs to be modified based on the first melody and the second melody.

[0074] The original beat points of the first melody and the second melody can be determined respectively. The beat type and tempo corresponding to the first melody and the second melody can be determined by the original beat points. The beat type includes four-quarter beat, three-quarter beat, etc., and the tempo can be represented by the number of beats per minute.

[0075] After the original beat points of the first melody and the second melody are determined respectively, the beat points of the first melody and the second melody can be aligned to find the common rhythm pattern or beat grid of the first melody and the second melody. If the rhythms of the first melody and the second melody are significantly different, the rhythm of one of the melodies can be selected as a basis for adjusting the beat of the edited melody.

[0076] Identify the beat points in the edited melody and analyze the beat pattern of the edited melody, including beat type, tempo, and rhythm changes. Extract key beat features in the edited melody, such as stress position, rhythm density, etc.

[0077] The beat points of the edited melody are compared with the original beat points of the first melody and the second melody. The time offset that needs to be adjusted for each beat point in the edited melody can be determined so that it is aligned with the original beat point. According to the calculated time offset, the edited melody is time-stretched or compressed.

[0078] After the edited melody is time-stretched or compressed, the edited melody after the rhythm adjustment can be smoothed to eliminate possible rhythmic mutations or unnaturalness.

[0079] By adopting the embodiments of the present disclosure, by respectively determining the original beat points of the first melody and the second melody, the rhythmic benchmarks of the two melodies can be clarified, and the beat points of the edited melody can be adjusted to align with these original beat points, so as to ensure that the edited melody is rhythmically consistent with the first melody and the second melody. By automatically adjusting the beat of the edited melody with the first melody and the second melody, the edited melody is rhythmically matched with the first melody and the second melody, which can simplify the rhythm adjustment steps and improve work efficiency.

[0080] Among them, in an optional embodiment, according to the first melody and the second melody, the pitch of the edited melody after the rhythm is adjusted is adjusted using the part used for pitch adjustment in the humming segment adjustment algorithm to obtain the edited melody after the rhythm and pitch are adjusted, including: determining the key notes corresponding to each beat point in the edited melody after the rhythm is adjusted; determining any of the key notes as the target key notes; determining the first initial note corresponding to the target key note from the first melody and determining the second initial note corresponding to the target key note from the second melody; and And the second initial note is determined as the triplet mapping corresponding to the target key note; the target key note and the next key note adjacent to the target key note are determined as the binary mapping corresponding to the target key note; the pitch of the edited melody is adjusted by the triplet mapping and binary mapping corresponding to each key note, so as to obtain the edited melody with adjusted rhythm and pitch; wherein each of the triplet mappings represents: the pitch difference between the edited melody with adjusted rhythm and the initial melody; each of the binary mappings represents: the pitch difference between each of the key notes within the edited melody with adjusted rhythm.

[0081] Determine each beat point in the edited melody after the rhythm is adjusted. Find the corresponding notes at each beat point, which are the key notes. The key notes can be automatically identified or manually marked by the user.

[0082] It is necessary to determine the triplet mapping and the bigram mapping corresponding to each key note. The pitch of the edited melody is adjusted by the triplet mapping and the bigram mapping corresponding to each key note to match the pitch of the first melody and the second melody.

[0083] The triplet mapping and the doublet mapping corresponding to any key note are determined as follows: This key note can be determined as the target key note. In the first melody, find the note at the same beat point or a similar beat point as the target key note as the first initial note. Similarly, find the corresponding note in the second melody as the second initial note. If there is no directly corresponding note in the first melody or the second melody, the corresponding first initial note or second initial note can be estimated by interpolation or averaging. Create a triple data structure to store the target key note, the first initial note, and the second initial note. For example, the triple mapping can be recorded as:<Bi,Ni,Fi> , corresponding to <first initial note, target key note, second initial note> respectively. The target key note and the next key note adjacent to it can be determined as a two-tuple mapping. Similarly, a two-tuple mapping can also be created to store the target key note and the next key note. For example, the two-tuple mapping can be recorded as:<Ki,Ki+1> , corresponding to <target key note, the key note after the target key note> respectively.

[0084] By the above method, the triplet mapping and the binary mapping corresponding to each key note in each edited melody are determined. Each triplet mapping reflects the difference in pitch between the edited melody and the initial melody. Each binary mapping reflects the difference in pitch between adjacent key notes in the edited melody.

[0085] Each triplet mapping and each binary mapping are used as the basis for adjusting the pitch of the edited melody. All key notes in the edited melody can be traversed, and for each key note, its pitch is adjusted to be close to the first initial note or the second initial note according to its triplet mapping, and at the same time, according to its binary mapping, the pitch difference between adjacent key notes is ensured to be kept within a reasonable range.

[0086] By using the embodiments of the present disclosure, by determining the key notes corresponding to each beat point in the edited melody after the rhythm is adjusted, the key notes for the melody contour and the key notes for expressing emotions in the edited melody can be determined. Each key note is processed separately to obtain the corresponding triplet mapping and binary mapping, ensuring that every part of the melody can be carefully adjusted and optimized. By establishing the triplet mapping and binary mapping corresponding to the target key notes, the consistency and naturalness of the edited melody in pitch can be ensured, the pitch of the first melody and the second melody can be effectively matched, and a new musical experience can be created on the basis of maintaining the characteristics of the initial melody.

[0087] Among them, in an optional embodiment, after obtaining the edited melody with adjusted rhythm and pitch, it also includes: fine-tuning the edited melody with adjusted rhythm and pitch to obtain the fine-tuned edited melody; using the edited melody with adjusted rhythm and pitch to replace the modified segment of the initial melody to obtain the target melody, including: using the fine-tuned edited melody to replace the modified segment of the initial melody to obtain the target melody.

[0088] After getting the edited melody with adjusted rhythm and pitch, the user can check the edited melody by playing it to confirm whether there are any dissonant, abrupt or unexpected parts, and can make fine adjustments to the dissonant, abrupt or unexpected parts. For slight deviations in rhythm, such as too long or too short note values, use the time stretching or compression function in the editing tool to make precise adjustments; for slight deviations in pitch, use the pitch adjustment tool to fine-tune each note in the melody to ensure the accuracy of the pitch.

[0089] The process of fine-tuning the edited melody can be a manual operation by the user, and the user can accurately determine the rhythm and pitch that do not meet the expectations in the edited melody.

[0090] After fine-tuning, users can also repeatedly audition the edited melody to ensure that the rhythm and pitch are optimal. If you find that there are still unsatisfactory aspects, continue to adjust until the melody is exactly as expected.

[0091] In the original melody, accurately find the starting and ending positions of the modified segment that needs to be replaced, and use the edited melody to replace the position of the modified segment in the original melody.

[0092] With the embodiments of the present disclosure, the fine-tuning process allows users to finely control each note and each rhythm point in the melody, thereby enhancing the delicacy and emotional depth of musical expression. The modified fragment of the initial melody is replaced with the edited melody after rhythm and pitch adjustment and fine-tuning, which helps to ensure that the melody is highly consistent with the overall musical work in style, rhythm and tonality. Allowing users to fine-tune the melody provides more innovative space and experimental opportunities for music creation.

[0093] Figure 2 is a song generation flow chart shown in an embodiment of the present disclosure. Figure 2 As shown, first the user sets the song type and song theme; a series of candidate initial music songs are generated according to the song type and song theme through the artificial intelligence music big model; one is selected from the candidate initial music songs and used as the initial music song for further editing; the user can judge whether the initial music song meets expectations; if the initial music song meets expectations, the initial melody corresponding to the initial music song can be determined as the target melody, and the chord adjustment stage is directly entered, and the overall chord of the initial music song is adjusted through the artificial intelligence chord big model; if the initial music song does not meet expectations, the modified segment that needs to be modified is determined, the user's humming voice is received, and the artificial intelligence accompaniment big model is used to convert the humming voice into an edited melody. After obtaining the edited melody, the user can judge whether the edited melody meets expectations; if the edited melody does not meet expectations, the user's humming voice can be received again, and a new edited melody can be obtained according to the humming voice; if the edited melody meets expectations, the edited melody replaces the modified segment to obtain the target melody; the target melody is adjusted by the artificial intelligence chord big model; for the target melody after the chord adjustment, the artificial intelligence vocal big model is used to generate the singing vocals to obtain a complete music song.

[0094] Among them, in an optional embodiment, when determining the modified segment in the initial melody, the structure, emotional trend and melody characteristics of the initial melody can be automatically analyzed; and the emotional color contained in the melody, such as cheerfulness, sadness, excitement, etc., can be identified. According to the user's historical editing records, preference feedback and other information, the user's editing preferences and style are learned; on this basis, segments that the user may want to modify are intelligently recommended.

[0095] Figure 3 is a block diagram of a song generation device shown in an embodiment of the present disclosure. Figure 3 As shown, the song generating device comprises: The acquisition module 310 is used to call the artificial intelligence music model to obtain an initial music according to the song type and song theme input by the user; the initial music includes lyrics and an initial melody; A determination module 320, configured to determine a modified segment in the initial melody; The receiving module 330 is used to receive the humming voice of the user for the modified segment; An editing module 340 is used to call an artificial intelligence accompaniment model to generate an edited melody according to the humming voice; A replacement module 350, configured to replace the modified segment of the initial melody with the edited melody to obtain a target melody; The completion module 360 ​​is used to call the artificial intelligence vocal model to generate singing vocals according to the target melody and the lyrics to obtain a complete piece of music.

[0096] Optionally, the editing module is specifically used to execute: Determine a portion of the initial melody located before the modified segment in the initial music as a first melody, and determine a portion of the initial melody located after the modified segment in the initial music as a second melody; Get the humming segment adjustment algorithm; According to the first melody and the second melody, the rhythm of the edited melody is adjusted using the part of the humming segment adjustment algorithm used for rhythm adjustment to obtain an edited melody with adjusted rhythm; According to the first melody and the second melody, the pitch of the edited melody after the rhythm is adjusted is adjusted using the part used for pitch adjustment in the humming segment adjustment algorithm to obtain the edited melody after the rhythm and pitch are adjusted; The edited melody after the rhythm and pitch are adjusted is used to replace the modified segment of the initial melody to obtain the target melody.

[0097] Optionally, the song generating device further includes: A fine-tuning module, used for fine-tuning the edited melody after the rhythm and pitch are adjusted to obtain the edited melody after fine-tuning; The editing module is specifically used to execute: The modified segment of the initial melody is replaced by the edited melody after fine-tuning to obtain the target melody.

[0098] Optionally, the song generating device further includes: The return module is used to return to the steps of: receiving the user's humming voice for the modified segment; calling the artificial intelligence accompaniment model to generate an edited melody according to the humming voice; and using the edited melody to replace the modified segment of the initial melody to obtain the target melody.

[0099] Optionally, the initial melody includes a main melody and an accompaniment, and the determination module is specifically used to execute: Determine the main melody corresponding to the modified segment, and / or determine the accompaniment corresponding to the modified segment; The editing module is specifically used to execute: Generate a humming melody according to the humming voice; the humming melody is used to: replace the main melody corresponding to the modified segment, and / or replace the accompaniment corresponding to the modified segment; Determining a retained portion of the modified segment that is not replaced by the humming melody; The notes and beats of the humming melody are adjusted and combined with the retained parts to obtain an edited melody.

[0100] Optionally, the editing module is specifically used to execute: respectively determining original beat points corresponding to the first melody and the second melody; Determining the beat points of the edited melody by identifying the beat pattern of the edited melody; The beat points of the edited melody are adjusted to be aligned with the original beat points to obtain the edited melody with adjusted rhythm.

[0101] Optionally, the editing module is specifically used to execute: Determining key notes corresponding to each beat point in the edited melody after the rhythm adjustment; determining any of the key notes as a target key note; Determining a first initial note corresponding to the target key note from the first melody and determining a second initial note corresponding to the target key note from the second melody; Determine the target key note, the first initial note, and the second initial note as a triplet mapping corresponding to the target key note; Determine the target key note and a subsequent key note adjacent to the target key note as a binary mapping corresponding to the target key note; The pitch of the edited melody is adjusted by using the triplet mapping and the binary mapping corresponding to each key note to obtain the edited melody with adjusted rhythm and pitch; Wherein, each of the triplet mappings represents: the pitch difference between the edited melody after the rhythm adjustment and the initial melody; each of the binary mappings represents: the pitch difference between each of the key notes within the edited melody after the rhythm adjustment.

[0102] Optionally, the song generating device further includes: The chord module is used to adjust the chords of the target melody as a whole by adjusting the generation parameters in the artificial intelligence chord model; the generation parameters include rhythm, pitch, coordination and sound quality.

[0103] Optionally, the completion module is specifically used to execute: Deleting or adding each segment in the target melody to obtain an adjusted target melody; Modifying the lyrics to obtain adjusted lyrics; Obtaining a target music piece according to the adjusted target melody and the adjusted lyrics; According to the target music, the artificial intelligence vocal model is called to generate the singing vocals to obtain the complete music.

[0104] Optionally, the completion module is specifically used to execute: According to the singing effect of the vocals, the music section, lyrics, rhythm and pitch of the target music are readjusted to obtain an adjusted target music; According to the adjusted target music, the artificial intelligence vocal model is called to generate the singing vocals, and the singing vocals are regenerated to obtain the complete music.

[0105] It should be noted that the device embodiment is similar to the method embodiment, so the description is relatively simple, and the relevant parts can be referred to the method embodiment.

[0106] The present disclosure also provides an electronic device, referring to Figure 4 , Figure 4 is a schematic diagram of an electronic device shown in an embodiment of the present disclosure. Figure 4 As shown, the electronic device 400 includes: a memory 410 and a processor 420. The memory 410 and the processor 420 are connected via a bus communication. A computer program is stored in the memory 410. The computer program can be run on the processor 420 to implement the steps in the song generation method based on artificial intelligence in the embodiment of the present disclosure.

[0107] The embodiment of the present disclosure also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps in the song generation method based on artificial intelligence disclosed in the embodiment of the present disclosure are implemented.

[0108] The embodiments of the present disclosure also provide a computer program product, including a computer program, which, when executed by a processor, implements the steps in the artificial intelligence-based song generation method disclosed in the embodiments of the present disclosure.

[0109] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0110] It should be understood by those skilled in the art that the embodiments of the present disclosure may be provided as methods, devices or computer program products. Therefore, the embodiments of the present disclosure may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments of the present disclosure may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0111] The embodiments of the present disclosure are described with reference to the flowcharts and / or block diagrams of the methods, apparatuses, electronic devices, and computer program products according to the embodiments of the present disclosure. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0112] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0113] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0114] Although some embodiments of the present disclosure have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiment and all changes and modifications that fall within the scope of the present disclosure.

[0115] The above is a detailed introduction to the song generation method based on artificial intelligence provided by the present disclosure. Specific examples are used in this article to illustrate the principles and implementation methods of the present disclosure. The description of the above embodiments is only used to help understand the method of the present disclosure and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present disclosure, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present disclosure.

Claims

1. A song generation method based on artificial intelligence, characterized in that: include: According to the song type and song theme input by the user, the artificial intelligence music model is called to obtain the initial music; The initial music includes lyrics and an initial melody; determining a modified segment of the initial melody; receiving a humming voice of a user for the modified segment; According to the humming voice, calling the artificial intelligence accompaniment model to generate an edited melody; Using the edited melody to replace the modified segment of the initial melody to obtain a target melody; According to the target melody and the lyrics, an artificial intelligence vocal model is called to generate singing vocals to obtain a complete piece of music.

2. The method according to claim 1, characterized in that Using the edited melody to replace the modified segment of the initial melody to obtain a target melody comprises: Determine a portion of the initial melody located before the modified segment in the initial music as a first melody, and determine a portion of the initial melody located after the modified segment in the initial music as a second melody; Get the humming segment adjustment algorithm; According to the first melody and the second melody, the rhythm of the edited melody is adjusted using the part of the humming segment adjustment algorithm used for rhythm adjustment to obtain an edited melody with adjusted rhythm; According to the first melody and the second melody, the pitch of the edited melody after the rhythm is adjusted is adjusted using the part used for pitch adjustment in the humming segment adjustment algorithm to obtain the edited melody after the rhythm and pitch are adjusted; The edited melody after the rhythm and pitch are adjusted is used to replace the modified segment of the initial melody to obtain the target melody.

3. The method according to claim 2, characterized in that After getting the edited melody with adjusted rhythm and pitch, it also includes: Fine-tuning the edited melody after the rhythm and pitch are adjusted to obtain the fine-tuned edited melody; The step of replacing the modified segment of the initial melody with the edited melody after the rhythm and pitch are adjusted to obtain the target melody comprises: The modified segment of the initial melody is replaced by the edited melody after fine-tuning to obtain the target melody.

4. The method according to claim 1, characterized in that Also includes: In the case that the target melody does not meet the user's needs, returning to the step of: receiving the user's humming voice for the modified segment; According to the humming voice, an artificial intelligence accompaniment model is called to generate an edited melody; the edited melody is used to replace the modified segment of the initial melody to obtain a target melody.

5. The method according to claim 1, characterized in that The initial melody includes a main melody and an accompaniment, and determining a modified segment in the initial melody includes: Determine the main melody corresponding to the modified segment, and / or determine the accompaniment corresponding to the modified segment; According to the humming voice, the artificial intelligence accompaniment model is called to generate an edited melody, including: Generate a humming melody according to the humming voice; the humming melody is used to: replace the main melody corresponding to the modified segment, and / or replace the accompaniment corresponding to the modified segment; Determining a retained portion of the modified segment that is not replaced by the humming melody; The notes and beats of the humming melody are adjusted and combined with the retained parts to obtain an edited melody.

6. The method according to claim 2, characterized in that According to the first melody and the second melody, the rhythm of the edited melody is adjusted using the part of the humming segment adjustment algorithm used for rhythm adjustment to obtain the edited melody after the rhythm is adjusted, including: respectively determining original beat points corresponding to the first melody and the second melody; Determining the beat points of the edited melody by identifying the beat pattern of the edited melody; The beat points of the edited melody are adjusted to be aligned with the original beat points to obtain the edited melody with adjusted rhythm.

7. The method according to claim 2, characterized in that According to the first melody and the second melody, the pitch of the edited melody after the rhythm adjustment is adjusted using the part used for pitch adjustment in the humming segment adjustment algorithm to obtain the edited melody after the rhythm and pitch are adjusted, comprising: Determining key notes corresponding to each beat point in the edited melody after the rhythm adjustment; determining any of the key notes as a target key note; Determining a first initial note corresponding to the target key note from the first melody and determining a second initial note corresponding to the target key note from the second melody; Determine the target key note, the first initial note, and the second initial note as a triplet mapping corresponding to the target key note; Determine the target key note and a subsequent key note adjacent to the target key note as a binary mapping corresponding to the target key note; The pitch of the edited melody is adjusted by using the triplet mapping and the binary mapping corresponding to each key note to obtain the edited melody with adjusted rhythm and pitch; Wherein, each of the triplet mappings represents: the pitch difference between the edited melody after the rhythm adjustment and the initial melody; each of the binary mappings represents: the pitch difference between each of the key notes within the edited melody after the rhythm adjustment.

8. The method according to claim 1, characterized in that After getting the target melody, it also includes: By adjusting the generation parameters in the artificial intelligence chord model, the chords of the target melody are adjusted as a whole; the generation parameters include rhythm, pitch, coordination and sound quality.

9. The method according to claim 1, characterized in that: According to the target melody and the lyrics, the artificial intelligence vocal model is called to generate the singing vocals to obtain a complete piece of music, including: Deleting or adding each segment in the target melody to obtain an adjusted target melody; Modifying the lyrics to obtain adjusted lyrics; Obtaining a target music piece according to the adjusted target melody and the adjusted lyrics; According to the target music, the artificial intelligence vocal model is called to generate the singing vocals, and the singing vocals are generated to obtain the complete music.

10. The method according to claim 9, characterized in that According to the target music, the artificial intelligence vocal model is called to generate singing vocals, and the singing vocals are generated to obtain a complete music, including: According to the singing effect of the vocals, the music section, lyrics, rhythm and pitch of the target music are readjusted to obtain an adjusted target music; According to the adjusted target music, the artificial intelligence vocal model is called to generate the singing vocals, and the singing vocals are regenerated to obtain the complete music.

Images