SONG CREATION METHOD, DEVICE, SYSTEM AND STORAGE MEDIUM

Abstract

The present disclosure relates to a method, device, system and storage medium for song creation, which obtains a target lyrics text input by a user, aligns the target lyrics text with the singing melody of an initial song, determines the correspondence between the text units in the target lyrics text and the notes in the singing melody, and synthesizes the target lyrics text based on the correspondence between the text units in the target lyrics text and the notes in the singing melody to obtain a singing voice that recites the target lyrics text with the singing melody, and further integrates the singing voice with the accompaniment audio of the initial song to generate a target song, which essentially allows the user to write lyrics by himself, and create a song based on the lyrics, singing melody and accompaniment audio created by the user, to form a completely new song. In this way, the user can create a song based on the lyrics he created, even if he does not have professional skills in creating music, and improves the efficiency and interest of the user in song creation.

Description

[Technical field]

[0001] [CROSS REFERENCE TO RELATED APPLICATIONS] This application is based on and claims priority to an application having Chinese application number 202210494217.7, filing date May 7, 2022, and title "Song generation method, apparatus, system and storage medium," the entire disclosure of which is incorporated herein by reference.

[0002] The present disclosure relates to the field of artificial intelligence technology, and in particular to a song generation method, device, system, and storage medium. [Background technology]

[0003] Creating songs requires a certain level of specialized skills. For general users who lack specialized skills, it is extremely difficult. For this reason, how to help composers create songs efficiently has become an urgent issue. Summary of the Invention

[0004] To solve the above technical problems, the present disclosure provides a song generation method, device, system, and storage medium.

[0005] According to a first aspect, the present disclosure provides a method for producing a method for detecting a pulmonary circulation disorder, comprising: Obtaining a target lyric text entered by a user; aligning the target lyrics text with a singing melody of an initial song to determine a correspondence between text units in the target lyrics text and notes in the singing melody, the singing melody being a singing melody for initial lyrics in the initial song; synthesizing the target lyrics text based on a correspondence between text units in the target lyrics text and notes in the singing melody to obtain a singing voice for singing the target lyrics text with the singing melody; and integrating the singing voice with an audio accompaniment of the initial song to generate a target song.

[0006] In some embodiments, prior to aligning the target lyric text with the vocal melody of the initial song as described above, the method further comprises: selecting an initial song from a plurality of preset songs in response to an initial song selection operation; and determining a corresponding singing melody and accompaniment audio based on the initial song.

[0007] In some embodiments, aligning the target lyric text with a vocal melody of the initial song includes: Dividing the singing melody into a plurality of melody paragraphs; Dividing the target lyric text into a plurality of lyric paragraphs, the number of the plurality of lyric paragraphs being equal to the number of the plurality of melody paragraphs; and aligning the plurality of lyric paragraphs and the plurality of melody paragraphs one by one, and determining a correspondence between a text unit in the lyric paragraph and a note in the corresponding melody paragraph.

[0008] In some embodiments, dividing the vocal melody into a plurality of melody paragraphs comprises: determining a paragraph division point for each predetermined measure in the singing melody; adjusting the number of paragraph division points based on the number of notes included in the melody paragraph corresponding to each paragraph division point; Adjusting the position of each paragraph division point based on a note attack distance, the note attack distance including a duration pitch of a note attack and / or a pitch of a note attack.

[0009] In some embodiments, adjusting the number of paragraph division points based on the number of notes included in the melody paragraph corresponding to each paragraph division point comprises: When the number of notes included in the melody paragraph corresponding to any one of the paragraph division points is smaller than a first threshold value, deleting the any one of the paragraph division points; When the number of notes included in the melody paragraph corresponding to any one of the paragraph division points is greater than a second threshold value, the paragraph division point is increased by one.

[0010] In some embodiments, adjusting the position of each paragraph division point based on the note attack distance comprises: searching for a position where a note attack distance satisfies a predetermined condition within a predetermined beat range around any one of the paragraph division points, and determining the position as the position of the any one of the paragraph division points; The specified condition is that the duration pitch of the note attack before and after any one of the paragraph division points is the largest, or that the duration pitch of the note attack before and after any one of the paragraph division points is equal and the pitch is the largest.

[0011] In some embodiments, splitting the target lyric text into a plurality of lyric paragraphs comprises: segmenting the target lyrics text into words and determining a part of speech corresponding to each word; and dividing the target lyric text into a plurality of lyric paragraphs based on the part of speech corresponding to each word, predefined linguistic rules, and the length of the vocal melody. In some embodiments, aligning the plurality of lyric paragraphs with the plurality of melody paragraphs one by one comprises: For each of the melody paragraphs, obtain a plurality of lyric alignment templates corresponding to the predetermined melody paragraphs, each of the lyric alignment templates corresponding to a different number of lyric characters; selecting a target lyric alignment template from the plurality of lyric alignment templates, the number of characters corresponding to the target lyric alignment template being the number of characters of a lyric paragraph corresponding to the melody paragraph; and aligning the melody paragraph with a lyric paragraph corresponding to the melody paragraph based on the target lyric alignment template.

[0012] In some embodiments, the plurality of lyric alignment templates corresponding to the melody paragraph include a first lyric alignment template, a second lyric alignment template and a third lyric alignment template; In the first lyric alignment template, each note in the melody paragraph corresponds to one text unit; In the second lyrics alignment template, adjacent notes in the melody paragraph that have the closest note attack distance are merged into one note pair, and one note pair corresponds to one text unit; In the third lyric alignment template, adjacent notes in the second lyric alignment template that have the closest note attack distance are merged into one note pair, and one note pair corresponds to one text unit.

[0013] According to a second aspect, the present disclosure provides a method for producing a method for detecting a pulmonary circulation, comprising: an acquisition unit for acquiring a target lyric text input by a user; an alignment unit for aligning the target lyrics text with a singing melody of an initial song to determine a correspondence between a text unit in the target lyrics text and a note in the singing melody, the singing melody being a singing melody for an initial lyrics of the initial song; a synthesis unit for synthesizing the target lyrics text based on a correspondence between text units in the target lyrics text and notes in the singing melody to obtain a singing voice for singing the target lyrics text with the singing melody; A song generating device is further proposed, comprising a generating unit for integrating said singing voice and an accompaniment audio of said initial song to generate a target song.

[0014] According to a third aspect, the present disclosure further provides a system including at least one computing device and at least one storage device storing instructions that, when executed by the at least one computing device, cause the at least one computing device to perform steps of the song generation method described above.

[0015] According to a fourth aspect, the present disclosure further provides a computer-readable storage medium storing a program or instructions which, when executed by at least one computing device, cause the at least one computing device to perform the steps of the song generation method described above. [Brief description of the drawings]

[0016] The drawings herein are incorporated in and constitute a part of the specification, show embodiments conforming to the present disclosure, and are used together with the specification to explain the principles of the present disclosure. In order to more clearly explain the technical solutions in the embodiments or prior art of the present disclosure, the following briefly describes the drawings that need to be used in the description of the embodiments or prior art. Obviously, a person skilled in the art can obtain other drawings based on these drawings without creative efforts. [Figure 1] 1 is a flowchart of a song generation method according to an embodiment of the present disclosure. [Diagram 2] 1 is a flowchart of another song generation method according to an embodiment of the present disclosure. [Diagram 3] FIG. 2 is a schematic diagram of a musical staff for a melody according to an embodiment of the present disclosure. [Figure 4] FIG. 2 is a schematic diagram of a musical staff of one melody paragraph according to an embodiment of the present disclosure. [Diagram 5] FIG. 2 is a schematic diagram of a display interface for realizing generating a song based on a target lyric text in a terminal according to an embodiment of the present disclosure; [Figure 6] FIG. 1 is a schematic diagram of a configuration of a song generating device according to an embodiment of the present disclosure. [Figure 7]1 is an exemplary block diagram of a system including at least one computing device and at least one memory device that stores instructions, according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0017] In order to make the above objects, features and advantages of the present disclosure more clearly understandable, the following further describes the aspects of the present disclosure. It should be noted that, unless contradictory, the embodiments and features in the embodiments of the present disclosure can be combined with each other.

[0018] In order to fully understand the present disclosure, many specific details are described in the following description, but the present disclosure may be embodied in other forms different from those described herein. Apparently, the embodiments in the specification are only some embodiments of the present disclosure, but not all embodiments.

[0019] 1 is a flow chart of a song generation method according to an embodiment of the present disclosure. This embodiment is applicable to a case where a song is created based on lyrics text at a client, and the song generation method may be executed by a song generation device, which may be implemented in a software and / or hardware manner, and the device may be disposed in an electronic device, such as a terminal, including but not limited to a smartphone, palmtop, tablet, wearable device with display, desktop, laptop, all-in-one device, smart home device, etc. Or, this embodiment is applicable to a case where a song is created based on lyrics text at a service side, and the song generation method may be executed by a song generation device, which may be implemented in a software and / or hardware manner, and the device may be disposed in an electronic device, such as a server.

[0020] As shown in FIG. 1, the song generation method may specifically include steps S110 to S140.

[0021] S110: The target lyrics text input by the user is obtained.

[0022] There are multiple ways to implement this step, and the present application does not impose any limitations thereon. In some embodiments, in response to a trigger operation for generating a song, a song generation interface is displayed on the display of the terminal. The song generation interface includes a text input box. When a user needs to create a song, the user inputs a target lyric text into the text input box. The song generation system achieves the purpose of obtaining the target lyric text by identifying the target lyric text input by the user.

[0023] S120: Align the target lyrics text and the singing melody of the initial song to determine a correspondence between text units in the target lyrics text and notes in the singing melody, where the singing melody is a singing melody for the initial lyrics in the initial song.

[0024] The singing melody refers to the main melody of the song that the user wants to create.

[0025] By performing the alignment, it is possible to determine the correspondence between the text units in the target lyric text and the notes in the singing melody. The text units may be one or a combination of characters, words, sentences, phonetic units, etc.

[0026] In some embodiments, before this step, the song generating method further includes: selecting an initial song from a preset plurality of songs in response to an initial song selection operation, and determining a corresponding singing melody and accompaniment audio according to the initial song. This setting is essentially a matter of pre-setting a song database containing a plurality of songs, and allowing a user to select a song that the user likes from the song database as the initial song.

[0027] S130: The target lyrics text is subjected to voice synthesis based on the correspondence between the text units in the target lyrics text and the notes in the singing melody, to obtain a singing voice for singing the target lyrics text with the singing melody.

[0028] The voice synthesis is singing voice synthesis (SVS), which synthesizes singing based on lyrics and song melody. Compared to text to speech (TTS), which makes a machine "speak", singing voice synthesis is more entertaining because it makes a machine sing. A singing synthesis model may be generated by training in advance. In this way, by inputting the timbre, custom text, and song melody to the singing synthesis model, the singing synthesis model can output audio including pronunciation corresponding to the custom text. Conventional technology can be used for training the singing synthesis model, and no further explanation will be given.

[0029] S140: The singing voice and the accompaniment audio of the initial song are integrated to generate a target song.

[0030] The accompaniment audio refers to the chords of the song the user wants to create, and serves to complement the main melody.

[0031] In some embodiments, a one-to-one correspondence between a vocal melody and an audio accompaniment may actually be provided, i.e., when a user selects a vocal melody, an audio accompaniment is simultaneously selected since there is a one-to-one correspondence between the two.

[0032] Alternatively, the singing melody and the accompaniment audio may not correspond to each other, that is, the user needs to select the singing melody and then the accompaniment audio.

[0033] The synthesis refers to fusing the singing voice with the accompaniment audio to form a new song. The synthesis method is a prior art, and will not be described further here.

[0034] The above technical solution essentially allows users to write lyrics themselves, and create songs based on the lyrics, singing melody and accompanying audio created by the users themselves to form a completely new song. In this way, even if users do not have professional skills in music creation, they can create songs based on the lyrics they write themselves, which improves the efficiency and interest of users in song creation.

[0035] Fig. 2 is a flow chart of another song generating method according to an embodiment of the present disclosure. Fig. 2 is a specific example of Fig. 1. Referring to Fig. 2, the song generating method includes the following steps.

[0036] S210: The target lyrics text input by the user is obtained.

[0037] S220: Divide the vocal melody of the initial song into a plurality of melody paragraphs.

[0038] In some embodiments, the vocal melody may or may not actually be pre-processed.

[0039] If the singing melody is pre-processed, the singing melody includes a plurality of paragraph division points, and the positions of the paragraph division points in the singing melody are constant. Based on the paragraph division points, the singing melody can be divided into a plurality of melody paragraphs.

[0040] If the singing melody is not pre-processed, the singing melody does not contain paragraph division points, in which case the singing melody needs to be processed to have paragraph division points with fixed positions, and further achieve the purpose of dividing the singing melody into multiple melody paragraphs.

[0041] If the singing melody is not pre-processed, there are multiple ways to implement this step, and the present application does not impose any limitations on this. In some embodiments, the paragraph division points may be determined and further divided based only on the bars of the singing melody, or the paragraph division points may be determined and further divided based only on the number of notes.

[0042] In some embodiments, the following provides a method for determining paragraph division points in a singing melody, the method includes: determining one paragraph division point for every predetermined bar; adjusting the number of paragraph division points according to the number of notes included in the melody paragraph corresponding to each paragraph division point; and adjusting the position of each paragraph division point according to a note attack distance, where the note attack distance includes the duration pitch of the note attack and / or the pitch pitch of the note attack. This setting is essentially a preliminary determination of paragraph division points according to the number of bars, and then adjusting the paragraph division points according to the number of notes and the note attack distance to obtain the final position of the paragraph division points. This setting can reduce the difficulty of the subsequent alignment operation.

[0043] Adjusting the number of paragraph division points based on the number of notes contained in the melody paragraph corresponding to each paragraph division point includes deleting any one of the paragraph division points if the number of notes contained in the melody paragraph corresponding to any one of the paragraph division points is smaller than a first threshold value, and increasing the paragraph division point by one if the number of notes contained in the melody paragraph corresponding to any one of the paragraph division points is greater than a second threshold value. By setting in this way, the number of notes contained in each melody paragraph can be made uniform, which is advantageous for matching subsequent lyrics characters with notes.

[0044] Adjusting the position of each paragraph division point based on the note attack distance includes searching for a position where the note attack distance satisfies a predetermined condition within a predetermined range of beats around any one of the paragraph division points, and setting that position as the position of the any one of the paragraph division points, wherein the predetermined condition is that the duration pitch of the note attacks before and after the any one of the paragraph division points is the largest, or that the duration pitches of the note attacks before and after the any one of the paragraph division points are equal and the pitch pitch is the largest.

[0045] FIG. 3 is a staff schematic diagram of one melody according to an embodiment of the present disclosure. In some embodiments, FIG. 3 includes two lines of melody, each line having four bars. First, one paragraph division point is determined for every two bars, and the first determined paragraph division point in the first line is a1 in the figure, and the first determined paragraph division point in the second line is b1 in the figure. Then, the first determined paragraph division point is corrected and adjusted based on the number of notes and the note attack distance. After the adjustment, the final determined paragraph division point in the first line is a2 in the figure, and the final determined paragraph division point in the second line is b2 in the figure. Finally, the melody in the first line is divided into two melody paragraphs, 1 and 2, and the melody in the second line is divided into two melody paragraphs, 3 and 4.

[0046] S230: Divide the target lyric text into a plurality of lyric paragraphs, and the number of the plurality of lyric paragraphs is the same as the number of the plurality of melody paragraphs.

[0047] There are multiple ways to implement this step, and the present application does not impose any limitations thereon. In some embodiments, the target lyric text is segmented into words, a part of speech corresponding to each word is determined, and the target lyric text is divided into a plurality of lyric paragraphs based on the part of speech corresponding to each word, predefined linguistic rules, and the length of the singing melody.

[0048] Parts of speech refer to the classification of words based primarily on grammatical features (including syntactic function and morphological changes) while also taking into account lexical meaning, and include nouns, verbs, adjectives, pronouns, adverbs, and particles.

[0049] Linguistic rules are segmentation restriction rules established based on semantic completeness, such as indivisible between adverbs and verbs, indivisible between numerals and classifiers, indivisible between pronouns and numerals, etc. Segmenting the target lyric text into multiple lyric paragraphs based on the parts of speech corresponding to each word and predefined linguistic rules means that after segmentation, indivisible phrases are located in the same lyric paragraph.

[0050] Dividing the target lyrics text into a plurality of lyrics paragraphs based on the length of the singing melody means that after division, the number of characters in each lyrics paragraph is equal to or less than the number of notes in the melody paragraph corresponding to the lyrics paragraph.

[0051] In reality, there are often multiple linguistic rules. Considering that the number of notes in a melody paragraph is finite, a dynamic optimization strategy is set up to minimize the difference in length of each lyric phrase after division under the conditions of as many linguistic rules as possible.

[0052] In some embodiments, the target lyric text entered by the user is (outside 1) TIFF2025506527000002.tif7127" (Translation: I learned to love freely only after meeting you) (length: 11 characters). The target lyrics text is segmented into words, (outside 2) TIFF2025506527000003.tif7127, the phrases "after", "talent", "academic", "love", "obtain", and "freedom" are obtained. The parts of speech for each of the obtained phrases are written as follows. (Outside 3) TIFF2025506527000004.tif7127-pronoun, after-direction, talent-adverb, study-verb, love-verb, get-particle, free-adjective. Based on the dynamic optimization strategy, the target lyrics text is finally divided into:

[0053] Achievement (outside 4) TIFF2025506527000005.tif7127 (length: 4 characters), Talented, loved and free (length: 7 characters).

[0054] The linguistic rule used in this segmentation process is the indivisibility between adverbs and verbs. After segmentation, the difference in length between the two lyric phrases is three letters.

[0055] S240: A plurality of lyric paragraphs and a plurality of melody paragraphs are aligned one by one, and a correspondence between a text unit in a lyric paragraph and a note in the corresponding melody paragraph is determined.

[0056] In some embodiments, a specific implementation of this step includes:

[0057] First, a correspondence between the lyrics paragraph and the melody paragraph is established.

[0058] In some embodiments, if the target lyric text can be divided into eight lyric paragraphs, there are a total of eight melody paragraphs, and a correspondence between the first lyric paragraph and the first melody paragraph, a correspondence between the second lyric paragraph and the second melody paragraph, ..., a correspondence between the eighth lyric paragraph and the eighth melody paragraph is established.

[0059] Then, the text units in the lyric paragraph that have a corresponding relationship are associated with the notes in the melody paragraph.

[0060] There are several specific implementation methods for achieving "associating text units in a lyric paragraph with notes in a melody paragraph that have a corresponding relationship," and this application does not impose any restrictions on these. In some embodiments, the n-th character in the lyric text unit and the n-th note in the melody paragraph are aligned in order.

[0061] Alternatively, a specific implementation method of this step includes: for each melody paragraph, obtaining a plurality of lyric alignment templates corresponding to the predetermined melody paragraph, each lyric alignment template corresponding to a different number of lyric characters; selecting a target lyric alignment template from the plurality of lyric alignment templates, the number of characters corresponding to the target lyric alignment template being the number of characters of the lyric paragraph corresponding to the melody paragraph; and aligning the lyric paragraph corresponding to the melody paragraph with the melody paragraph based on the target lyric alignment template.

[0062] In some embodiments, if a melody paragraph contains seven notes, three lyrics alignment templates are set for the melody paragraph, as follows:

[0063] The first template is xxxxxxx and is applied to lyric paragraphs containing seven characters.

[0064] The second template is xxx-xxx and is applied to lyric paragraphs containing six characters.

[0065] The third template is xx--xxx, which applies to lyric paragraphs containing five characters.

[0066] An "x" represents a note with corresponding lyrics, and a "-" represents a note without corresponding lyrics.

[0067] Let's assume that "Saigakuka Aitoku Jiyuu" in the above example corresponds to the melody paragraph. Since "Saigakuka Aitoku Jiyuu" has a total of seven characters, the first template is selected as the target lyrics alignment template. Then, the first note in the melody paragraph is aligned with "Saigaku", the second note is aligned with "Gaku", ... and the seventh note is aligned with "Yu".

[0068] Suppose that for a certain melody paragraph, a lyric alignment template corresponding to the melody paragraph has not been determined in advance. In practice, a first lyric alignment template, a second lyric alignment template and a third lyric alignment template can be generated based on the melody paragraph. In the first lyric alignment template, each note in the melody paragraph corresponds to one text unit (e.g., character). Until the number of text units (e.g., characters) that the finally obtained lyric alignment template can match is equal to a set threshold, the following steps are repeatedly performed: based on the previous lyric alignment template, the adjacent notes in the melody paragraph with the closest note attack distance are integrated into one note pair, and the note pair is regarded as one new note, and is associated with only one character to obtain a new lyric alignment template. Here, the set threshold is a positive integer, and is greater than or equal to 1 and less than or equal to the total number of notes in the melody paragraph.

[0069] In some embodiments, a melody paragraph contains a total of seven notes, and the set threshold is four. Each note in the melody paragraph is associated with one character to obtain a first lyric alignment template. The first lyric alignment template can match seven characters. The adjacent notes in the melody paragraph with the closest note attack distance are combined and then associated with one character to obtain a second lyric alignment template, which can match six characters. The adjacent notes in the second lyric alignment template with the closest note attack distance are combined and then associated with one character to obtain a third lyric alignment template, which can match five characters. The adjacent notes with the closest note attack distance in the third lyric alignment template are integrated and then matched with one character to obtain a fourth lyric alignment template, which can match four characters and is equal to the set threshold, and no new lyric alignment templates are generated any more. Therefore, four lyric alignment templates are set for the melody paragraph.

[0070] In some embodiments, FIG. 4 is a staff schematic diagram of one melody paragraph according to an embodiment of the present disclosure. Referring to FIG. 4, each note in the melody paragraph is associated with one character to obtain a first lyric alignment template. Since the adjacent notes surrounded by the rounded frame 1 have the closest attack distance, these two notes are integrated and associated with one character to obtain a second lyric alignment template. At this time, the number of characters that can be matched by it is reduced by one compared to the first lyric alignment template. After integrating the adjacent notes surrounded by the rounded frame 1, the adjacent notes surrounded by the rounded frame 2 have the closest attack distance, these two notes are integrated and associated with one character to obtain a third lyric alignment template. At this time, the number of characters that can be matched by it is reduced by one compared to the second lyric alignment template. After merging the adjacent notes surrounded by the rounded frame 2, the adjacent note surrounded by the rounded frame 3 has the closest attack distance, and these two notes are merged and associated with one character to obtain the fourth lyric alignment template. At this time, the number of characters that can be matched with it is reduced by one with respect to the third lyric alignment template. By repeating this process, multiple lyric alignment templates are obtained. In FIG. 4, in some embodiments, the top six merged note pairs are displayed.

[0071] In some embodiments, while performing the process of "based on the previous lyric alignment template, merging adjacent notes in the melody paragraph with the closest note attack distance into one note pair, treating the note pair as one new note and corresponding to only one character to obtain a new lyric alignment template," if there are multiple sets of adjacent notes in the melody paragraph, and the multiple sets of adjacent notes satisfy the conditions that the note attack distance is the same and the note attack distance is the closest, the first beat, third beat and the first half beat of each note in a measure are merged.

[0072] S250: The target lyrics text is voice-synthesized based on the correspondence between the text units in the target lyrics text and the notes in the singing melody to obtain a singing voice that sings the target lyrics text with the singing melody.

[0073] S260: The singing voice and the accompaniment audio of the initial song are integrated to generate a target song.

[0074] The above technical solution details a method for aligning a target lyric text and a singing melody, which can reduce the difficulty of aligning the target lyric text and the singing melody, is conducive to realizing the purpose of generating a song based on the target lyric text, and enhances the efficiency and interest of users in song creation.

[0075] 5 is a schematic diagram of a display interface for realizing song generation based on target lyric text in a terminal according to an embodiment of the present disclosure. Referring to FIG. 5, the display interface divides song generation into three steps. The first step is a custom option part, the second step is a text input part, and the third step is a result playback part.

[0076] In the first step, the user can configure the song setting item, the tone setting item, and the intelligent completion item. All of these three setting items are configured as pull-down lists.

[0077] When the user triggers the song setting item, multiple songs may be displayed in a pull-down list format to allow the user to select one song, and each song is pre-set with a unique melody and accompaniment.

[0078] When the user triggers the tone setting item, a plurality of tones, for example, a male tone, a female tone, a child tone, etc., may be displayed in a pull-down list format, and the user may select one tone.

[0079] When the user triggers the intelligent completion item, multiple intelligent completion methods may be displayed in a pull-down list manner. Intelligent completion may be considered as making adjustment settings on the custom text entered by the user to adapt it to the selected melody. For example, if the custom text entered by the user can be divided into a maximum of five lyric phrases, but the target melody contains six melody paragraphs, intelligent completion is performed on the custom text entered by the user, for example, by completing it with "la la la" to form one new lyric phrase, so that the custom lyrics entered by the user finally become lyrics containing six lyric phrases. Or, after the custom text entered by the user is divided, if a lyric phrase contains five characters, but the lyric alignment template corresponding to the target melody has the smallest number of characters that can be matched is seven, for example, the lyric alignment template is used to complete it with "la la" so that the number of characters contained in the lyric phrase matches the number of characters that can be matched by the lyric alignment template. In order to facilitate the user to quickly understand the meaning of the intelligent completion setting item, an interpretation explanation may be provided below the intelligent completion setting item. In some embodiments, the interpretation instruction is "If the number of characters in the lyrics entered is insufficient, an intelligent completion method will be used to complete the song."

[0080] In the second step, in order to facilitate the user to quickly understand the lyrics creation method, a advice is added below the lyrics input box, which states "It is most effective to input four Chinese texts, each containing 8-17 Chinese characters and separated by punctuation or wrapping. If the input text is too long, it will be intelligently reorganized." The advice content can be changed as needed. In addition, the user can arrange the audio format of the generated song. After the arrangement selection settings in the first step and the second step are both completed, when the user triggers the "Generate Song" control in the second step, the lyrics of the generated song may be displayed in the lyrics input box. At this time, the lyrics are divided into lyrics phrases.

[0081] In a third step, the user can preview the generated song by clicking a play control on the music player.The user can download the generated song by clicking a download control.

[0082] Although the above-mentioned method embodiments are all expressed as a combination of a series of operations for ease of explanation, those skilled in the art will appreciate that the present invention is not limited to the order of operations described, since some steps may be performed in other orders or simultaneously according to the present invention. Next, those skilled in the art will appreciate that the embodiments described in the specification are all preferred embodiments, and the operations and modules involved are not necessarily essential to the present invention.

[0083] The technical solution according to the embodiment of the present disclosure obtains a target lyric text input by a user, aligns the target lyric text with the singing melody of an initial song, determines the correspondence between the text units in the target lyric text and the notes in the singing melody, and synthesizes the target lyric text according to the correspondence between the text units in the target lyric text and the notes in the singing melody to obtain a singing voice that recites the target lyric text with the singing melody, and then combines the singing voice with the accompaniment audio of the initial song to generate a target song, which essentially allows the user to write lyrics by himself, and create a song based on the lyrics, singing melody and accompaniment audio created by the user, to form a completely new song. In this way, the user can create a song based on the lyrics he creates without having professional skills in music creation, which improves the efficiency and interest of the user in song creation.

[0084] 6 is a schematic diagram of a configuration of a song generating device according to an embodiment of the present disclosure. The song generating device according to an embodiment of the present disclosure may be disposed on a client or a server. Referring to FIG. 6, the song generating device specifically includes: an acquisition unit 61 for acquiring a target lyrics text input by a user; an alignment unit 62 for aligning a target lyrics text with a singing melody of an initial song to determine a correspondence between text units in the target lyrics text and notes in the singing melody, the singing melody being a singing melody for an initial lyrics of the initial song; a synthesis unit 63 for synthesizing the target lyrics text with a singing voice based on a correspondence between the text units in the target lyrics text and the notes in the singing melody to obtain a singing voice for singing the target lyrics text with the singing melody; and a generating unit 64 for integrating the singing voice and the accompaniment audio of the initial song to generate the target song.

[0085] In some embodiments, the song generation device comprises: a selection unit for selecting an initial song from a plurality of preset songs in response to an initial song selection operation; The apparatus further includes a determining unit for determining a corresponding singing melody and accompaniment audio based on the initial song.

[0086] In some embodiments, alignment portion 62 may include: Dividing the singing melody into a plurality of melody paragraphs; Dividing the target lyric text into a plurality of lyric paragraphs, the number of the plurality of lyric paragraphs being equal to the number of the plurality of melody paragraphs; The alignment method is used to align the plurality of lyric paragraphs and the plurality of melody paragraphs one by one, and to determine a correspondence between a text unit in the lyric paragraph and a note in the corresponding melody paragraph.

[0087] In some embodiments, the alignment unit 62 dividing the vocal melody into a plurality of melody paragraphs includes determining one paragraph division point for each predetermined bar in the vocal melody, adjusting the number of paragraph division points based on the number of notes contained in the melody paragraph corresponding to each paragraph division point, and adjusting the position of each paragraph division point based on a note attack distance, wherein the note attack distance includes the duration pitch of the note attack and / or the pitch of the note attack.

[0088] In some embodiments, adjusting the number of paragraph division points based on the number of notes contained in the melody paragraph corresponding to each paragraph division point by the alignment unit 62 includes, for any one paragraph division point, deleting the paragraph division point if the number of notes contained in the melody paragraph corresponding to the paragraph division point is smaller than a first threshold, and adding one paragraph division point if the number of notes contained in the melody paragraph corresponding to the paragraph division point is greater than a second threshold.

[0089] In some embodiments, adjusting the position of each paragraph division point based on the note attack distance by the alignment unit 62 includes searching for a position for any one of the paragraph division points within a predetermined range of beats around the paragraph division point where the note attack distance satisfies a predetermined condition, and setting that position as the position of the paragraph division point, wherein the predetermined condition is that the duration pitch of the note attacks before and after the paragraph division point is the largest, or that the duration pitches of the note attacks before and after the paragraph division point are equal and the pitch pitch is the largest.

[0090] In some embodiments, the alignment unit 62 splitting the target lyric text into a plurality of lyric paragraphs includes performing word segmentation on the target lyric text to determine a part of speech corresponding to each word, and splitting the target lyric text into a plurality of lyric paragraphs based on the part of speech corresponding to each word, predefined linguistic rules, and the length of the vocal melody.

[0091] In some embodiments, aligning the plurality of lyric paragraphs with the plurality of melody paragraphs one by one by the alignment unit 62 includes: for each of the melody paragraphs, obtaining a plurality of lyric alignment templates corresponding to the predetermined melody paragraph, each of the lyric alignment templates corresponding to a different number of lyric characters; selecting a target lyric alignment template from the plurality of lyric alignment templates, the number of characters corresponding to the target lyric alignment template being the number of characters of the lyric paragraph corresponding to the melody paragraph; and aligning the lyric paragraph corresponding to the melody paragraph with the melody paragraph based on the target lyric alignment template.

[0092] In some embodiments, the plurality of lyric alignment templates corresponding to the melody paragraph include a first lyric alignment template, a second lyric alignment template and a third lyric alignment template; In the first lyric alignment template, each note in the melody paragraph corresponds to one text unit; In the second lyrics alignment template, adjacent notes in the melody paragraph that have the closest note attack distance are merged into one note pair, and one note pair corresponds to one text unit; In the third lyric alignment template, adjacent notes in the second lyric alignment template that have the closest note attack distance are merged into one note pair, and one note pair corresponds to one text unit.

[0093] The song generation device according to the embodiments of the present disclosure can perform the steps performed by the client or the server in the song generation method according to the method embodiments of the present disclosure, and the execution steps and beneficial effects provided therein will not be further described here.

[0094] In some embodiments, the division of each means in the song generating device is only a division of logical functions, and there may be other division methods when actually realized, for example, at least two means in the song generating device may be realized as one means, and each means in the song generating device may be divided into multiple sub-means. As can be understood, each means or sub-means may be realized by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in a hardware manner or a software manner depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to realize the described functions for each specific application.

[0095] 7 is an exemplary block diagram of a system including at least one computing device and at least one storage device for storing instructions according to an embodiment of the present disclosure. In some embodiments, the system may be used for big data processing, and the at least one computing device and the at least one storage device may be arranged in a distributed manner, making the system a distributed data processing cluster.

[0096] 7, the system includes at least one computing device 51 and at least one memory device 52 for storing instructions. As can be appreciated, the memory device 52 in this embodiment may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memory.

[0097] In some embodiments, storage device 52 stores elements such as executable units or data structures, or a subset or extension thereof, of an operating system and application programs.

[0098] The operating system includes various system programs, such as a framework layer, a core library layer, a driver layer, etc., which are used to realize various basic tasks and process hardware-based tasks. The application programs include various application programs, such as a media player, a browser, etc., which are used to realize various application tasks. A program for realizing the song generation method according to the embodiment of the present disclosure may be included in the application programs.

[0099] In the embodiments of the present disclosure, the at least one computing device 51 may call programs or instructions stored in the at least one storage device 52, in particular programs or instructions stored in application programs. The at least one computing device 51 is used to execute steps of each embodiment of the song generation method according to the embodiments of the present disclosure.

[0100] The song generating method according to the embodiment of the present disclosure may be applied to or realized by a computing device 51. The computing device 51 may be an integrated circuit chip having the ability of processing signals. In the process of realization, each step of the above method may be completed by an instruction in the form of an integrated logic circuit of hardware or software in the computing device 51. The above computing device 51 may be a general-purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component. The general-purpose processor may be a microprocessor, or the processor may be any conventional processor, etc.

[0101] The steps of the song generation method according to the embodiment of the present disclosure may be directly embodied to be executed and completed by a hardware decoding processor, or may be executed and completed by a combination of hardware and software units in the decoding processor. The software units may be located in a storage medium matured in the art, such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the storage device 52, and the computing device 51 reads the information in the storage device 52 and combines the hardware to complete the steps of the method.

[0102] The embodiments of the present disclosure further propose a computer-readable storage medium storing a program or instructions which, when executed by at least one computing device, causes the at least one computing device to perform steps of each embodiment of the song generation method. To avoid repetition, no further description will be given here. Here, the computing device may be the computing device 51 shown in FIG. 6. In some embodiments, the computer-readable storage medium is a non-transitory computer-readable storage medium.

[0103] An embodiment of the present disclosure further proposes a computer program product, the computer program product including a computer program, the computer program being stored in a non-transitory computer-readable storage medium, and at least one processor of a computer reading and executing the computer program from the storage medium to cause the computer to perform steps of each embodiment of the song generation method, which will not be described further here in order to avoid repetition.

[0104] An embodiment of the present disclosure further proposes a computer program comprising instructions which, when executed by a processor, cause said processor to perform a song generation method according to any embodiment of the present disclosure.

[0105] It is to be noted that, in this specification, the terms "comprises", "including", or any other variation thereof, are intended to cover the non-exclusive "comprises", whereby a process, method, article, or apparatus that includes a set of elements not only includes those elements, but also includes other elements not expressly listed or that are inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "including..." does not exclude the presence of other identical elements in a process, method, article, or apparatus that includes the element.

[0106] As will be understood by one of ordinary skill in the art, while some embodiments described herein include certain features and not other features that are included in other embodiments, it is meant that a combination of features from different embodiments forms a different embodiment, within the scope of the present disclosure.

[0107] As can be understood by those skilled in the art, the description of each embodiment is focused on, and the details of one embodiment that are not described in detail can be referred to the descriptions of other embodiments.

[0108] Although the embodiments of the present disclosure have been described with reference to the drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present disclosure, and such modifications and variations fall within the scope limited by the appended claims.

Claims

1. Obtaining a target lyric text entered by a user; aligning the target lyrics text with a singing melody of an initial song to determine a correspondence between text units in the target lyrics text and notes in the singing melody, the singing melody being a singing melody for initial lyrics in the initial song; synthesizing the target lyrics text based on a correspondence between text units in the target lyrics text and notes in the singing melody to obtain a singing voice for singing the target lyrics text with the singing melody; and integrating the singing voice with an audio accompaniment of the initial song to generate a target song.

2. Prior to aligning the target lyric text with a vocal melody of an initial song, the method further comprises: selecting an initial song from a plurality of preset songs in response to an initial song selection operation; The method of claim 1 , further comprising: determining a corresponding singing melody and accompanying audio based on the initial song.

3. Aligning the target lyric text with a vocal melody of an initial song includes: Dividing the singing melody into a plurality of melody paragraphs; Dividing the target lyric text into a plurality of lyric paragraphs, the number of the plurality of lyric paragraphs being equal to the number of the plurality of melody paragraphs; 3. The method of claim 1, further comprising aligning the lyric paragraphs and the melody paragraphs one by one, and determining a correspondence between text units in the lyric paragraphs and notes in the corresponding melody paragraphs.

4. Dividing the singing melody into a plurality of melody paragraphs includes: determining a paragraph division point for each predetermined measure in the singing melody; adjusting the number of paragraph division points based on the number of notes included in the melody paragraph corresponding to each paragraph division point; 4. The method of claim 3, further comprising adjusting a position of each paragraph division point based on a note attack distance, the note attack distance including a duration pitch of a note attack and / or a pitch of a note attack.

5. Adjusting the number of paragraph division points based on the number of notes included in the melody paragraph corresponding to each of the paragraph division points When the number of notes included in the melody paragraph corresponding to any one of the paragraph division points is smaller than a first threshold value, deleting the any one of the paragraph division points; 5. The song generating method according to claim 4, further comprising: if the number of notes contained in the melody paragraph corresponding to any one of the paragraph division points is greater than a second threshold value, increasing the paragraph division point by one.

6. Adjusting the position of each paragraph division point based on note attack distance searching for a position where a note attack distance satisfies a predetermined condition within a predetermined beat range around any one of the paragraph division points, and determining the position as the position of the any one of the paragraph division points; 6. A song generation method according to claim 4 or 5, wherein the predetermined condition is that the duration pitch of the note attacks before and after any one of the paragraph division points is the largest, or that the duration pitch of the note attacks before and after any one of the paragraph division points is equal and the pitch pitch is the largest.

7. Segmenting the target lyric text into a plurality of lyric paragraphs includes: segmenting the target lyrics text into words and determining a part of speech corresponding to each word; and dividing the target lyric text into a plurality of lyric paragraphs based on parts of speech corresponding to each word, predefined linguistic rules and the length of the singing melody.

8. The aligning of the plurality of lyric paragraphs and the plurality of melody paragraphs one by one includes: For each of the melody paragraphs, obtain a plurality of lyric alignment templates corresponding to the predetermined melody paragraphs, each of the lyric alignment templates corresponding to a different number of lyric characters; selecting a target lyric alignment template from the plurality of lyric alignment templates, the number of characters corresponding to the target lyric alignment template being the number of characters of a lyric paragraph corresponding to the melody paragraph; 8. The method of claim 7, further comprising aligning the melody paragraph with a lyric paragraph corresponding to the melody paragraph based on the target lyric alignment template.

9. the plurality of lyric alignment templates corresponding to the melody paragraph include a first lyric alignment template, a second lyric alignment template and a third lyric alignment template; In the first lyric alignment template, each note in the melody paragraph corresponds to one text unit; In the second lyrics alignment template, adjacent notes in the melody paragraph that have the closest note attack distance are merged into one note pair, and one note pair corresponds to one text unit; 9. The song generation method of claim 8, wherein in the third lyric alignment template, adjacent notes in the second lyric alignment template with the closest note attack distance are merged into one note pair, and one note pair corresponds to one text unit.

10. The aligning of the plurality of lyric paragraphs and the plurality of melody paragraphs one by one includes: Establishing a correspondence between the lyric paragraph and the melody paragraph; 8. The method of claim 7, further comprising: matching text units in a lyric paragraph with notes in a melody paragraph that have a corresponding relationship.

11. Segmenting the target lyric text into a number of lyric paragraphs based on parts of speech corresponding to each word and predefined linguistic rules, and after the segmentation, locating indivisible phrases in the same lyric paragraph; and / or The song generation method according to any one of claims 7 to 10, further comprising dividing the target lyrics text into a plurality of lyrics paragraphs based on the length of the singing melody, and after the division, making the number of characters in each lyrics paragraph equal to or less than the number of notes in the melody paragraph corresponding to the lyrics paragraph.

12. an acquisition unit for acquiring a target lyric text input by a user; an alignment unit for aligning the target lyrics text with a singing melody of an initial song to determine a correspondence between a text unit in the target lyrics text and a note in the singing melody, the singing melody being a singing melody for an initial lyrics of the initial song; a synthesis unit for synthesizing the target lyrics text based on a correspondence between text units in the target lyrics text and notes in the singing melody to obtain a singing voice for singing the target lyrics text with the singing melody; a generator for integrating the singing voice and an audio accompaniment of the initial song to generate a target song.

13. 12. A system comprising at least one computing device and at least one storage device storing instructions which, when executed by said at least one computing device, cause said at least one computing device to perform steps of a song generation method according to any one of claims 1 to 11.

14. A computer readable storage medium storing a program or instructions which, when executed by at least one computing device, causes the at least one computing device to perform the steps of the song production method according to any one of claims 1 to 11.

15. A computer program comprising instructions which, when executed by a processor, cause the processor to perform the song production method of any one of claims 1 to 11.

Images