AI song editing method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides an AI song editing method and device, electronic equipment and a storage medium, after a first song and a corresponding first lyric are generated, the first lyric of the first song is displayed, and the first song is audio data generated based on an artificial intelligence technology; in response to a modification instruction for a first character in the first lyrics, generating a second lyrics, the second lyrics including a second character corresponding to the playing timestamp of the first character; in response to generation of the second lyrics, a second song is generated based on the second characters, the second song comprises a target song segment and a non-target song segment, the target song segment is a song segment corresponding to the second characters, and the melody of the non-target song segment is the same as the melody of the corresponding song segment in the first song. The melody of the song is not changed while the lyrics are accurately modified, and the lyric modification flexibility and accuracy of the AI song are improved.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the field of Internet technology, and in particular to an AI song editing method, device, electronic device, and storage medium. Background Art

[0002] Currently, Artificial Intelligence (AI) technology has been increasingly applied to various industries. For example, by training a song generation model through music samples, the model can be given the ability to "create" music, thereby generating "original songs" that meet user requirements, that is, songs generated based on AIGC technology, which can also be referred to as AI songs.

[0003] After AI songs are generated using AIGC technology, since the lyrics in the AI ​​songs are generated by the AI ​​model, problems such as inappropriate lyrics and inaccurate meaning often occur. In the prior art, the only way is to add corresponding restrictions to the prompt words and regenerate the AI ​​songs in an attempt to obtain songs that meet user requirements.

[0004] However, in the scenario of adjusting the lyrics of AI songs, the solutions in the prior art have problems such as being unable to accurately modify the lyrics of AI songs and the complicated interaction process, which affects the user experience. Summary of the invention

[0005] The embodiments of the present disclosure provide an AI song editing method, device, electronic device, and storage medium to overcome the problem of being unable to accurately modify the lyrics of AI songs and the complex interaction process.

[0006] In a first aspect, an embodiment of the present disclosure provides an AI song editing method, comprising:

[0007] After generating a first song and the corresponding first lyrics, displaying the first lyrics of the first song, wherein the first song is audio data generated based on artificial intelligence technology; generating second lyrics in response to a modification instruction for a first character in the first lyrics, wherein the second lyrics contain second characters corresponding to a playback timestamp of the first characters; generating a second song based on the second characters in response to the generation of the second lyrics, wherein the second song includes a target song segment and a non-target song segment, wherein the target song segment is a song segment corresponding to the second characters, and the non-target song segment is a song segment remaining in the second song except the target song segment, and the melody of the non-target song segment is the same as the melody of the second original song segment corresponding to the playback timestamp of the non-target song segment in the first song.

[0008] In a second aspect, an embodiment of the present disclosure provides an AI song editing device, comprising:

[0009] A display module, configured to display first lyrics of a first song after generating a first song and corresponding first lyrics, wherein the first song is audio data generated based on artificial intelligence technology;

[0010] A processing module, configured to generate a second lyric in response to a modification instruction for a first character in the first lyric, wherein the second lyric includes a second character corresponding to a playback timestamp of the first character;

[0011] A generation module is used to generate a second song based on the second text in response to the generation of the second lyrics, wherein the second song includes a target song segment and a non-target song segment, the target song segment is the song segment corresponding to the second text, the non-target song segment is the song segment remaining in the second song except the target song segment, and the melody of the non-target song segment is the same as the melody of the second original song segment corresponding to the playback timestamp of the non-target song segment in the first song.

[0012] In a third aspect, an embodiment of the present disclosure provides an electronic device, including: a processor and a memory;

[0013] The memory stores computer-executable instructions;

[0014] The processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the AI ​​song editing method described in the first aspect and various possible designs of the first aspect.

[0015] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium, in which computer execution instructions are stored. When a processor executes the computer execution instructions, the AI ​​song editing method described in the first aspect and various possible designs of the first aspect is implemented.

[0016] In a fifth aspect, an embodiment of the present disclosure provides a computer program product, including a computer program, which, when executed by a processor, implements the AI ​​song editing method described in the first aspect and various possible designs of the first aspect.

[0017] The AI ​​song editing method, device, electronic device and storage medium provided in this embodiment, after generating a first song and the corresponding first lyrics, display the first lyrics of the first song, wherein the first song is audio data generated based on artificial intelligence technology; in response to a modification instruction for the first text in the first lyrics, generate a second lyric, wherein the second lyric contains a second text corresponding to the playback timestamp of the first text; in response to the generation of the second lyrics, generate a second song based on the second text, wherein the second song includes a target song segment and a non-target song segment, wherein the target song segment is the song segment corresponding to the second text, and the non-target song segment is the song segment remaining in the second song except the target song segment, and the melody of the non-target song segment is the same as the melody of the second original song segment corresponding to the playback timestamp of the non-target song segment in the first song. By displaying the first lyrics of the first song, and modifying the first lyrics to the second lyrics in response to the modification instruction, and then generating the target song segment in the second song based on the second text in the second lyrics that changes relative to the first lyrics, while the non-target song segment in the second song does not change, while achieving accurate modification of the lyrics, the melody of the song is not changed, thereby improving the flexibility and accuracy of the lyrics modification of AI songs and improving the interaction efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0019] Figure 1 An application scenario diagram of the AI ​​song editing method provided in the embodiment of the present disclosure;

[0020] Figure 2 Schematic diagram of the process of the AI ​​song editing method provided in the embodiment of the present disclosure Figure 1 ;

[0021] Figure 3 A schematic diagram of an interface of a target application provided by an embodiment of the present disclosure;

[0022] Figure 4 for Figure 2 A flowchart of a specific implementation method of step S102 in the illustrated embodiment;

[0023] Figure 5 A schematic diagram of a process for generating second lyrics provided by an embodiment of the present disclosure;

[0024] Figure 6 A schematic diagram of a process for generating a second song provided by an embodiment of the present disclosure;

[0025] Figure 7 Schematic diagram of the AI ​​song editing method provided in the embodiment of the present disclosure Figure 2 ;

[0026] Figure 8 for Figure 2 A flowchart of a possible implementation of step S205 in the illustrated embodiment;

[0027] Fig. 9 for Figure 2 A flowchart of another possible implementation of step S205 in the illustrated embodiment;

[0028] Fig.10 A structural block diagram of an AI song editing device provided in an embodiment of the present disclosure;

[0029] Fig.11 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure;

[0030] Fig.12 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0031] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0032] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0033] The application scenarios of the embodiments of the present disclosure are explained below:

[0034] Figure 1This is an application scenario diagram of the AI ​​song editing method provided in the embodiment of the present disclosure. The AI ​​song editing method provided in the embodiment of the present disclosure can be applied to applications (APP, Application) with song generation and editing functions, such as song applications, short video applications, etc. More specifically, it can be applied to the application scenario of modifying the lyrics in AI songs. The execution subject of this embodiment can be a terminal device running the above-mentioned application with song generation and editing functions, or a server deploying the server corresponding to the above-mentioned application, or other electronic devices with similar functions. Among them, when the execution subject is a terminal device, the terminal device executes the method provided in this embodiment by running the above-mentioned application; when the execution subject is a server, the server of the above-mentioned application with song generation function or video generation function can be partially or completely run on the server, and the method provided in this embodiment is executed on the server side, and the terminal device runs the client of the application, and the communication between the server and the terminal device is based on the server-client, so that the terminal device can obtain the execution result of the method provided in this embodiment and display it as needed.

[0035] Among them, in some embodiments, the terminal device or server can implement the AI ​​song editing method provided by the embodiment of the present disclosure by running various computer executable instructions or computer programs. For example, computer executable instructions can be program-level commands, machine instructions or software instructions. The computer program can be a native program or software module in the operating system; it can be a local application, that is, a program that needs to be installed in the operating system to run, or it can be a small program embedded in any APP, that is, a program that runs based on a browser environment. In summary, the above-mentioned computer executable instructions can be instructions in any form, and the above-mentioned computer program can be an application, module or plug-in in any form, and the specific implementation form can be configured as needed. Furthermore, in the process of implementing the AI ​​song editing method provided by the embodiment of the present disclosure, the terminal device can execute the method by running a computer executable instruction or computer program set locally, or by calling a computer executable instruction or computer program set in an external server. In some embodiments, the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud storage, cloud communications, cloud databases, cloud computing, cloud functions, network services, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN), as well as big data and artificial intelligence platforms. Among them, cloud services can be interactive processing services for terminal devices to call.

[0036] refer to Figure 1 As shown in , taking the terminal device as an example, a target application with a song generation function is running in the terminal device. After the target application is started, the user operates the terminal device to input prompt words for generating songs into the target application, such as "sadness", "lyrical", "clear river", etc., as shown in the figure. Afterwards, the target application calls the song generation model deployed in the cloud based on the above prompt words, generates a corresponding "original song", that is, an AI song (shown as Music_AI_No.001 in the figure), and sends it back to the terminal device. After receiving the AI ​​song, the terminal device will display the lyrics corresponding to the AI ​​song (indicated by "X" in the figure), and play the AI ​​song based on user instructions.

[0037] In the prior art, after AI songs are generated through AI technology, since the lyrics in the AI ​​songs are generated by the AI ​​model, users cannot edit them accurately, so problems such as inappropriate lyrics and inaccurate meaning often occur. In response to the above problems, the prior art can only avoid the problem of inappropriate lyrics by adding corresponding restrictions in the prompt words and regenerating the AI ​​songs. However, at the same time, since the melody and lyrics of the AI ​​songs are generated by the model at the same time, there is a coupling between the two. Therefore, after the AI ​​songs are regenerated, the melody of the newly generated AI songs may change, and it is impossible to modify the lyrics without changing the melody. As a result, the lyrics of the AI ​​songs cannot be accurately modified, and the interaction process is complicated, which affects the user experience.

[0038] The disclosed embodiment provides an AI song editing method to solve the above-mentioned problems.

[0039] refer to Figure 2 , Figure 2 Schematic diagram of the process of the AI ​​song editing method provided in the embodiment of the present disclosure Figure 1 The method of this embodiment can be applied in a terminal device, and the AI ​​song editing method includes:

[0040] Step S101: After generating a first song and corresponding first lyrics, display the first lyrics of the first song, wherein the first song is audio data generated based on artificial intelligence technology.

[0041] Step S102: In response to the modification instruction for the first text in the first lyrics, generate second lyrics, the second lyrics including second text corresponding to the playback timestamp of the first text.

[0042] For example, reference Figure 1In the application scenario diagram shown, the terminal device generates an AI song based on AIGC technology by running a target application with a song generation function, that is, the first song. Among them, the process of generating such AI songs based on AIGC technology usually includes receiving a prompt word (prompt) input by the user, and calling a pre-trained song generation model in combination with the prompt word to generate music matching the prompt word, which will not be described in detail here. After the first song is generated, the first song and the corresponding first song text (i.e., the lyrics of the first song) are displayed in the target application.

[0043] Furthermore, after displaying the first lyrics of the first song, when the user needs to modify the lyrics of the first song, a modification instruction will be input into the terminal device. The modification instruction is for the first text in the first lyrics, specifically, one or more of adding, deleting, and changing the content of the first lyrics, wherein the content that changes in the first lyrics is the first text. Figure 3 A schematic diagram of an interface of a target application provided by an embodiment of the present disclosure, such as Figure 3 As shown, exemplarily, after the target application generates the first song, the song name of the first song is Music_AI_No.001. When playing or jumping to the first song, the first lyrics of the first song are synchronously displayed in the playback interface of the target application, such as "AAAAA", "BBBBB", etc. shown in the figure. Among them, the modification instruction modifies the 2nd, 3rd, and 4th characters "AAA" in the text segment "AAAAA", and adjusts them to "CCC", then the modified text segment becomes "ACCCA". The characters "AAA" before the above modification are the first characters; and the modified characters "CCC" are the second characters. Correspondingly, the lyrics text containing the above modified text segment is the second lyrics. In another possible implementation, the first characters modified by the modification instruction include multiple, and the positions of at least two first characters in the first lyrics are not continuous, then the positions of the corresponding generated second characters in the second lyrics are also not continuous.

[0044] In one possible implementation, Figure 4 As shown, the specific implementation of step S102 includes:

[0045] Step S1021: In response to a modification instruction for a first character in a target lyric text segment in the first lyric, a corresponding text segment to be determined is obtained, and the number of characters in the text segment to be determined is detected.

[0046] Step S1022: If the number of characters in the to-be-determined text segment is equal to the number of characters in the target lyrics text segment, the to-be-determined text segment is determined as the modified lyrics text segment.

[0047] Step S1023: Replace the target lyrics text segment in the first lyrics with the modified lyrics text segment to generate second lyrics.

[0048] Exemplarily, the modification instruction input by the user is used to modify the first text in the target lyrics text segment in the first lyrics, wherein the first lyrics are composed of multiple lyrics text segments, each of which can correspond to a sentence, for example, can be divided by punctuation. And each lyrics text segment is composed of one or more characters, and the characters are, for example, single Chinese characters. After modifying one or more first characters based on the modification instruction, a modified text segment, i.e., a to-be-determined text segment, is first generated, and then the to-be-determined text segment is tested to ensure that the number of Chinese characters in the modified to-be-determined text segment is consistent with the number of characters in the target lyrics text segment, that is, the modification instruction cannot change the number of characters in the target lyrics text segment. When it is confirmed through testing that the number of characters in the to-be-determined text segment is equal to the number of characters in the target lyrics text segment, the to-be-determined text segment generated after the modification is determined as the modified lyrics text segment. Afterwards, the target lyrics text segment in the first lyrics is replaced with the modified lyrics text segment to generate the second lyrics, thereby realizing the generation of the second lyrics; in another case, if the number of characters in the to-be-determined text segment is not equal to the number of characters in the target lyrics text segment after detection, a prompt message can be displayed to prompt the user to re-enter the modification instruction.

[0049] Due to the particularity of AI songs, when the amount of lyrics in an AI song changes, the vocal component in the AI ​​song (i.e., the vocal singing part of the song) will change, which may cause the vocal component to not match the melody of the song itself. For example, after the amount of lyrics increases, while ensuring that the melody does not change, the speed of the vocal singing will inevitably increase, resulting in a disharmony between the melody of the song and the vocal singing, affecting the quality of the final generated song. In this embodiment, in response to the above problem, when the user modifies the lyrics (after the terminal device responds to the modification instruction), the terminal device detects the number of characters in the text segment to be determined generated based on the modification instruction, and only executes the subsequent steps when it is consistent with the number of characters in the original target lyrics text segment, thereby ensuring that the number of characters in the lyrics text before and after the modification is consistent, avoiding the disharmony between the melody and the vocal singing, affecting the quality of the second song finally generated, and improving the song quality of the second song.

[0050] Further, optionally, the steps in this embodiment also include:

[0051] Step S1024: If the number of characters in the text segment to be determined is not equal to the number of characters in the target lyrics text segment, call the preset generation model to process the text segment to be determined, and generate an optimized text segment. The number of characters in the optimized text segment is equal to the number of characters in the target lyrics text segment, and has the same semantics as the text segment to be determined.

[0052] Step S1025: Replace the target lyrics text segment in the first lyrics with the optimized text segment to generate the second lyrics.

[0053] Exemplarily, in another case, if the number of characters in the to-be-determined text segment is not equal to the number of characters in the target lyrics text segment, in addition to the solution of rejecting the modification instruction and displaying a prompt message, the modification instruction input by the user can be optimized by utilizing the characteristics of generating and modifying AI songs on the same platform (i.e., the process of generating the first song and generating the second song based on the first song are both implemented by the target application), so as to obtain a text segment with the same number of characters as the target lyrics text segment and the same semantics as the to-be-determined text segment, i.e., the optimized text segment. This process can be implemented by a preset generation model, i.e., the to-be-determined text segment is processed by a preset generation model to generate an optimized text segment, wherein the preset generation model in the steps of this embodiment can be a song generation model for generating the first song and the corresponding first lyrics, and the to-be-determined text segment is optimized by utilizing the prior knowledge in the song generation model, so that the generated optimized text segment can be more consistent with the original lyrics (first lyrics), and have better musical properties, such as better sense of rhythm, so that the second song generated by the modified second lyrics has better musical quality.

[0054] Figure 5 A schematic diagram of a process for generating second lyrics provided by an embodiment of the present disclosure, such as Figure 5 As shown, first, based on the modification instruction, the target lyrics text segment (shown as Seg_1 in the figure) where the modified first text is located is determined from the first lyrics (the text is represented by "X" in the figure), and the target lyrics text segment is modified to the to-be-determined text segment (shown as Seg_2 in the figure) by modifying the first text; then, the number of characters in the to-be-determined text segment is detected, and if the number of characters in the to-be-determined text segment is equal to the number of characters in the target lyrics text segment (shown as Y path in the figure), the to-be-determined text segment is used to replace the original target lyrics text segment, thereby generating the second lyrics; if the number of characters in the to-be-determined text segment is not equal to the number of characters in the target lyrics text segment (shown as N path in the figure), the song generation model is used to process the to-be-determined text, for example, the second text is modified to generate an optimized text segment (shown as Seg_3 in the figure), and the optimized text segment is used to replace the original target lyrics text segment, thereby generating the second lyrics.

[0055] In the steps of this embodiment, when the number of characters in the text segment to be determined is not equal to the number of characters in the target lyrics text segment, the song generation module optimizes the text segment to be determined, generates an optimized text segment that better matches the original first song, and generates second lyrics based on the optimized text segment. On the one hand, it ensures that the number of characters in the lyrics before and after the modification is consistent, thereby avoiding the problem of disharmony between the melody and the vocal singing. On the other hand, it can improve the consistency of the modified second lyrics with the original first lyrics, as well as the musicality of the second lyrics themselves, so that the second song generated by the modified second lyrics has better music quality.

[0056] Step S103: In response to generation of the second lyrics, a second song is generated based on the second text, wherein the second song includes a target song segment and a non-target song segment, the target song segment is the song segment corresponding to the second text, the non-target song segment is the remaining song segment in the second song except the target song segment, and the melody of the non-target song segment is the same as the melody of the second original song segment corresponding to the playback timestamp of the non-target song segment in the first song.

[0057] Exemplarily, further, after the second lyrics are generated, that is, the lyrics of the first song are modified, the terminal device will regenerate the second lyrics based on the second text in the second lyrics. Specifically, the terminal device regenerates the corresponding original song segment (i.e., the first original song segment) in the first song through the second text to generate the corresponding target song segment, while the song segments of other parts of the first song (i.e., the second original song segment) are not adjusted and directly used as non-target song segments. Then, the target song segment and the non-target song segment are merged and combined to generate the second song. In a possible implementation method, the above steps can be implemented by a song generation model. For example, the second text, the playback timestamp corresponding to the second text, and the first song are input into the above song generation model, and the song generation model can output a second song with the above characteristics. In this process, only the song segment corresponding to the second text involved in the modification instruction is regenerated. Therefore, at least the melody of other parts of the song will not be affected, thereby achieving accurate modification of the specified lyrics in the AI ​​song.

[0058] Figure 6 A schematic diagram of a process for generating a second song provided by an embodiment of the present disclosure is shown below in combination with Figure 6 For further introduction to the above process, refer to Figure 6As shown, first, based on the modification instruction input by the user, the first text text_1 in the first lyric is determined, and the first text text_1 can be composed of one or more Chinese characters, and its content is, for example, "He looked at me and didn't speak." Then the first text text_1 is modified to the second text text_2, and the content of the second text text_2 is, for example, "He didn't look at me and didn't speak." Among them, the playback timestamp of the first text includes t1 and t2, which respectively represent the start time and end time of the first text text_1. After that, based on the second text text_2, a new song segment, namely the target song segment (shown as Seg_1 in the figure) is generated through the song generation model; on the other hand, based on the playback timestamps including t1 and t2, a non-target song segment (shown as Seg_2 in the figure) is intercepted from the first song; finally, the target song segment and the non-target song segment are combined to generate the second song.

[0059] In this embodiment, after generating the first song and the corresponding first lyrics, the first lyrics of the first song are displayed, wherein the first song is audio data generated based on artificial intelligence technology; in response to the modification instruction for the first text in the first lyrics, the second lyrics are generated, and the second lyrics contain the second text corresponding to the playback timestamp of the first text; in response to the generation of the second lyrics, the second song is generated based on the second text, wherein the second song includes a target song segment and a non-target song segment, the target song segment is the song segment corresponding to the second text, the non-target song segment is the song segment remaining in the second song except the target song segment, and the melody of the non-target song segment is the same as the melody of the second original song segment corresponding to the playback timestamp of the non-target song segment in the first song. By displaying the first lyrics of the first song, and modifying the first lyrics to the second lyrics by responding to the modification instruction, and then generating the target song segment in the second song based on the second text in the second lyrics that changes relative to the first lyrics, while the non-target song segment in the second song does not change, while achieving accurate modification of the lyrics, the melody of the song is not changed, and the flexibility and accuracy of the lyrics modification of the AI ​​song are improved, and the interaction efficiency is improved.

[0060] refer to Figure 7 , Figure 7 Schematic diagram of the AI ​​song editing method provided in the embodiment of the present disclosure Figure 2 In this embodiment Figure 2 On the basis of the illustrated embodiment, step S102-step S103 is further refined, and the AI ​​song editing method includes:

[0061] Step S201: Obtain a first song and a corresponding first lyric, wherein the first lyric includes a plurality of lyric text segments, and each lyric text segment corresponds to an original song segment.

[0062] Step S202: Displaying the lyrics text segment of the first lyrics in separate lines, and the modification control corresponding to the lyrics text segment, wherein the modification control is configured to trigger the lyrics text to an editable state.

[0063] Step S203: In response to a first instruction for modifying a control corresponding to a target lyric text segment, triggering the target lyric text segment to be in an editable state.

[0064] Step S204: In response to a second instruction for the target lyric text segment in an editable state, the first characters in the target lyric text segment are modified into second characters to generate second lyrics.

[0065] Exemplarily, after the terminal device generates the first song and the corresponding first lyrics through the target application, it displays each lyric text segment of the first lyrics in a line-by-line display manner, wherein the lyric text segment may be a paragraph or a sentence in the first lyrics. Each lyric text segment may be displayed correspondingly with the playback progress of the first song. Specifically, for example, as the first song is played or jumped, when the first song is played or jumped to the T moment of the first song, the lyric text segment where the lyrics corresponding to the T moment are located is also displayed synchronously, wherein specifically, the lyric text segment may be displayed separately, or multiple lyric text segments may be displayed, but the lyric text segment corresponding to the T moment is highlighted, and the specific implementation method is not limited.

[0066] Furthermore, when displaying each lyric text segment, a modification control is also displayed in the row where the lyric text segment is located. When the modification control is triggered, for example, when the user clicks on the modification control (triggering the first instruction), the lyric text segment in the row is triggered to be editable. After that, the user can edit the lyric text segment in the row (triggering the second instruction). When the editing is completed, for example, when the cursor leaves the row, the lyric text segment is restored to a non-editable state. In the steps of this embodiment, by setting a modification control for each lyric text segment, the editable state of the lyric text segment is controlled, thereby avoiding mis-editing and mis-modification of the lyric text segment and improving the operation efficiency in the lyric modification process.

[0067] Optionally, it also includes:

[0068] Step S204A: before and after responding to the modification instruction, display the play timestamp and / or play duration corresponding to the lyrics text segment.

[0069] Exemplarily, the playback timestamp corresponding to the lyrics text segment, for example, the playback time in the first song corresponding to the start position of the lyrics text segment, is used to characterize the position of the vocal component (vocal singing) corresponding to the lyrics text segment in the first song; and the playback duration characterizes the duration of the vocal component corresponding to the lyrics text segment in the first song. After the terminal device responds to the modification instruction, if the length of the lyrics text segment changes, it is possible that the position and duration of the vocal component corresponding to the lyrics text segment also respond to the change. In this embodiment, before and after responding to the modification instruction, the playback timestamp and / or playback duration corresponding to the lyrics text segment are displayed to show the song changes caused by the modification instruction, so that the user can control the content and duration of the finally generated song based on this information, thereby improving the interaction efficiency in the case where the lyrics change causes the song length to change.

[0070] Step S205: In response to generation of the second lyrics, a second song is generated based on the second text, wherein the second song includes a target song segment and a non-target song segment, the target song segment is the song segment corresponding to the second text, the non-target song segment is the remaining song segment in the second song except the target song segment, and the melody of the non-target song segment is the same as the melody of the original song segment corresponding to the playback timestamp of the non-target song segment in the first song.

[0071] Exemplarily, after modifying the first text and generating the second text, the terminal device will synchronously modify the content of the song in the first song based on the newly generated second text, so that the song content is consistent with the lyrics. The song content referred to here includes at least music components and vocal components, wherein the vocal component refers to the component sung by a person in the song, which is related to the lyrics. After the lyrics change, the corresponding vocal component also needs to be adjusted accordingly based on AIGC technology; and the music component refers to the various soundtracks used to express the melody of the song, harmonies that do not involve lyrics, and other components. After the lyrics change, in one implementation method, the music component does not change, that is, the song melody does not change, but only the vocal component changes. In this case, if Figure 8 As shown, the specific implementation of step S205 includes:

[0072] Step S205A-1: based on the playback timestamp of the first text, determining a first original song segment and a second original song segment in the first song, wherein the first original song segment is the original song segment corresponding to the playback timestamp of the first text, and the second original song segment is the original song segments remaining in the first song except the first original song segment;

[0073] Step S205A-2: Obtaining pronunciation data of the second text, and replacing the vocal component in the first original song segment based on the pronunciation data to generate a target song segment, wherein the melody of the target song segment is the same as the melody of the first original song segment corresponding to the playback timestamp of the target song segment in the first song;

[0074] Step S205A-3: Generate a second song based on the target song segment and the second original song segment in the first song.

[0075] Exemplarily, in the steps of this embodiment, first, the terminal device determines the first original song segment and the second original song segment in the first song based on the playback timestamp of the first text indicated by the modification instruction, wherein the first original song segment is the original song segment corresponding to the playback timestamp of the first text, and the second original song segment is the original song segment remaining in the first song except the first original song segment, that is, the first song is divided into two parts. Afterwards, the pronunciation data of the second text generated after the modification is obtained, and the speech data is pre-recorded audio data, or audio data generated by inputting the second text into a pre-trained speech model. Afterwards, the voice data of the second text replaces the vocal component in the first original song segment, thereby generating a target song segment, and this process can be implemented by a pre-trained speech model. Among them, the song melody of the target song segment is the same as the song melody of the corresponding part in the first song, that is, the music component does not change. Finally, the generated target song segment is spliced ​​with the second original song segment in the first song to generate a second song. Among them, the above steps S205A-1 and S205A-2 can be executed at one time based on the same voice pattern, that is, after the first song, the second text and the corresponding playback timestamp are input into the song generation model, the song generation model will obtain the pronunciation data corresponding to the second text by calling external data, and then replace the vocal component in the first original song segment based on the speech data to generate the target song segment. The specific implementation process will not be repeated here.

[0076] In the steps of this embodiment, the vocal component of the corresponding position in the first song is replaced by the pronunciation data of the second text, thereby generating a target song segment, and then the target song segment and the second original song segment in the first song are spliced ​​to generate a second song. The implementation method is simple to execute, and only the vocal component of the modified song segment needs to be adjusted to generate the second song. At the same time, it ensures that the melody of the generated second song is completely consistent with the melody of the first song, achieving the purpose of accurately modifying the lyrics without changing the melody while only modifying the song.

[0077] In another possible implementation, the music component of the modified song segment changes, that is, the song melody and the vocal component change at the same time. Fig. 9 As shown, the specific implementation of step S205 includes:

[0078] Step S205B-1: determining a first original song segment and a second original song segment in the first song based on the play timestamp of the first text;

[0079] Step S205B-2: calling the song generation model to generate a target song segment based on the second text and the second original song segment; wherein the melody of the target song segment is different from the melody of the first original song segment corresponding to the playback timestamp of the target song segment in the first song;

[0080] Step S205B-3: Generate a second song based on the target song segment and the second original song segment.

[0081] Exemplarily, similarly, first, the terminal device determines the first original song segment and the second original song segment in the first song based on the playback timestamp of the first text indicated by the modification instruction, and then calls the song generation model to re-"create" the second text, and at the same time, combines the second original song segment as context information to generate an audio that matches the second original song segment, that is, the target song segment. Because the song segment corresponding to the second text is regenerated in two dimensions of vocal components and music components through the song generation model, and the second text is changed from the first text, therefore, the melody of the target song segment is usually different from the melody of the first original song segment corresponding to the playback timestamp of the target song segment in the first song, but the melody of the target song segment can be made to match the modified second text more closely, and because the context information (the second original song segment) is combined, the consistency of the melody of the generated second song is also changed, thereby improving the song quality of the second song.

[0082] Furthermore, in a possible implementation, the first song and the second song belong to the same target song project, specifically, one AI song corresponds to one song project, that is, the first song and the second song can be understood as different versions of the same AI song, and both are located under the same song project. Optionally, this embodiment also includes:

[0083] Step S206: Display the generation record page corresponding to the target song project, the generation record page is used to display at least one of the following: the song generation record corresponding to the target song project, the song generation record includes at least the generation record of the first song and the generation record of the second song; the change content of the second song relative to the first song.

[0084] Exemplarily, after modifying the first song and generating the second song, the generation record page corresponding to the target song project to which the first song and the second song belong can be further displayed in the target application, and the generation record page records the song generation records corresponding to the target song project, such as the generation record of the first song and the generation record of the second song, and after further modifying the second song, the generation record of the third song is generated. The generation record may include relevant information such as the generation time and the modification user; on the other hand, the generation record page also records the changes of the modified song to the song before modification, for example, the changes of the second song relative to the first song in this embodiment, the changes include: the comparison of the first and second texts displayed; the changes in melody caused by the change of lyrics, so that the user can obtain the modification history of the target song project through the generation record page corresponding to the target song project, and restore different song versions, thereby improving the efficiency of modifying the song.

[0085] Corresponding to the AI ​​song editing method of the above embodiment, Fig.10 This is a block diagram of the structure of the AI ​​song editing device provided in the embodiment of the present disclosure. The method described in the above embodiment can be executed by the AI ​​song editing device, which can be implemented by software and / or hardware, and can be integrated in an electronic device with certain data processing functions. Among them, the electronic device may include but is not limited to mobile terminals with big data processing capabilities, as well as fixed terminals with big data processing capabilities such as desktop computers and supercomputers.

[0086] For the sake of convenience, only the parts related to the embodiments of the present disclosure are shown. Fig.10 , the AI ​​song editing device 3 includes:

[0087] A display module 31 is used to display first lyrics of the first song after generating the first song and the corresponding first lyrics, wherein the first song is audio data generated based on artificial intelligence technology;

[0088] A processing module 32, for generating a second lyric in response to a modification instruction for a first word in the first lyric, wherein the second lyric includes a second word corresponding to a playback timestamp of the first word;

[0089] A generation module 33 is used to generate a second song based on the second text in response to the generation of the second lyrics, wherein the second song includes a target song segment and a non-target song segment, the target song segment is the song segment corresponding to the second text, the non-target song segment is the song segment remaining in the second song except the target song segment, and the melody of the non-target song segment is the same as the melody of the second original song segment corresponding to the playback timestamp of the non-target song segment in the first song.

[0090] According to one or more embodiments of the present disclosure, the first lyrics include multiple lyrics text segments, each lyrics text segment corresponds to an original song segment; when displaying the first lyrics of the first song, the display module 31 is specifically used to: display the lyrics text segment of the first lyrics in lines, and the modification control corresponding to the lyrics text segment, the modification control is configured to trigger the lyrics text to an editable state; the modification instruction includes a first instruction and a second instruction, and the processing module 32 is specifically used to: in response to the first instruction of the modification control corresponding to the target lyrics text segment, trigger the target lyrics text segment to an editable state; in response to the second instruction of the target lyrics text segment in an editable state, modify the first text in the target lyrics text segment to the second text to generate second lyrics.

[0091] According to one or more embodiments of the present disclosure, the display module 31 is further used to: display the playback timestamp and / or playback duration corresponding to the lyrics text segment before and after responding to the modification instruction.

[0092] According to one or more embodiments of the present disclosure, the processing module 32 is specifically used to: obtain the corresponding text segment to be determined in response to a modification instruction for the first word in the target lyrics text segment in the first lyrics; detect the number of words in the text segment to be determined, and if the number of words in the text segment to be determined is equal to the number of words in the target lyrics text segment, determine the text segment to be determined as the modified lyrics text segment; replace the target lyrics text segment in the first lyrics with the modified lyrics text segment to generate the second lyrics.

[0093] According to one or more embodiments of the present disclosure, the processing module 32 is also used for: if the number of characters in the text segment to be determined is not equal to the number of characters in the target lyrics text segment, then calling a preset generation model to process the text segment to be determined, generating an optimized text segment, the number of characters in the optimized text segment being equal to the number of characters in the target lyrics text segment, and having the same semantics as the text segment to be determined; replacing the target lyrics text segment in the first lyrics with the optimized text segment, generating a second lyrics.

[0094] According to one or more embodiments of the present disclosure, when the generation module 33 generates the second song based on the second text, it is specifically used to: determine the first original song segment and the second original song segment in the first song based on the playback timestamp of the first text, the first original song segment being the original song segment corresponding to the playback timestamp of the first text, and the second original song segment being the original song segments remaining in the first song except the first original song segment; obtain the pronunciation data of the second text, and replace the vocal component in the first original song segment based on the pronunciation data to generate a target song segment, wherein the melody of the target song segment is the same as the melody of the first original song segment corresponding to the playback timestamp of the target song segment in the first song; generate the second song based on the target song segment and the second original song segment in the first song.

[0095] According to one or more embodiments of the present disclosure, when generating a second song based on the second text, the generation module 33 is specifically used to: determine the first original song segment and the second original song segment in the first song based on the playback timestamp of the first text; call the song generation model to generate a target song segment based on the second text and the second original song segment; wherein the melody of the target song segment is different from the melody of the first original song segment corresponding to the playback timestamp of the target song segment in the first song; and generate the second song based on the target song segment and the second original song segment.

[0096] According to one or more embodiments of the present disclosure, the first song and the second song belong to the same target song project, and the display module 31 is further used to: display a generation record page corresponding to the target song project, the generation record page is used to display at least one of the following: a song generation record corresponding to the target song project, the song generation record at least including a generation record of the first song and a generation record of the second song; changes in the second song relative to the first song.

[0097] Among them, the display module 31, the processing module 32 and the generating module 33 are connected in sequence. The AI ​​song editing device 3 provided in this embodiment can execute the technical solution of the above method embodiment, and its implementation principle and technical effect are similar, which will not be repeated in this embodiment.

[0098] Fig.11 A schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure is shown in FIG. Fig.11 As shown, the electronic device 4 includes:

[0099] A processor 41, and a memory 42 communicatively connected to the processor 41;

[0100] The memory 42 stores computer executable instructions;

[0101] The processor 41 executes the computer execution instructions stored in the memory 42 to implement the following Figure 2-Figure 9 The AI ​​song editing method in the illustrated embodiment.

[0102] Optionally, the processor 41 and the memory 42 are connected via a bus 43 .

[0103] For related instructions, please refer to Figure 2-Figure 9 The relevant descriptions and effects corresponding to the steps in the corresponding embodiments can be understood, and no further elaboration is made here.

[0104] The present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, which are used to implement the present invention when executed by a processor. Figure 2-Figure 9 The AI ​​song editing method provided in any one of the corresponding embodiments.

[0105] The present invention provides a computer program product, including a computer program. When the computer program is executed by a processor, the present invention is realized. Figure 2-Figure 9 The AI ​​song editing method provided in any one of the corresponding embodiments.

[0106] In order to implement the above embodiment, the embodiment of the present disclosure also provides an electronic device.

[0107] refer to Fig.12 , which shows a schematic diagram of the structure of an electronic device 900 suitable for implementing the embodiment of the present disclosure, and the electronic device 900 may be a terminal device or a server. The terminal device may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, personal digital assistants (PDAs), tablet computers, portable multimedia players (PMPs), vehicle terminals (such as vehicle navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Fig.12 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0108] like Fig.12 As shown, the electronic device 900 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage device 908 to a random access memory (RAM) 903. In the RAM 903, various programs and data required for the operation of the electronic device 900 are also stored. The processing device 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0109] Typically, the following devices may be connected to the I / O interface 905: input devices 906 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 907 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 908 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 909. The communication device 909 may allow the electronic device 900 to communicate with other devices wirelessly or by wire to exchange data. Although Fig.12The electronic device 900 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0110] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device 909, or installed from a storage device 908, or installed from a ROM 902. When the computer program is executed by the processing device 901, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0111] It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0112] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0113] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device executes the method shown in the above embodiment.

[0114] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0115] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0116] The units or modules involved in the embodiments described in the present disclosure may be implemented by software or hardware, wherein the name of a unit or module does not, in some cases, constitute a limitation on the unit itself.

[0117] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0118] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0119] In a first aspect, according to one or more embodiments of the present disclosure, there is provided an AI song editing method, comprising:

[0120] After generating a first song and the corresponding first lyrics, displaying the first lyrics of the first song, wherein the first song is audio data generated based on artificial intelligence technology; generating second lyrics in response to a modification instruction for a first character in the first lyrics, wherein the second lyrics contain second characters corresponding to a playback timestamp of the first characters; generating a second song based on the second characters in response to the generation of the second lyrics, wherein the second song includes a target song segment and a non-target song segment, wherein the target song segment is a song segment corresponding to the second characters, and the non-target song segment is a song segment remaining in the second song except the target song segment, and the melody of the non-target song segment is the same as the melody of the original song segment in the first song corresponding to the playback timestamp of the non-target song segment.

[0121] According to one or more embodiments of the present disclosure, the first lyrics include multiple lyrics text segments, each of which corresponds to an original song segment; the display of the first lyrics of the first song includes: displaying the lyrics text segment of the first lyrics in lines, and a modification control corresponding to the lyrics text segment, the modification control being configured to trigger the lyrics text to an editable state; the modification instruction includes a first instruction and a second instruction, and generating the second lyrics in response to the modification instruction for the first text in the first lyrics includes: in response to the first instruction for the modification control corresponding to the target lyrics text segment, triggering the target lyrics text segment to an editable state; in response to the second instruction for the target lyrics text segment in the editable state, modifying the first text in the target lyrics text segment to the second text to generate the second lyrics.

[0122] According to one or more embodiments of the present disclosure, the method further includes: displaying a playback timestamp and / or playback duration corresponding to the lyric text segment before and after responding to the modification instruction.

[0123] According to one or more embodiments of the present disclosure, the method of generating the second lyric in response to a modification instruction for the first character in the first lyric includes: obtaining a corresponding text segment to be determined in response to a modification instruction for the first character in a target lyric text segment in the first lyric; detecting the number of characters in the text segment to be determined, and if the number of characters in the text segment to be determined is equal to the number of characters in the target lyric text segment, determining the text segment to be determined as a modified lyric text segment; and replacing the target lyric text segment in the first lyric with the modified lyric text segment to generate the second lyric.

[0124] According to one or more embodiments of the present disclosure, the method further includes: if the number of characters in the text segment to be determined is not equal to the number of characters in the target lyrics text segment, calling a preset generation model to process the text segment to be determined, generating an optimized text segment, the number of characters in the optimized text segment being equal to the number of characters in the target lyrics text segment and having the same semantics as the text segment to be determined; replacing the target lyrics text segment in the first lyrics with the optimized text segment, generating the second lyrics.

[0125] According to one or more embodiments of the present disclosure, generating a second song based on the second text includes: determining a first original song segment and a second original song segment in the first song based on a playback timestamp of the first text, the first original song segment being the original song segment corresponding to the playback timestamp of the first text, and the second original song segment being the remaining original song segments in the first song except the first original song segment; obtaining pronunciation data of the second text, and replacing the vocal component in the first original song segment based on the pronunciation data to generate a target song segment, wherein the melody of the target song segment is the same as the melody of the first original song segment corresponding to the playback timestamp of the target song segment in the first song; generating the second song based on the target song segment and the second original song segment in the first song.

[0126] According to one or more embodiments of the present disclosure, generating a second song based on the second text includes: determining a first original song segment and a second original song segment in the first song based on a playback timestamp of the first text; calling a song generation model to generate a target song segment based on the second text and the second original song segment; wherein the melody of the target song segment is different from the melody of the first original song segment corresponding to the playback timestamp of the target song segment in the first song; and generating the second song based on the target song segment and the second original song segment.

[0127] According to one or more embodiments of the present disclosure, the first song and the second song belong to the same target song project, and the method further includes: displaying a generation record page corresponding to the target song project, the generation record page being used to display at least one of the following: a song generation record corresponding to the target song project, the song generation record including at least a generation record of the first song and a generation record of the second song; and changes in the second song relative to the first song.

[0128] In a second aspect, according to one or more embodiments of the present disclosure, there is provided an AI song editing device, comprising:

[0129] A display module, configured to display first lyrics of a first song after generating a first song and corresponding first lyrics, wherein the first song is audio data generated based on artificial intelligence technology;

[0130] A processing module, configured to generate a second lyric in response to a modification instruction for a first character in the first lyric, wherein the second lyric includes a second character corresponding to a playback timestamp of the first character;

[0131] A generation module is used to generate a second song based on the second text in response to the generation of the second lyrics, wherein the second song includes a target song segment and a non-target song segment, the target song segment is the song segment corresponding to the second text, the non-target song segment is the song segment remaining in the second song except the target song segment, and the melody of the non-target song segment is the same as the melody of the second original song segment corresponding to the playback timestamp of the non-target song segment in the first song.

[0132] According to one or more embodiments of the present disclosure, the first lyrics include multiple lyrics text segments, each of which corresponds to an original song segment; when displaying the first lyrics of the first song, the display module is specifically used to: display the lyrics text segments of the first lyrics in lines, and the modification controls corresponding to the lyrics text segments, and the modification controls are configured to trigger the lyrics text to an editable state; the modification instructions include a first instruction and a second instruction, and the processing module is specifically used to: in response to the first instruction of the modification control corresponding to the target lyrics text segment, trigger the target lyrics text segment to an editable state; in response to the second instruction of the target lyrics text segment in the editable state, modify the first text in the target lyrics text segment to the second text to generate the second lyrics.

[0133] According to one or more embodiments of the present disclosure, the display module is further used to: display the playback timestamp and / or playback duration corresponding to the lyrics text segment before and after responding to the modification instruction.

[0134] According to one or more embodiments of the present disclosure, the processing module is specifically used to: obtain the corresponding text segment to be determined in response to a modification instruction for the first character in the target lyrics text segment in the first lyrics; detect the number of characters in the text segment to be determined, and if the number of characters in the text segment to be determined is equal to the number of characters in the target lyrics text segment, determine the text segment to be determined as a modified lyrics text segment; replace the target lyrics text segment in the first lyrics with the modified lyrics text segment to generate the second lyrics.

[0135] According to one or more embodiments of the present disclosure, the processing module is also used for: if the number of characters in the text segment to be determined is not equal to the number of characters in the target lyrics text segment, then calling a preset generation model to process the text segment to be determined to generate an optimized text segment, the number of characters in the optimized text segment being equal to the number of characters in the target lyrics text segment and having the same semantics as the text segment to be determined; replacing the target lyrics text segment in the first lyrics with the optimized text segment to generate the second lyrics.

[0136] According to one or more embodiments of the present disclosure, when the generation module generates the second song based on the second text, it is specifically used to: determine the first original song segment and the second original song segment in the first song based on the playback timestamp of the first text, the first original song segment being the original song segment corresponding to the playback timestamp of the first text, and the second original song segment being the original song segments remaining in the first song except the first original song segment; obtain the pronunciation data of the second text, and replace the vocal component in the first original song segment based on the pronunciation data to generate a target song segment, wherein the melody of the target song segment is the same as the melody of the first original song segment corresponding to the playback timestamp of the target song segment in the first song; generate the second song based on the target song segment and the second original song segment in the first song.

[0137] According to one or more embodiments of the present disclosure, when the generation module generates the second song based on the second text, it is specifically used to: determine the first original song segment and the second original song segment in the first song based on the playback timestamp of the first text; call the song generation model to generate a target song segment based on the second text and the second original song segment; wherein the melody of the target song segment is different from the melody of the first original song segment corresponding to the playback timestamp of the target song segment in the first song; and generate the second song based on the target song segment and the second original song segment.

[0138] According to one or more embodiments of the present disclosure, the first song and the second song belong to the same target song project, and the display module is further used to: display a generation record page corresponding to the target song project, and the generation record page is used to display at least one of the following: a song generation record corresponding to the target song project, and the song generation record includes at least a generation record of the first song and a generation record of the second song; and changes in the second song relative to the first song.

[0139] In a third aspect, according to one or more embodiments of the present disclosure, there is provided an electronic device, comprising: at least one processor and a memory;

[0140] The memory stores computer-executable instructions;

[0141] The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the AI ​​song editing method described in the first aspect and various possible designs of the first aspect.

[0142] In a fourth aspect, according to one or more embodiments of the present disclosure, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer execution instructions. When a processor executes the computer execution instructions, the AI ​​song editing method described in the first aspect and various possible designs of the first aspect is implemented.

[0143] In a fifth aspect, according to one or more embodiments of the present disclosure, a computer program product is provided, including a computer program, which, when executed by a processor, implements the AI ​​song editing method described in the first aspect and various possible designs of the first aspect.

[0144] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other to form a technical solution.

[0145] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0146] Although the subject matter has been described in language specific to structural features and / or methodological logical actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims.

Claims

1. An AI song editing method, characterized in that: include: After generating a first song and corresponding first lyrics, displaying first lyrics of the first song, wherein the first song is audio data generated based on artificial intelligence technology; In response to a modification instruction for a first word in the first lyric, generating a second lyric, wherein the second lyric includes a second word corresponding to a playback timestamp of the first word; In response to the generation of the second lyrics, a second song is generated based on the second text, wherein the second song includes a target song segment and a non-target song segment, the target song segment is the song segment corresponding to the second text, the non-target song segment is the song segment remaining in the second song except the target song segment, and the melody of the non-target song segment is the same as the melody of the original song segment in the first song corresponding to the playback timestamp of the non-target song segment.

2. The method according to claim 1, characterized in that The first lyrics include a plurality of lyrics text segments, each of which corresponds to an original song segment; The displaying of the first lyrics of the first song comprises: Displaying the lyrics text segments of the first lyrics in separate lines, and a modification control corresponding to the lyrics text segments, wherein the modification control is configured to trigger the lyrics text to an editable state; The modification instruction includes a first instruction and a second instruction, and the generating of the second lyric in response to the modification instruction for the first word in the first lyric includes: In response to a first instruction for modifying a control corresponding to a target lyric text segment, triggering the target lyric text segment to be in an editable state; In response to a second instruction for the target lyric text segment in the editable state, the first characters in the target lyric text segment are modified into the second characters to generate the second lyrics.

3. The method according to claim 2, characterized in that The method further comprises: Before and after responding to the modification instruction, the play timestamp and / or play duration corresponding to the lyrics text segment is displayed.

4. The method according to claim 1, characterized in that: The step of generating a second lyric in response to a modification instruction for a first word in the first lyric comprises: In response to a modification instruction for a first character in a target lyrics text segment in the first lyrics, a corresponding to-be-determined text segment is obtained; Detecting the number of characters in the to-be-determined text segment, and if the number of characters in the to-be-determined text segment is equal to the number of characters in the target lyrics text segment, determining the to-be-determined text segment as a modified lyrics text segment; The target lyrics text segment in the first lyrics is replaced with the modified lyrics text segment to generate the second lyrics.

5. The method according to claim 4, characterized in that The method further comprises: If the number of characters in the to-be-determined text segment is not equal to the number of characters in the target lyrics text segment, calling a preset generation model to process the to-be-determined text segment to generate an optimized text segment, the number of characters in the optimized text segment being equal to the number of characters in the target lyrics text segment and having the same semantics as the to-be-determined text segment; The target lyrics text segment in the first lyrics is replaced with the optimized text segment to generate the second lyrics.

6. The method according to claim 1, characterized in that The step of generating a second song based on the second text includes: Based on the playback timestamp of the first text, determining a first original song segment and a second original song segment in the first song, wherein the first original song segment is the original song segment corresponding to the playback timestamp of the first text, and the second original song segment is the original song segments remaining in the first song except the first original song segment; Acquire pronunciation data of the second text, and replace the vocal component in the first original song segment based on the pronunciation data to generate a target song segment, wherein the melody of the target song segment is the same as the melody of the first original song segment corresponding to the playback timestamp of the target song segment in the first song; The second song is generated based on the target song segment and a second original song segment in the first song.

7. The method according to claim 1, characterized in that The step of generating a second song based on the second text includes: Determine a first original song segment and a second original song segment in the first song based on a playback timestamp of the first text; Calling a song generation model to generate a target song segment based on the second text and the second original song segment; wherein the melody of the target song segment is different from the melody of the first original song segment corresponding to the playback timestamp of the target song segment in the first song; The second song is generated based on the target song segment and the second original song segment.

8. The method according to claim 1, characterized in that The first song and the second song belong to the same target song project, and the method further includes: Display the generation record page corresponding to the target song item, wherein the generation record page is used to display at least one of the following: a song generation record corresponding to the target song item, the song generation record including at least a generation record of the first song and a generation record of the second song; The changes made to the second song relative to the first song.

9. An AI song editing device, characterized in that: include: A display module, configured to display first lyrics of a first song after generating a first song and corresponding first lyrics, wherein the first song is audio data generated based on artificial intelligence technology; A processing module, configured to generate a second lyric in response to a modification instruction for a first character in the first lyric, wherein the second lyric includes a second character corresponding to a playback timestamp of the first character; A generation module is used to generate a second song based on the second text in response to the generation of the second lyrics, wherein the second song includes a target song segment and a non-target song segment, the target song segment is the song segment corresponding to the second text, the non-target song segment is the song segment remaining in the second song except the target song segment, and the melody of the non-target song segment is the same as the melody of the second original song segment corresponding to the playback timestamp of the non-target song segment in the first song.

10. An electronic device, characterized in that: include: Processor and memory; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor executes the AI ​​song editing method as described in any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions. When the processor executes the computer-executable instructions, the AI ​​song editing method as described in any one of claims 1 to 8 is implemented.

12. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the AI ​​song editing method as described in any one of claims 1 to 8 is implemented.

Images