Lyric generation method and device, storage medium and program product

Abstract

The invention relates to a lyric generation method and device, a storage medium and a program product, and relates to the technical field of audio processing. The method comprises the following steps: acquiring a first song, wherein the first song is a song lacking lyrics; acquiring a reference song, wherein the reference song has lyrics, and the similarity between the reference song and the first song meets a reference condition; and generating the lyric text of the first song by taking the lyric text of the reference song as a reference, thereby improving the accuracy of generating the lyric text.

Description

Lyrics generation methods, devices, storage media, and software products Technical Field

[0001] This application relates to the field of audio processing technology, and in particular to a lyric generation method, device, storage medium, and program product. Background Technology

[0002] With the rapid development of internet technology, audio-based text generation technology has gradually matured.

[0003] In related technologies, audio can be converted into acoustic features using automatic speech recognition technology. These acoustic features are then input into a trained machine learning model to obtain a text sequence output by the model. This text sequence is the text corresponding to the audio. This approach can be applied to lyric generation scenarios, such as obtaining the lyrics corresponding to a song's audio using speech recognition technology.

[0004] However, the above-mentioned solutions rely on a relatively simple audio-based lyric generation method, lack optimization processing for the generated lyrics, and have a low accuracy rate. Summary of the Invention

[0005] This application provides a lyrics generation method, device, storage medium, and program product, which can improve the accuracy of lyrics generation. The technical solution is as follows: On one hand, a lyrics generation method is provided, the method comprising: obtaining a first song, wherein the first song is a song lacking lyrics; obtaining a reference song, wherein the reference song is a song having lyrics and whose similarity to the first song meets reference conditions; and generating lyrics text of the first song with reference to the lyrics text of the reference song.

[0006] On the other hand, a lyrics generation device is provided, the device comprising: a first song acquisition module for acquiring a first song, wherein the first song is a song lacking lyrics; a reference song acquisition module for acquiring a reference song, wherein the reference song has lyrics and its similarity to the first song meets reference conditions; and a lyrics text generation module for generating lyrics text of the first song with reference to the lyrics text of the reference song.

[0007] In one possible implementation, the lyrics text generation module is configured to: perform speech recognition processing on the first song with reference to the lyrics text of the benchmark song to obtain candidate lyrics text of the first song; and obtain the lyrics text of the first song based on the candidate lyrics text and the lyrics text of the benchmark song.

[0008] In one possible implementation, the lyrics text generation module is configured to: when performing speech recognition processing on the first song, use the lyrics text of the benchmark song as a reference text mapping table, obtain a first lyrics text from the reference text mapping table, wherein the first lyrics text is a lyrics text in the reference text mapping table whose pronunciation format is the same as that of the first song; if the first lyrics text is not obtained from the reference text mapping table, obtain a second lyrics text from a model mapping table, wherein the second lyrics text is a lyrics text in the model mapping table whose pronunciation format is the same as that of the first song, wherein the model mapping table is a mapping table of the speech recognition processing system that contains the mapping relationship between pronunciation format and text; and concatenate the first lyrics text and the second lyrics text in chronological order to obtain the concatenation result as the candidate lyrics text.

[0009] In one possible implementation, the lyrics text generation module is configured to: divide the candidate lyrics text into two or more candidate sub-segments; divide the lyrics text of the benchmark song into two or more sub-lyric texts, wherein the number of characters in the sub-lyric texts is the same as the number of characters in the candidate sub-segments; calculate the text similarity between each candidate sub-segment and each sub-lyric text; and obtain the lyrics text of the first song based on the text similarity.

[0010] In one possible implementation, the lyrics text generation module is configured to: obtain the sub-lyric text with the highest text similarity to the candidate sub-segment; replace the candidate sub-segment in the candidate lyrics text with the sub-lyric text with the highest text similarity to the candidate sub-segment; and obtain the replaced candidate lyrics text as the lyrics text of the first song.

[0011] In one possible implementation, the first song acquisition module is used to acquire songs lacking lyrics from a song set as the first song, wherein the song set contains multiple songs and the similarity between the multiple songs satisfies the reference condition; the benchmark song acquisition module is used to acquire the song with the highest number of plays and lyrics from the song set as the benchmark song.

[0012] In one possible implementation, the device further includes: an information acquisition module, configured to acquire playback information and feature information of at least two songs before acquiring the first song, wherein the at least two songs include songs with lyrics and songs without lyrics, the playback information includes at least one of play count and release time, and the feature information includes at least one of audio fingerprint and lyrics text; and a song addition module, configured to determine a target song from the at least two songs and add it to the song set based on the playback information and the feature information.

[0013] In one possible implementation, the song adding module is configured to: acquire songs from the at least two songs whose play count is greater than a threshold and / or whose release time is earlier than a specified time, and add them to the song set as reference songs; determine the similarity between a second song and the reference songs based on the feature information of the at least two songs, wherein the second song is a song other than the reference song from the at least two songs; and add the second song as a target song to the song set when the similarity between the second song and the reference song is greater than a similarity threshold.

[0014] In one possible implementation, where the feature information includes lyrics text, and both the second song and the reference song have lyrics text, the song adding module is configured to calculate the similarity between the lyrics text of the second song and the lyrics text of the reference song using an edit distance algorithm.

[0015] In another aspect, a computer device is provided, the computer device comprising a processor and a memory, the memory storing at least one instruction, at least one program, code set or instruction set, the at least one instruction, the at least one program, the code set or instruction set being loaded and executed by the processor to implement the lyrics generation method as described above.

[0016] In another aspect, a computer-readable storage medium is provided, wherein at least one instruction, at least one program, code set, or instruction set is stored therein, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement the lyrics generation method described above.

[0017] In another aspect, a computer program product is provided, comprising a computer program stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium and executes the computer program, causing the computer device to perform the lyrics generation method provided in the various optional implementations described above.

[0018] The technical solution provided in this application can include the following beneficial effects: The computer device obtains a first song lacking lyrics and a benchmark song with lyrics that meets the reference conditions for similarity with the first song, and then uses the lyrics of the benchmark song as a reference to generate the lyrics of the first song. Since the benchmark song meets the reference conditions for similarity with the first song, that is, the feature similarity between the benchmark song and the first song is high in a certain dimension, the features of the lyrics of the benchmark song have reference value for generating the lyrics of the first song. The lyrics of the benchmark song can effectively ensure the reliability of the reference lyrics and greatly improve the accuracy of the generated lyrics of the first song.

[0019] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0020] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0021] Figure 1 is a system configuration diagram of a lyrics generation system according to an embodiment of this application; Figure 2 is a flowchart of a lyrics generation method provided in an embodiment of this application; Figure 3 is a flowchart of a lyrics generation method provided in an embodiment of this application; Figure 4 is a schematic diagram of a lyrics text replacement method provided in an embodiment of this application; Figure 5 is a flowchart of a lyrics generation method provided in an embodiment of this application; Figure 6 is a system diagram of lyrics generation provided in an embodiment of this application; Figure 7 is a flowchart of a lyrics generation optimization method provided in an embodiment of this application; Figure 8 is a schematic diagram of lyrics matching provided in an embodiment of this application; Figure 9 is a flowchart of lyrics optimization generation provided in an embodiment of this application; Figure 10 is a flowchart of lyrics optimization generation provided in an embodiment of this application; Figure 11 is a block diagram of a lyrics generation apparatus provided in an exemplary embodiment of this application; Figure 12 is a structural schematic diagram of a computer device provided in an exemplary embodiment of this application. Detailed Implementation

[0022] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0023] It should be noted that this application may display a prompt interface, pop-up window, or output voice prompts before and during the collection of user-related data (such as user-uploaded audio). These prompt interfaces, pop-ups, or voice prompts are used to inform the user that their data is being collected. This ensures that the application only begins the steps for collecting user-related data after receiving confirmation from the user regarding the prompt interface or pop-up window; otherwise (i.e., without user confirmation), the steps for collecting user-related data end, meaning no user-related data is collected. In other words, all user data collected in this application is collected with the user's consent and authorization, and the collection, use, and processing of related user data must comply with the relevant laws, regulations, and standards of the relevant countries and regions.

[0024] Figure 1 is a system configuration diagram of a lyrics generation system according to an embodiment of this application. As shown in Figure 1, the system includes a requesting device 120 and a service device 130 for lyrics generation.

[0025] The requesting device 120 can be a personal computer device such as a desktop computer, a laptop computer, or a personal workstation.

[0026] The service provider equipment 130 can be a single server, or a combination of several servers, or a virtualization platform, or a cloud computing service center.

[0027] The requesting device 120 and the serving device 130 can be connected via a communication network. Optionally, this communication network can be a wired network or a wireless network.

[0028] Optionally, the aforementioned wireless or wired networks use standard communication technologies and / or protocols. The network is typically the Internet, but can also be any network, including but not limited to Local Area Networks (LANs), Metropolitan Area Networks (MANs), Wide Area Networks (WANs), mobile, wired or wireless networks, private networks, or any combination of virtual private networks. In some embodiments, technologies and / or formats including Hypertext Markup Language (HTML), Extensible Markup Language (XML), etc., are used to represent data exchanged over the network. Furthermore, conventional encryption technologies such as Secure Socket Layer (SSL), Transport Layer Security (TLS), Virtual Private Networks (VPNs), and Internet Protocol Security (IPsec) can be used to encrypt all or some links. In other embodiments, custom and / or dedicated data communication technologies can be used to replace or supplement the aforementioned data communication technologies.

[0029] In the system shown in Figure 1, the requesting device 120 can provide a song with missing lyrics to the service device 130. After receiving the song with missing lyrics, the service device 130 generates lyrics text and sends the generated lyrics text to the requesting device 120.

[0030] Specifically, the user can upload a first song (a song lacking lyrics) to the requesting device 120. This first song can be a cover version by the user, and includes, but is not limited to, instrumental music, backing tracks, unfinished song drafts, and cover versions. The requesting device 120 sends the first song to the service device 130. The service device 130 obtains a base song (a song with lyrics that meets the reference criteria for similarity to the first song). Using the lyrics of the base song as a reference, the service device 130 generates the lyrics for the first song and sends the generated lyrics to the requesting device 120.

[0031] Figure 2 is a flowchart of a lyrics generation method provided in an embodiment of this application. This lyrics generation method can be executed by a computer device, such as the service provider device 130 shown in Figure 1 above, or other computer devices such as a server, or a personal computer, etc. The lyrics generation method may include the following steps: Step 210: Obtain a first song, where the first song is a song lacking lyrics.

[0032] The aforementioned songs lacking lyrics refer to songs that have audio but lack corresponding lyrics.

[0033] Specifically, the first song can be a song that lacks all the lyrics corresponding to the audio, or a song that lacks only some of the lyrics corresponding to the audio.

[0034] The first song mentioned above can be a song uploaded by a user or loaded from the cloud by a computer device.

[0035] Step 220: Obtain the benchmark song, which is a song with lyrics and whose similarity to the first song meets the reference conditions.

[0036] The similarity between the songs other than the first song and the first song meets the reference condition can be that the similarity between the first song and the other songs is greater than a similarity threshold, for example, the similarity between the first song and the other songs is greater than 90%. Alternatively, the songs that meet the reference condition can also be the N songs with the highest similarity obtained after sorting the first song with multiple other songs from high to low similarity, where N is an integer greater than or equal to 1. For example, after sorting the first song with two other songs from high to low similarity, the song with the highest similarity is obtained as the song that meets the reference condition for similarity with the first song.

[0037] The aforementioned similarity can be determined by a computer device based on specified dimensional features of the song. For example, the similarity can specifically be the audio fingerprint similarity of the song. In this embodiment, the computer device can calculate the similarity using a similarity calculation formula, which includes, but is not limited to, the cosine similarity formula, the Euclidean distance similarity formula, and the Hamming distance similarity formula.

[0038] The aforementioned audio fingerprint is a unique digital feature extracted from an audio signal using a specific algorithm. It is similar to an audio "barcode" and can be used to identify the same or similar audio content.

[0039] When the similarity is the similarity of the audio fingerprints of the songs, the computer device extracts the audio fingerprints of the first song and the other songs respectively, and calculates the similarity between the audio fingerprint of the first song and the audio fingerprints of the other songs using the similarity calculation formula.

[0040] Specifically, computer equipment can use audio processing technology to extract audio features from the first song and other songs. Based on the audio features of the first song and other songs, binary fingerprint sequences corresponding to the first song and other songs are generated through local sensitive hashing or a set of minimum values ​​based on hashing. After obtaining the binary fingerprint sequences corresponding to the first song and other songs, the similarity between the binary fingerprint sequences corresponding to the first song and other songs is calculated as the audio fingerprint similarity between the first song and other songs.

[0041] Step 230: Using the lyrics of the benchmark song as a reference, generate the lyrics of the first song.

[0042] The above reference to the lyrics of a benchmark song indicates that when the computer device generates the lyrics of the first song, it refers to the lyrics of the benchmark song. Specifically, it may refer to at least one of the text content, structure, format, and meaning of the lyrics of the benchmark song.

[0043] In some embodiments, the computer device may incorporate the lyrics of a benchmark song as input or update conditions for the lyrics generation model during the process of generating the lyrics of a first song.

[0044] In this embodiment, the computer device can directly integrate the lyrics of a benchmark song into the operation of the lyrics generation model, so that the lyrics of the benchmark song serve as the core context or constraint that the lyrics generation model references or follows when generating lyrics.

[0045] Optionally, when the lyrics of a benchmark song are used as input, the computer device uses the lyrics of the benchmark song and the first song as input to the lyrics generation model, and obtains the lyrics of the first song generated by the lyrics generation model based on the lyrics of the benchmark song and the first song.

[0046] Optionally, when using the lyrics of a benchmark song as the update condition, after the computer device generates the basic lyrics text through the lyrics generation model (i.e., the lyrics text corresponding to the first song that has not yet been updated), the optimizer in the lyrics generation model performs an optimization operation on the basic lyrics text based on the lyrics of the benchmark song. Specifically, the optimization operation may be to find the text with the highest relevance to the basic lyrics text from the lyrics of the benchmark song and update it as the lyrics text of the first song.

[0047] In other embodiments, the computer device may use the lyrics of a reference song as a preliminary screening criterion or a final screening criterion before generating the lyrics of a first song.

[0048] In this embodiment, the computer device does not use the lyrics of the benchmark song to guide the generation process, but instead uses the lyrics of the benchmark song as an external, independent evaluation criterion.

[0049] For example, the computer device first generates a batch of candidate lyrics using a lyrics generation algorithm, and then further filters them based on the similarity between the candidate lyrics and the lyrics of the benchmark song, retaining the candidate lyrics with a similarity greater than a specified similarity with the lyrics of the benchmark song as the lyrics of the first song.

[0050] In the above scheme, the computer device acquires a first song lacking lyrics and a benchmark song with lyrics that meets the reference conditions for similarity with the first song. Then, it uses the lyrics of the benchmark song as a reference to generate the lyrics of the first song. Since the benchmark song meets the reference conditions for similarity with the first song, that is, the benchmark song and the first song have high feature similarity in a certain dimension, the features of the lyrics of the benchmark song have reference value for generating the lyrics of the first song. The lyrics of the benchmark song can effectively ensure the reliability of the reference lyrics and greatly improve the accuracy of the generated lyrics of the first song.

[0051] Based on the solutions shown in one or more embodiments corresponding to Figure 2 above, and referring to Figure 3, a flowchart of a lyrics generation method provided by an embodiment of this application is shown. As shown in Figure 3, the above step 230 can be implemented as steps 230a and 230b, specifically as follows: Step 230a: Taking the lyrics text of a reference song as a reference, perform speech recognition processing on the first song to obtain candidate lyrics text of the first song.

[0052] The above-mentioned speech recognition processing is the operation of converting natural speech signals into text. Specifically, the above-mentioned speech recognition processing can be implemented through ASR (Automatic Speech Recognition) technology.

[0053] In some embodiments, the computer device may first perform speech recognition processing on a first song, and then, using the lyrics of a benchmark song as a reference, further perform correction processing on the result after performing speech recognition processing.

[0054] In this embodiment, the computer device performs ASR processing on the audio signal of the first song, uses the lyrics text of the reference song as an external knowledge base, and performs post-processing correction on the basic lyrics text identified after performing ASR processing.

[0055] For example, computer equipment can use a language model to fill in the text gaps in the basic lyrics text based on the lyrics text of a benchmark song, and correct the basic lyrics text identified after ASR processing into a candidate lyrics text of a first song with a reasonable structure and coherent content.

[0056] In other embodiments, the computer device may, when performing speech recognition processing on the first song, combine the lyrics of a reference song to obtain candidate lyrics for the first song.

[0057] In this embodiment, the computer device can integrate the lyrics of the reference song into the decoding stage of the recognition model during the speech recognition processing of the first song.

[0058] For example, by adding the lyrics of a benchmark song into the decoder of the ASR model, the ASR model can directly guide the speech recognition process to generate lyrics that are similar in style and content to the benchmark song when performing speech recognition on the first song. This allows the candidate lyrics output by the ASR model to inherit the linguistic features of the benchmark song's lyrics.

[0059] Step 230b: Based on the candidate lyrics text and the lyrics text of the benchmark song, obtain the lyrics text of the first song.

[0060] In some embodiments, a computer device may generate lyrics for a first song based on candidate lyrics and the lyrics of a benchmark song, using a lyrics generation model.

[0061] For example, the lyrics of the first song, candidate lyrics, and the lyrics of the benchmark song are input into the lyrics generation model. The lyrics generation model is a machine learning model that has the ability to generate the lyrics of the first song based on the lyrics of the first song, candidate lyrics, and the lyrics of the benchmark song.

[0062] In other embodiments, the computer device may obtain the lyrics of a first song based on the lyrics of a reference song and in combination with candidate lyrics.

[0063] For example, calculate the sub-similarity between each part of the candidate lyrics text and each sub-lyric text of the base song's lyrics text, replace part of the text in the candidate lyrics text with the base song's lyrics text whose sub-similarity is higher than the similarity threshold, and finally obtain the first song.

[0064] In this embodiment, the computer device incorporates the lyrics of a benchmark song into the speech recognition process, thereby enhancing the speech recognition model's ability to predict the lyrics content. This makes the obtained candidate lyrics more closely resemble the actual needs of the first song in terms of semantics and syllable structure, effectively reducing noise interference caused by isolated speech recognition. This enables the speech recognition system to more intelligently infer lyrics fragments that conform to the melody direction and language habits, thus improving the accuracy of obtaining the lyrics of the first song.

[0065] Based on the solutions shown in one or more of the above embodiments, in one possible implementation, step 230a can be implemented as follows: when performing speech recognition processing on the first song, the lyrics text of the benchmark song is used as a reference text mapping table, and the first lyrics text is obtained from the reference text mapping table. The first lyrics text is the lyrics text in the reference text mapping table whose pronunciation format is the same as that of the first song. If the first lyrics text is not obtained from the reference text mapping table, the second lyrics text is obtained from the model mapping table. The second lyrics text is the lyrics text in the model mapping table whose pronunciation format is the same as that of the first song. The model mapping table is a mapping table of the speech recognition processing system that contains the mapping relationship between pronunciation format and text. The first lyrics text and the second lyrics text are concatenated in chronological order, and the concatenation result is obtained as the candidate lyrics text.

[0066] In some embodiments, the first song contains two or more audio segments. For each audio segment in the first song, the computer device first obtains the first lyrics text corresponding to the audio segment from a reference text mapping table. If the first lyrics text is not obtained from the reference text mapping table, the second lyrics text corresponding to the audio segment is obtained from a model mapping table. Finally, the first lyrics text or the second lyrics text of each audio segment is concatenated according to the playback time order of each audio segment in the first song to obtain the complete candidate lyrics text of the first song.

[0067] In some embodiments, the aforementioned audio segment corresponds to one or more pronunciation formats.

[0068] The aforementioned reference text mapping table contains the mapping relationship between each character in the lyrics text of the benchmark song and its pronunciation format. One character can correspond to multiple pronunciation formats, and one pronunciation format can correspond to multiple characters.

[0069] Specifically, when the computer device performs speech recognition processing on the first song, it needs to convert the audio of the first song into text, that is, convert a pronunciation format into a character. In some embodiments, a pronunciation format can be converted into multiple characters with different shapes but the same pronunciation. At this time, the computer device selects the character corresponding to the pronunciation format from the reference text mapping table. If the pronunciation format has multiple corresponding characters in the reference text mapping table, the computer device refers to the converted context lyrics of the first song and selects the character with the best semantic match.

[0070] The aforementioned model mapping table contains different pronunciation formats and one or more characters corresponding to each pronunciation format. In this embodiment, when the computer device performs speech recognition processing on the first song, that is, when converting the pronunciation format corresponding to the first song into text, it first selects the text corresponding to the pronunciation format from the reference text mapping table. If the first lyrics text is not obtained from the reference text mapping table, the second lyrics text is further obtained from the model mapping table.

[0071] In some embodiments, after acquiring the first lyrics text and the second lyrics text, the computer device can concatenate the first lyrics text and the second lyrics text according to the pronunciation format and the time sequence in which they are played in the first song.

[0072] For example, the audio of the first song is 3 seconds long. Each second of audio corresponds to one pronunciation format, meaning that each second of audio can be converted into one word. The computer device converts the first second of the audio of the first song into the first lyric text 1, the second second of the audio into the first lyric text 2, and the third second of the audio into the second lyric text 1. At this time, the computer device concatenates the first lyric text and the second lyric text according to the audio-to-text time order of the first song, resulting in: "First lyric text 1 + First lyric text 2 + Second lyric text 1".

[0073] In this embodiment, the computer device fully utilizes highly relevant words (reference text mapping table) within the specific song context for priority matching, improving the accuracy of lyric text recognition. Simultaneously, it retains text from the model mapping table of the general model to ensure recognition integrity. Furthermore, the chronological order of the lyrics ensures logical continuity and avoids misaligned sentence breaks. By constructing a dual mapping mechanism—prioritizing the use of the lyrics text of the benchmark song as the reference text mapping table, and then backtracking to the mapping table of the general speech recognition model when the first lyric text cannot be obtained from the reference text mapping table—the accuracy of obtaining candidate lyric text for the first song is improved.

[0074] Based on the solutions shown in one or more of the above embodiments, in one possible implementation, step 230b can be implemented as follows: dividing the candidate lyrics text into two or more candidate sub-segments; dividing the lyrics text of the benchmark song into two or more sub-lyric texts, wherein the number of characters in the sub-lyric texts is the same as the number of characters in the candidate sub-segments; calculating the text similarity between each candidate sub-segment and each sub-lyric text; and obtaining the lyrics text of the first song based on the text similarity.

[0075] The aforementioned candidate sub-paragraphs are a portion of the candidate lyrics text, and the aforementioned sub-lyrics text is a portion of the lyrics text of the benchmark song.

[0076] In some embodiments, the computer device can calculate the text similarity between each candidate sub-segment and each sub-lyric text using a similarity formula. That is, a candidate sub-segment can have multiple calculated text similarities, and each text similarity represents the similarity between the candidate sub-segment and a sub-lyric text.

[0077] In this embodiment, the computer device divides the candidate lyrics text and the base song lyrics text according to the number of characters, that is, the number of characters corresponding to the divided candidate sub-segments and the sub-lyrics texts are the same.

[0078] In other embodiments, candidate sub-segments can be obtained by a computer device randomly dividing candidate lyrics text, or by a computer device dividing candidate lyrics text according to specified conditions, including but not limited to specified duration (e.g., every 10 seconds), song structure (e.g., intro, interlude, chorus, verse), and syllables (e.g., every 8 syllables). Similarly, sub-lyric text can be obtained by a computer device randomly dividing the lyrics text of a base song, or by a computer device dividing the lyrics text of a base song according to specified conditions, including but not limited to specified duration (e.g., every 10 seconds), song structure (e.g., intro, interlude, chorus, verse), and syllables (e.g., every 8 syllables).

[0079] In this embodiment, the computer device divides the long text into sub-segments of equal word count, which not only maintains the rhythmic consistency of the divided lyrics text, but also facilitates local comparison between the candidate lyrics text and the lyrics text of the benchmark song, avoiding the global similarity from masking local deviations. This effectively solves the problems of inappropriate word choice and grammatical inconsistencies that may exist in speech recognition output, and improves the language quality of the generated lyrics by using high-quality benchmark lyrics for semantic guidance.

[0080] Based on the solutions shown in one or more of the above embodiments, in one possible implementation, the computer device obtains the sub-lyrics text with the highest text similarity to the candidate sub-segment; replaces the candidate sub-segment in the candidate lyrics text with the sub-lyrics text with the highest text similarity to the candidate sub-segment; and obtains the replaced candidate lyrics text as the lyrics text of the first song.

[0081] In some embodiments, the computer device selects the highest text similarity from multiple text similarities for each candidate sub-segment and obtains the sub-lyrics text corresponding to the highest text similarity. The highest text similarity indicates that the sub-lyrics text is more likely to be the same as the candidate sub-segment in a specified dimension. For example, the highest text similarity indicates that the sub-lyrics text is more likely to have the same text form as the candidate sub-segment.

[0082] The above-mentioned replacement with the sub-lyric text with the highest text similarity to the candidate sub-paragraph specifically means replacing the candidate sub-paragraph in the candidate lyric text with the sub-lyric text with the highest text similarity, that is, replacing the content of the candidate sub-paragraph in the candidate lyric text with the sub-lyric text with the text with the highest text similarity to the candidate sub-paragraph.

[0083] For example, please refer to Figure 4, which shows a schematic diagram of a lyrics text replacement method provided in an embodiment of this application.

[0084] Please refer to part (a) of Figure 4: The computer device divides the candidate lyrics text into 4 candidate sub-segments (A1~A4); the lyrics text of the base song is divided into 4 sub-lyrics texts (B1~B4), the number of words in the sub-lyrics texts is the same as the number of words in the candidate sub-segments; the text similarity between each candidate sub-segment and each sub-lyrics text is calculated; taking candidate sub-segment A2 as an example, the text similarity between candidate sub-segment A2 and sub-lyrics texts B1~B4 is calculated respectively. Specifically, the text similarity between candidate sub-segment A2 and sub-lyrics text B1 is 80%, the text similarity between candidate sub-segment A2 and sub-lyrics text B2 is 75%, the text similarity between candidate sub-segment A2 and sub-lyrics text B3 is 90%, and the text similarity between candidate sub-segment A2 and sub-lyrics text B4 is 98%.

[0085] Please refer to part (b) of Figure 4: The computer device obtains the sub-lyric text B4 with the highest text similarity to the candidate sub-segment A2, replaces the candidate sub-segment A2 in the candidate lyric text with the sub-lyric text B4 with the highest text similarity to the candidate sub-segment A2; and obtains the replaced candidate lyric text as the lyric text of the first song.

[0086] In this embodiment of the application, the computer device optimizes and enhances the semantics of the preliminary recognition results by replacing candidate sub-segments with the benchmark sub-lyrics with the highest text similarity. The replacement mechanism ensures that the replaced lyrics not only conform to the rhythm of the original song but also have higher language quality. Compared with directly outputting the recognition results of audio-to-text conversion, it significantly reduces problems such as misspellings and neologisms, and improves the quality of the generated lyrics.

[0087] Based on the solutions shown in one or more embodiments corresponding to Figure 2 above, in some embodiments, referring to Figure 5, a flowchart of a lyrics generation method provided by an embodiment of this application is shown. As shown in Figure 5, step 210 above can be implemented as step 210a, and step 220 above can be implemented as step 220a, specifically as follows: Step 210a: Obtain songs lacking lyrics from the song set as first songs. The song set contains multiple songs, and the similarity between the multiple songs meets the reference conditions.

[0088] The above song collection is a set of multiple songs, including songs without lyrics and songs with lyrics.

[0089] Step 220a: Select the song with the highest number of plays and lyrics from the song collection as the baseline song.

[0090] In some embodiments, the number of plays for the song can be the number of plays for the song on the music streaming platform within a specified time period, such as the number of plays for the song on the music streaming platform in the past week.

[0091] In some embodiments, the number of plays for the aforementioned song may also be the number of plays after the computer device loads the song from the cloud to its local storage.

[0092] In some embodiments, the number of plays for the aforementioned song may also be the total number of plays for the song on the song playback platform since its release.

[0093] In some embodiments, each song in the song collection has a corresponding play count identifier and a lyrics identifier. The play count identifier indicates the number of times a song has been played, and the lyrics identifier indicates whether the song has lyrics. The computer device can determine the play count and whether there are lyrics for each song in the song collection based on the play count identifier and the lyrics identifier, and then select the song with the highest play count and lyrics from the song collection as the benchmark song.

[0094] In this embodiment, play count reflects the social acceptance of a song. Selecting a song with high play count as a benchmark song indicates that the lyrics of that song are more in line with popular aesthetics and singing patterns. At the same time, limiting the similarity conditions among songs within the set ensures a high degree of matching in musical style and rhythmic structure, making lyric transfer more feasible. The above scheme avoids the style mismatch problem caused by blindly matching songs. The computer device selects the song with the highest play count and lyrics from the set of similar songs as the benchmark, ensuring that the selected benchmark song has both high popularity and high-quality lyrics, thus improving the quality of the obtained benchmark song.

[0095] Based on the scheme shown in one or more embodiments corresponding to Figure 2 above, in one possible implementation, before obtaining the first song, the computer device obtains playback information and feature information of at least two songs respectively. The at least two songs include songs with lyrics and songs without lyrics. The playback information includes at least one of playback count and release time, and the feature information includes at least one of audio fingerprint and lyrics text. Based on the playback information and feature information, the target song is determined from the at least two songs and added to the song set.

[0096] The release time mentioned above refers to the song's release time, which can be either the release date or the release time.

[0097] The aforementioned audio fingerprint is a digital feature extracted from an audio signal by a computer device using a specific algorithm, which can be used to uniquely identify the same or similar audio content.

[0098] In this embodiment, play count and release time reflect a song's social influence and timeliness, helping to prioritize popular or classic tracks. Audio fingerprints and lyrics provide objective musical and semantic features for quantifying the similarity between songs. The computer device constructs a song set by filtering target songs through both play information and feature information, effectively filtering out noisy data and low-quality songs. This creates a well-structured and highly correlated candidate pool, laying the foundation for accurate matching of benchmark songs and significantly improving the data quality of subsequent lyrics generation tasks.

[0099] Based on the scheme shown in one or more embodiments corresponding to Figure 2 above, in one possible implementation, at least two songs with a play count greater than a threshold and / or a release time earlier than a specified time are acquired as reference songs and added to the song set; based on the feature information of each of the at least two songs, the similarity between the second song and the reference song is determined, wherein the second song is the song other than the reference song among the at least two songs; when the similarity between the second song and the reference song is greater than a similarity threshold, the second song is added to the song set as the target song.

[0100] The above quantity threshold is a preset quantity, and the specified time is a preset historical moment.

[0101] The reference songs mentioned above can be songs with more play counts than the specified number of play counts among at least two songs, songs with a release date earlier than the specified time among at least two songs, or songs with more play counts than the specified number of play counts and a release date earlier than the specified time among at least two songs.

[0102] After the computer device identifies a reference song from at least two songs, the remaining songs are obtained as the second song. Based on the feature information of the second song and the reference song, the similarity between the second song and the reference song is determined.

[0103] Specifically, the aforementioned feature information includes at least one of the audio fingerprint and the lyrics text.

[0104] The computer device can calculate the similarity between the audio fingerprint of the second song and the audio fingerprint of the reference song, and it can also calculate the similarity between the lyrics of the second song and the lyrics of the reference song (when both the second song and the reference song have lyrics). After calculating the first similarity between the audio fingerprint of the second song and the audio fingerprint of the reference song, it can further calculate the second similarity between the lyrics of the second song and the lyrics of the reference song, and combine the first similarity and the second similarity to calculate the final similarity between the second song and the reference song.

[0105] In some embodiments, a computer device may calculate the similarity between a second song and a reference song based on the feature information of at least two songs using a similarity calculation formula.

[0106] The aforementioned similarity threshold is a relatively high threshold, for example, 95%. In this embodiment, if the similarity between the second song and the reference song is greater than the similarity threshold, it means that the second song and the reference song have a high similarity in the feature dimension. In this case, using the second song as the target song has high reliability.

[0107] In this embodiment, the computer device prioritizes including songs with high play counts or those released early as reference songs, ensuring the authority and representativeness of the benchmark source; further, it calculates the similarity between the remaining songs and the reference songs based on feature information, and adds the songs to the song set when the similarity exceeds a specified threshold, effectively controlling data noise and irrelevant interference, and achieving efficient and accurate target song mining.

[0108] Based on the scheme shown in one or more embodiments corresponding to Figure 2 above, in one possible implementation, when the feature information includes lyrics text and both the second song and the reference song have lyrics text, the similarity between the lyrics text of the second song and the lyrics text of the reference song is calculated by an edit distance algorithm.

[0109] The edit distance algorithm described above is an algorithm that calculates the minimum number of single-character edit operations required to transform one string into another.

[0110] In some embodiments, the computer device may obtain the string length of the lyrics text of the second song and the string length of the lyrics text of the reference song, calculate the edit distance between the two strings, and perform normalization processing on the calculated edit distance, that is, map the calculated edit distance to the interval [0, 1] as the similarity between the lyrics text of the second song and the lyrics text of the reference song. The closer the mapped edit distance is to 1, the higher the similarity between the lyrics text of the second song and the lyrics text of the reference song; conversely, the closer the mapped edit distance is to 0, the lower the similarity between the lyrics text of the second song and the lyrics text of the reference song.

[0111] When the feature information includes lyrics text, the computer device uses the edit distance algorithm to quantify the similarity between the second song and the reference song. The edit distance can effectively measure the degree of difference between the two lyrics texts at the character level. The difference at the character level can reflect the similarity between the lyrics texts of the second song and the lyrics texts of the reference song in terms of word choice, sentence structure, and rhyme structure. Calculating the similarity between the two lyrics texts using the edit distance algorithm can effectively improve the accuracy of similarity calculation. Based on any one or more embodiments corresponding to Figures 2 to 5, please refer to Figure 6, which shows a system diagram of lyrics generation provided by an embodiment of this application. As shown in Figure 6, the lyrics generation scheme shown in this embodiment includes four parts: a song set construction part, a song acquisition part, a candidate lyrics text acquisition part, and a target lyrics text generation part, which are described in detail below: In the song set construction part, the computer device acquires the playback information (including at least one of playback volume and release time) and feature information (including at least one of audio fingerprint and lyrics text) of at least two songs. The at least two songs include songs with lyrics and songs without lyrics. Based on the playback information and feature information, the target song is determined from the at least two songs and added to the song set.

[0112] Specifically, the computer device can acquire at least two songs whose play counts exceed a threshold and / or whose release time is earlier than a specified time as reference songs and add them to the song set; determine the similarity between the second song and the reference songs based on the feature information of each of the at least two songs, wherein the second song is the song other than the reference song among the at least two songs; when the similarity between the second song and the reference song is greater than the similarity threshold, add the second song as the target song to the song set.

[0113] Furthermore, when the feature information includes lyrics text, and both the second song and the reference song have lyrics text, the computer device can calculate the similarity between the lyrics text of the second song and the lyrics text of the reference song using an edit distance algorithm.

[0114] The computer device acquires songs that lack lyrics from a pre-constructed song set as the first song. The song set contains multiple songs, and the similarity between the songs meets the reference criteria. The device also acquires the song with the highest number of plays and lyrics from the pre-constructed song set as the benchmark song. The benchmark song is a song with lyrics that has similarity to the first song and meets the reference criteria.

[0115] When the computer device performs speech recognition processing on the first song, it uses the lyrics of the benchmark song as a reference text mapping table to obtain the first lyrics text. The first lyrics text is the lyrics text in the reference text mapping table that has the same pronunciation format as the first song. If the first lyrics text is not obtained from the reference text mapping table, the second lyrics text is obtained from the model mapping table. The second lyrics text is the lyrics text in the model mapping table that has the same pronunciation format as the first song. The model mapping table is a mapping table in the speech recognition processing system that contains the mapping relationship between pronunciation format and text. The first lyrics text and the second lyrics text are concatenated in chronological order to obtain the concatenated result as the candidate lyrics text.

[0116] The target lyrics text generation computer device divides the candidate lyrics text into multiple candidate sub-segments; divides the lyrics text of the benchmark song into multiple sub-lyric texts with the same number of characters as the candidate sub-segments; calculates the text similarity between each candidate sub-segment and each sub-lyric text; obtains the sub-lyric text with the highest text similarity to the candidate sub-segment; replaces the candidate sub-segments in the candidate lyrics text with the sub-lyric text with the highest text similarity to the candidate sub-segment; and obtains the replaced candidate lyrics text as the lyrics text of the target song, i.e., the target lyrics text. Exemplarily, based on any one or more embodiments corresponding to Figures 2 to 5, this application proposes a lyrics generation optimization method based on homonym clustering.

[0117] Referring to FIG. 7, which shows a flowchart of an optimized method for generating lyrics provided by an embodiment of the present application. This optimized method for generating lyrics can be executed by a computer device, and the specific steps are as follows: Step S71: Construct a set of clustered same words. In this step, the computer device can divide multiple similar songs into the same group in the following two ways.

[0118] 1) Audio fingerprint matching grouping: Each song has its corresponding audio fingerprint. The computer device analyzes the audio features of the song and extracts the audio fingerprint of the song. If the audio fingerprints of two songs are very similar (for example, the similarity between the audio fingerprints of two songs is greater than or equal to 70%), these two songs are grouped into one group.

[0119] 2) Lyric matching grouping: If the song has lyrics, directly compare the lyrics of each song; if the song has no lyrics, use the ASR speech recognition technology to convert the sound of the song into text and compare these lyrics or the converted text. For example, Song 1 by singer A has lyrics, while the cover of Song 1 by civilian A has no lyrics. Perform speech recognition processing on the cover of Song 1 by civilian A. Use the edit distance algorithm to calculate that Song 1 by singer A and the cover of Song 1 by civilian A are very similar, then group Song 1 by singer A and the cover of Song 1 by civilian A into one group.

[0120] Step S72: Select a reference song. From each song group constructed in step S71, sort them in descending order according to the song playback volume, and select the song with lyrics as the reference song. Other songs without lyrics in the group are marked as songs to be generated with lyrics.

[0121] Step S73: Initially generate lyric text for the song to be generated with lyrics. Use the lyric text of the reference song selected in step S72 as the reference text for this song group. When performing ASR speech-to-text processing on the song to be generated with lyrics in the group, preferentially use the words with similar pronunciations in the reference text, which can significantly reduce pronunciation typos.

[0122] The above reference text refers to the lyric text of the reference song. For example, the computer device first converts the lyric text of the reference song into a pronunciation format, such as shēng: sound; ming: life. When performing ASR on the song to be generated with lyrics, when encountering a character with the pronunciation of shēng (such as sheng / sheng / sound, etc.), preferentially find shēng: sound in the reference text and avoid finding sheng / sheng.

[0123] The specific implementation is to embed the reference text into the decoder of the ASR model in the form of a mapping table. When performing ASR, the decoder obtains the corresponding words and phrases in the reference text from the mapping table as the final generated content. Preferentially use the words and phrases in the mapping table of the reference text, and use the mapping table of the original ASR model if no match is found.

[0124] Step S74: The final lyrics text of the song to be generated is generated by calculating the edit distance between the lyrics initially generated in step S73 and the lyrics of the benchmark song, obtaining the longest segment that matches the benchmark lyrics, and then taking that segment as the final generated lyrics text.

[0125] The longest segment that matches the baseline lyrics refers to the longest part in the initially generated lyrics that is most similar to the baseline lyrics. This part could be a sentence, a paragraph, or even the entire lyrics in the initially generated lyrics.

[0126] In this embodiment, the computer device gradually shifts the initially generated lyrics by offsetting the text and comparing it with the reference lyrics. It calculates the edit distance after each offset by one character and takes the minimum edit distance as the largest segment that matches the reference lyrics. Referring to Figure 8, which shows a schematic diagram of lyrics matching provided in an embodiment of this application, the text segment with the smallest edit distance is obtained after offsetting by one character.

[0127] Step S75: Generate word-by-word lyrics. The final lyrics text from step S74 and the song audio are used to obtain the final word-by-word lyrics of the target song according to the word-by-word lyrics algorithm.

[0128] The aforementioned word-by-word lyrics algorithm is capable of generating timestamped lyrics text by inputting the audio and lyrics of a song. This algorithm can match the lyrics with audio time points, achieving synchronized display word by word and enhancing the user's audiovisual experience.

[0129] The target song mentioned above refers to the song for which lyrics are to be generated; for example, a group contains the following songs: 1. Base song A; 2. Song B1 for which lyrics are to be generated; 3. Song B2 for which lyrics are to be generated; 4. Song B3 for which lyrics are to be generated. During the generation process, each song for which lyrics are to be generated is processed separately with the base song A.

[0130] Furthermore, please refer to Figure 9, which shows a flowchart of lyrics optimization generation provided in one embodiment of this application.

[0131] As shown in Figure 9, the computer device constructs word similarity clusters based on the lyrics of the songs (for example, it can cluster according to the song type, which includes but is not limited to covers, excerpts, speed changes, pitch changes, etc.). From the word similarity clusters, the song with lyrics and the highest popularity is selected as the baseline song, and the baseline lyrics A of the baseline song are obtained. From the word similarity clusters, the songs without lyrics other than the baseline song are selected as the spread songs. The audio-to-text operation is performed on the spread songs to obtain the lyrics B corresponding to the spread songs.

[0132] Furthermore, the computer device matches lyrics B with the baseline lyrics A to obtain the baseline lyrics segment C. The matching rule can be to prioritize complete matching, then fuzzy matching, and finally smooth matching of individual words.

[0133] After obtaining the baseline lyrics segment C, the text similarity between each sub-text part of the baseline lyrics segment C and each sub-text part of the lyrics B is calculated. When the text similarity is greater than or equal to 0.9, the sub-text parts of the baseline lyrics segment C replace the sub-text parts of the lyrics B to generate the final lyrics.

[0134] In this embodiment of the application, the computer device constructs word similarity clusters and applies the lyrics calibration production of the benchmark songs with lyrics to the lyrics of the lyrics-less songs in the cluster, such as covers, excerpts, and other songs with speed and pitch changes. Please refer to Figure 10, which shows a flowchart of lyrics optimization generation provided in an embodiment of the application.

[0135] The above-mentioned optimization scheme for lyrics generation based on word clustering can effectively improve the accuracy of the generated lyrics text.

[0136] The above is merely one embodiment of this application and should not be considered as a limitation thereof. Those skilled in the art will understand that various modifications and variations can be made to the embodiments to adapt to different application requirements. Therefore, the scope of this application should be defined by the claims appended to the claims.

[0137] Please refer to Figure 11, which shows a block diagram of a lyrics generation apparatus provided in an exemplary embodiment of this application. This lyrics generation apparatus can be implemented as all or part of a computer device through hardware or a combination of hardware and software to implement all or part of the steps in the embodiments shown in Figures 2 to 5 above. As shown in Figure 11, the lyrics generation apparatus includes: a first song acquisition module 1101, used to acquire a first song, which is a song lacking lyrics; a reference song acquisition module 1102, used to acquire a reference song, which is a song with lyrics and whose similarity to the first song meets reference conditions; and a lyrics text generation module 1103, used to generate lyrics text for the first song with reference to the lyrics text of the reference song.

[0138] In one possible implementation, the lyrics text generation module 1103 is used to perform speech recognition processing on the first song with reference to the lyrics text of a benchmark song to obtain candidate lyrics text of the first song; and to obtain the lyrics text of the first song based on the candidate lyrics text and the lyrics text of the benchmark song.

[0139] In one possible implementation, the lyrics text generation module 1103 is used to: when performing speech recognition processing on the first song, use the lyrics text of the benchmark song as a reference text mapping table, obtain the first lyrics text from the reference text mapping table, the first lyrics text being the lyrics text in the reference text mapping table with the same pronunciation format as the first song; if the first lyrics text is not obtained from the reference text mapping table, obtain the second lyrics text from the model mapping table, the second lyrics text being the lyrics text in the model mapping table with the same pronunciation format as the first song, the model mapping table being a mapping table of the speech recognition processing system containing the mapping relationship between pronunciation format and text; and concatenate the first lyrics text and the second lyrics text in chronological order to obtain the concatenation result as candidate lyrics text.

[0140] In one possible implementation, the lyrics text generation module 1103 is used to: divide the candidate lyrics text into two or more candidate sub-segments; divide the lyrics text of the benchmark song into two or more sub-lyric texts, the number of characters in the sub-lyric texts being the same as the number of characters in the candidate sub-segments; calculate the text similarity between each candidate sub-segment and each sub-lyric text; and obtain the lyrics text of the first song based on the text similarity.

[0141] In one possible implementation, the lyrics text generation module 1103 is used to: obtain the sub-lyric text with the highest text similarity to the candidate sub-segment; replace the candidate sub-segment in the candidate lyrics text with the sub-lyric text with the highest text similarity to the candidate sub-segment; and obtain the replaced candidate lyrics text as the lyrics text of the first song.

[0142] In one possible implementation, the first song acquisition module 1101 is used to acquire songs lacking lyrics from the song set as the first song, the song set containing multiple songs, and the similarity between the multiple songs meets the reference conditions; the benchmark song acquisition module 1102 is used to acquire the song with the highest number of plays and lyrics from the song set as the benchmark song.

[0143] In one possible implementation, the device further includes: an information acquisition module, configured to acquire playback information and feature information of at least two songs before acquiring the first song, wherein the at least two songs include songs with lyrics and songs without lyrics, the playback information includes at least one of play count and release time, and the feature information includes at least one of audio fingerprint and lyrics text; and a song addition module, configured to determine a target song from the at least two songs and add it to the song set based on the playback information and feature information.

[0144] In one possible implementation, the song adding module is used to: acquire at least two songs whose play count is greater than a threshold and / or whose release time is earlier than a specified time as reference songs and add them to the song set; determine the similarity between the second song and the reference songs based on the feature information of the at least two songs, wherein the second song is the song other than the reference song among the at least two songs; and add the second song as the target song to the song set when the similarity between the second song and the reference song is greater than a similarity threshold.

[0145] In one possible implementation, where the feature information includes lyrics text, and both the second song and the reference song have lyrics text, the song adding module is used to calculate the similarity between the lyrics text of the second song and the lyrics text of the reference song using an edit distance algorithm.

[0146] Please refer to Figure 12, which is a schematic diagram of the structure of a computer device provided in an exemplary embodiment of this application. The computer device 1200 includes a Central Processing Unit (CPU) 1201, a system memory 1204 including Random Access Memory (RAM) 1202 and Read-Only Memory (ROM) 1203, and a system bus 1205 connecting the system memory 1204 and the CPU 1201. The computer device 1200 also includes a Basic Input / Output System (I / O System) 1206 to facilitate the transmission of information between various devices within the computer, and a mass storage device 1207 for storing the operating system 1213, application programs 1214, and other program modules 1215.

[0147] The basic input / output system 1206 includes a display 1208 for displaying information and an input device 1209 for user input, such as a mouse or keyboard. Both the display 1208 and the input device 1209 are connected to the central processing unit 1201 via an input / output controller 1210 connected to the system bus 1205. The basic input / output system 1206 may also include the input / output controller 1210 for receiving and processing input from multiple other devices such as a keyboard, mouse, or electronic stylus. Similarly, the input / output controller 1210 also provides output to a display screen, printer, or other types of output devices.

[0148] The mass storage device 1207 is connected to the central processing unit 1201 via a mass storage controller (not shown) connected to the system bus 1205. The mass storage device 1207 and its associated computer-readable media provide non-volatile storage for the computer device 1200. That is, the mass storage device 1207 may include computer-readable media (not shown) such as a hard disk or a CD-ROM (Compact Disc Read-Only Memory) drive.

[0149] Without loss of generality, the computer-readable medium may include computer storage media and communication media. Computer storage media include volatile and non-volatile, removable and non-removable media implemented using any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. Computer storage media include RAM (Random Access Memory), ROM (Read-Only Memory), EPROM (Erasable Programmable Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), flash memory or other solid-state storage technologies, CD-ROM, DVD (Digital Video Disc), or other optical storage, magnetic tape cassettes, magnetic tape, disk storage, or other magnetic storage devices. Of course, those skilled in the art will recognize that the computer storage media are not limited to the above-mentioned types. The system memory 1204 and the mass storage device 1207 described above can be collectively referred to as memory.

[0150] Computer device 1200 can be connected to the Internet or other network devices via network interface unit 1211 connected to the system bus 1205.

[0151] The memory also includes one or more programs stored in the memory, and the central processing unit 1201 executes the one or more programs to implement all or part of the steps in the method shown in Figures 2 to 5.

[0152] In an exemplary embodiment, a chip is also provided, the chip including programmable logic circuitry and / or program instructions, which, when the chip is run on a computer device, are used to implement all or part of the steps of the methods shown in the above embodiments of this application.

[0153] In an exemplary embodiment, a computer program product is also provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions to implement all or part of the steps of the methods shown in the above embodiments of this application.

[0154] In an exemplary embodiment, a computer-readable storage medium is also provided, which stores a computer program that is loaded and executed by a processor to implement all or part of the steps of the methods shown in the above embodiments of this application.

[0155] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0156] Those skilled in the art will recognize that the functions described in the embodiments of this application in one or more of the above examples can be implemented using hardware, software, firmware, or any combination thereof. When implemented using software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or code on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein communication media include any medium that facilitates the transfer of a computer program from one place to another. Storage media can be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0157] The above description is merely an optional embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for generating lyrics, characterized in that, The method includes: obtaining a first song, wherein the first song is a song lacking lyrics; obtaining a benchmark song, wherein the benchmark song has lyrics and its similarity to the first song meets a reference condition; and generating the lyrics text of the first song with reference to the lyrics text of the benchmark song.

2. The method according to claim 1, characterized in that, The step of generating the lyrics of the first song with reference to the lyrics of the benchmark song includes: performing speech recognition processing on the first song with reference to the lyrics of the benchmark song to obtain candidate lyrics of the first song; and obtaining the lyrics of the first song based on the candidate lyrics and the lyrics of the benchmark song.

3. The method according to claim 2, characterized in that, The step of performing speech recognition processing on the first song with reference to the lyrics of the benchmark song to obtain candidate lyrics for the first song includes: when performing speech recognition processing on the first song, using the lyrics of the benchmark song as a reference text mapping table, obtaining a first lyric text from the reference text mapping table, wherein the first lyric text is a lyric text in the reference text mapping table with the same pronunciation format as the first song; when the first lyric text is not obtained from the reference text mapping table, obtaining a second lyric text from a model mapping table, wherein the second lyric text is a lyric text in the model mapping table with the same pronunciation format as the first song, wherein the model mapping table is a mapping table of the speech recognition processing system that contains the mapping relationship between pronunciation format and text; and concatenating the first lyric text and the second lyric text in chronological order to obtain the concatenation result as the candidate lyric text.

4. The method according to claim 2, characterized in that, The step of obtaining the lyrics text of the first song based on the candidate lyrics text and the lyrics text of the benchmark song includes: dividing the candidate lyrics text into two or more candidate sub-segments; dividing the lyrics text of the benchmark song into two or more sub-lyric texts, wherein the number of characters in the sub-lyric texts is the same as the number of characters in the candidate sub-segments; calculating the text similarity between each candidate sub-segment and each sub-lyric text; and obtaining the lyrics text of the first song based on the text similarity.

5. The method according to claim 4, characterized in that, The step of obtaining the lyrics text of the first song based on the text similarity includes: obtaining the sub-lyric text with the highest text similarity to the candidate sub-segment; replacing the candidate sub-segment in the candidate lyrics text with the sub-lyric text with the highest text similarity to the candidate sub-segment; and obtaining the replaced candidate lyrics text as the lyrics text of the first song.

6. The method according to claim 1, characterized in that, The step of obtaining the first song includes: obtaining a song with missing lyrics from a song set as the first song, wherein the song set contains multiple songs and the similarity between the multiple songs satisfies the reference condition; the step of obtaining the benchmark song includes: obtaining the song with the highest number of plays and lyrics from the song set as the benchmark song.

7. The method according to claim 6, characterized in that, Before obtaining the first song, the method further includes: obtaining playback information and feature information of at least two songs, wherein the at least two songs include songs with lyrics and songs without lyrics, the playback information includes at least one of play count and release time, and the feature information includes at least one of audio fingerprint and lyrics text; and determining a target song from the at least two songs and adding it to the song set based on the playback information and the feature information.

8. The method according to claim 7, characterized in that, The step of determining a target song to add to the song set from the at least two songs based on the playback information and the feature information includes: acquiring songs from the at least two songs whose play count is greater than a quantity threshold and / or whose release time is earlier than a specified time as reference songs and adding them to the song set; determining the similarity between a second song and the reference song based on the feature information of each of the at least two songs, wherein the second song is a song from the at least two songs other than the reference song; and adding the second song as the target song to the song set when the similarity between the second song and the reference song is greater than a similarity threshold.

9. The method according to claim 8, characterized in that, When the feature information includes lyrics text, and both the second song and the reference song have lyrics text, determining the similarity between the second song and the reference song based on the feature information of each of the at least two songs includes: calculating the similarity between the lyrics text of the second song and the lyrics text of the reference song using an edit distance algorithm.

10. A lyric generation device, characterized in that, The device includes: a first song acquisition module for acquiring a first song, wherein the first song is a song lacking lyrics; a reference song acquisition module for acquiring a reference song, wherein the reference song has lyrics and its similarity to the first song meets reference conditions; and a lyrics text generation module for generating lyrics text of the first song with reference to the lyrics text of the reference song.

11. A computer device, characterized in that, The computer device includes a processor and a memory, the memory storing instructions which are executed by the processor to implement the lyrics generation method as described in any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that, The storage medium stores instructions that are executed by a processor of a computer device to implement the lyrics generation method as described in any one of claims 1 to 9.

13. A computer program product, characterized in that, The computer program product includes computer instructions stored in a computer-readable storage medium; the computer instructions are read and executed by a processor of a computer device to implement the lyrics generation method as described in any one of claims 1 to 9.

Images