Lyrics file generation method, computer device and computer-readable storage medium

Abstract

The application relates to a song lyric file generation method, a computer device and a storage medium. The method comprises the following steps: extracting a video frame set and audio data from a music video; extracting song lyric data of each video frame in the video frame set to obtain a first song lyric set, and extracting song lyric data of the audio data to obtain a second song lyric set; determining, from the second song lyric set, song lyric data corresponding to the timestamp of any song lyric data in the first song lyric set as target song lyric data; obtaining the first similarity degree of any song lyric data and the target song lyric data, and determining a target song lyric set based on the first similarity degree; calculating the second similarity degree of the target song lyric set and a reference song lyric set determined based on the target song lyric set by using the reference song lyric set, and adjusting the content of the song lyric in the target song lyric set to obtain a song lyric file corresponding to the music video. The method can avoid song lyric missing detection and false detection, and effectively improves the accuracy of song lyric file generation.

Description

Technical Field

[0001] This application relates to the field of Internet technology, and in particular to a method for generating lyrics files, a computer device, and a computer-readable storage medium. Background Technology

[0002] With the development of short video technology, more and more user-generated music videos have emerged, such as user-made cover music videos. For these user-generated music videos, it is necessary to generate corresponding LRC lyrics for these videos, allowing users to engage in secondary creation and dissemination.

[0003] In related technologies, lyrics can be generated based on lyric detection technology, but there will be a large number of missed detections and false detections. The accuracy of generating lyrics for user-made music short videos is not high, and there will be typos or mismatches between audio and lyrics in the time dimension. Summary of the Invention

[0004] Therefore, it is necessary to provide a lyric file generation method, computer device, and computer-readable storage medium that can improve the accuracy of lyric generation in response to the above-mentioned technical problems.

[0005] Firstly, this application provides a method for generating lyrics files. The method includes:

[0006] According to the preset frame extraction rules, multiple candidate video frames are extracted from the music video;

[0007] Based on the difference information between any candidate video frame and the previous candidate video frame of any candidate video frame, the target video frame is determined from the plurality of candidate video frames to obtain a video frame set;

[0008] Audio data is extracted from the music video according to a preset audio format.

[0009] The lyrics data of each video frame in the video frame set are extracted to obtain a first lyrics set, and the lyrics data of the audio data are extracted to obtain a second lyrics set;

[0010] From the second set of lyrics, determine the lyrics data that corresponds to the timestamp of any lyrics data in the first set of lyrics, and use it as the target lyrics data;

[0011] Obtain any one of the lyrics data and the target lyrics data to get a first similarity. If the first similarity is greater than a first threshold, add the any one of the lyrics data to the target lyrics set.

[0012] Using a reference track lyrics set determined based on the target lyrics set, a second similarity is calculated between the target lyrics set and the reference track lyrics set. The content of the lyrics in the target lyrics set is then adjusted to obtain the lyrics file corresponding to the music video.

[0013] In one embodiment, extracting lyrics data from each video frame in the video frame set includes:

[0014] For any given video frame, extract the text information contained in that video frame to obtain multiple extracted texts;

[0015] The multiple extracted texts are filtered according to preset text removal information to obtain filtered extracted text; the preset text removal information is used to remove non-lyric text content.

[0016] By aggregating the filtered and extracted text within any video frame, the lyrics data for that video frame are obtained.

[0017] In one embodiment, obtaining the first set of lyrics includes:

[0018] If it is detected that a certain video frame meets the inter-frame merging condition with a consecutive video frame adjacent to the certain video frame, the lyrics data of the certain video frame and the lyrics data of the consecutive video frames are merged to obtain merged lyrics, and the merged lyrics are added to the first lyrics set.

[0019] If a video frame is detected as not meeting the inter-frame merging condition, the lyrics data corresponding to that video frame is added to the first lyrics set.

[0020] In one embodiment, each lyric data in the first lyric set corresponds to a subtitle extraction region, and the step of detecting that a certain video frame and consecutive video frames adjacent to the certain video frame satisfy the inter-frame merging condition includes:

[0021] If the subtitle extraction area between a certain video frame and the next adjacent video frame is within a preset coordinate range;

[0022] Obtain the third similarity of lyrics data between a certain video frame and the next video frame;

[0023] If the third similarity satisfies a preset threshold, it is determined that the inter-frame merging condition is met between a certain video frame and consecutive video frames adjacent to the certain video frame.

[0024] In one embodiment, before the steps of calculating a second similarity between the target lyrics set and the reference track lyrics set using a reference track lyrics set determined based on the target lyrics set, adjusting the content of the lyrics in the target lyrics set, and obtaining the lyrics file corresponding to the music video, the method further includes:

[0025] Determine the reference tracks corresponding to each of the target lyrics; the reference tracks include multiple reference songs.

[0026] The set of lyrics for the reference tracks is obtained based on the reference lyrics of the reference songs that meet the song selection criteria.

[0027] In one embodiment, the step of using a reference track lyrics set determined based on the target lyrics set, calculating a second similarity between the target lyrics set and the reference track lyrics set, and adjusting the content of the lyrics in the target lyrics set to obtain the lyrics file corresponding to the music video includes:

[0028] For any target lyrics, determine the second similarity between the target lyrics and any reference lyrics in the reference song lyrics set, the first offset value of the target lyrics in the target lyrics set, and the second offset value of the reference lyrics in the reference song lyrics set;

[0029] The first offset value and the second offset value corresponding to the largest second similarity are used as the reference offset value;

[0030] Adjust the content of any target lyrics according to the reference offset value to obtain the adjusted target lyrics, and generate the lyrics file based on the adjusted target lyrics.

[0031] In one embodiment, adjusting the content of any target lyrics according to the reference offset value to obtain the adjusted target lyrics includes:

[0032] Based on the aforementioned baseline offset value, the matching region is determined from the reference track lyrics set by shifting the first offset value and the second offset value upwards and downwards respectively;

[0033] Using the reference lyrics in the matching area, typos in any of the target lyrics are corrected to obtain the adjusted target lyrics.

[0034] In one embodiment, before the step of extracting multiple candidate video frames from the music video according to a preset frame extraction rule, the method further includes:

[0035] Obtain preset video filtering information; the video filtering information includes video duration thresholds and reference parameter conditions;

[0036] If the duration of the input music video does not exceed the video duration threshold and the video parameters of the input music video meet the reference parameter conditions, the input music video will be used as the music video.

[0037] Secondly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to perform the following steps:

[0038] According to the preset frame extraction rules, multiple candidate video frames are extracted from the music video;

[0039] Based on the difference information between any candidate video frame and the previous candidate video frame of any candidate video frame, the target video frame is determined from the plurality of candidate video frames to obtain a video frame set;

[0040] Audio data is extracted from the music video according to a preset audio format.

[0041] The lyrics data of each video frame in the video frame set are extracted to obtain a first lyrics set, and the lyrics data of the audio data are extracted to obtain a second lyrics set;

[0042] From the second set of lyrics, determine the lyrics data that corresponds to the timestamp of any lyrics data in the first set of lyrics, and use it as the target lyrics data;

[0043] Obtain any one of the lyrics data and the target lyrics data to get a first similarity. If the first similarity is greater than a first threshold, add the any one of the lyrics data to the target lyrics set.

[0044] Using a reference track lyrics set determined based on the target lyrics set, a second similarity is calculated between the target lyrics set and the reference track lyrics set. The content of the lyrics in the target lyrics set is then adjusted to obtain the lyrics file corresponding to the music video.

[0045] Thirdly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, performs the following steps:

[0046] According to the preset frame extraction rules, multiple candidate video frames are extracted from the music video;

[0047] Based on the difference information between any candidate video frame and the previous candidate video frame of any candidate video frame, the target video frame is determined from the plurality of candidate video frames to obtain a video frame set;

[0048] Audio data is extracted from the music video according to a preset audio format.

[0049] The lyrics data of each video frame in the video frame set are extracted to obtain a first lyrics set, and the lyrics data of the audio data are extracted to obtain a second lyrics set;

[0050] From the second set of lyrics, determine the lyrics data that corresponds to the timestamp of any lyrics data in the first set of lyrics, and use it as the target lyrics data;

[0051] Obtain any one of the lyrics data and the target lyrics data to get a first similarity. If the first similarity is greater than a first threshold, add the any one of the lyrics data to the target lyrics set.

[0052] Using a reference track lyrics set determined based on the target lyrics set, a second similarity is calculated between the target lyrics set and the reference track lyrics set. The content of the lyrics in the target lyrics set is then adjusted to obtain the lyrics file corresponding to the music video.

[0053] Fourthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, performs the following steps:

[0054] According to the preset frame extraction rules, multiple candidate video frames are extracted from the music video;

[0055] Based on the difference information between any candidate video frame and the previous candidate video frame of any candidate video frame, the target video frame is determined from the plurality of candidate video frames to obtain a video frame set;

[0056] Audio data is extracted from the music video according to a preset audio format.

[0057] The lyrics data of each video frame in the video frame set are extracted to obtain a first lyrics set, and the lyrics data of the audio data are extracted to obtain a second lyrics set;

[0058] From the second set of lyrics, determine the lyrics data that corresponds to the timestamp of any lyrics data in the first set of lyrics, and use it as the target lyrics data;

[0059] Obtain any one of the lyrics data and the target lyrics data to get a first similarity. If the first similarity is greater than a first threshold, add the any one of the lyrics data to the target lyrics set.

[0060] Using a reference track lyrics set determined based on the target lyrics set, a second similarity is calculated between the target lyrics set and the reference track lyrics set. The content of the lyrics in the target lyrics set is then adjusted to obtain the lyrics file corresponding to the music video.

[0061] The aforementioned lyric file generation method, apparatus, computer equipment, and storage medium extract multiple candidate video frames from a music video according to a preset frame extraction rule. Based on the difference information between any candidate video frame and its preceding candidate video frame, a target video frame is determined from the multiple candidate video frames, resulting in a video frame set. Audio data is extracted from the music video according to a preset audio format. Lyric data is then extracted from each video frame in the video frame set to obtain a first lyric set. The extracted audio data yields a second lyric set. From the second lyric set, the lyric data corresponding to the timestamp of any lyric data in the first lyric set is determined as the target lyric data. Any lyric data is compared with the target lyric data to obtain a first similarity. If the first similarity is greater than a first threshold, the lyric data is added to the target lyric set. Then, using a reference track lyric set determined based on the target lyric set, a second similarity is calculated between the target lyric set and the reference track lyric set. The content of the lyrics in the target lyric set is adjusted to obtain the lyric file corresponding to the music video. This optimizes the generation of lyric files. Based on the similarity between the subtitle lyrics extracted from the music video and the audio lyrics, the target lyric set is selected and determined. The content of the lyrics is adjusted in conjunction with the reference track lyric set to obtain the lyric file. This avoids missed and false detections of lyrics and effectively improves the accuracy of lyric file generation. Attached Figure Description

[0062] Figure 1 This is a flowchart illustrating a method for generating lyrics files in one embodiment;

[0063] Figure 2a This is a schematic diagram of a music video extraction process in one embodiment;

[0064] Figure 2b This is a schematic diagram of a lyrics data extraction process in one embodiment;

[0065] Figure 2c This is a schematic diagram of another lyrics data extraction process in one embodiment;

[0066] Figure 3 This is a schematic diagram of a video frame in one embodiment;

[0067] Figure 4 This is a schematic diagram illustrating the comparison of a set of lyrics in one embodiment;

[0068] Figure 5 This is a flowchart illustrating another method for generating lyrics files in one embodiment;

[0069] Figure 6 This is a structural block diagram of a lyrics file generation device in one embodiment;

[0070] Figure 7 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0071] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0072] In one embodiment, such as Figure 1 As shown, a method for generating lyrics files is provided. This embodiment illustrates the application of this method to a server. It is understood that this method can also be applied to a terminal, or to a system including both a terminal and a server, and is implemented through interaction between the terminal and the server. In this embodiment, the method includes the following steps.

[0073] In step S101, multiple candidate video frames are extracted from the music video according to a preset frame extraction rule.

[0074] As an example, music videos can be short music videos created by users, such as short cover music videos uploaded by users themselves.

[0075] In step S102, the target video frame is determined from multiple candidate video frames based on the difference information between any candidate video frame and the previous candidate video frame of any candidate video frame, thus obtaining a video frame set.

[0076] For example, similar frames can be filtered based on the calculated differences between video frames, thus obtaining a set of video frames.

[0077] In one example, to avoid wasting computing resources, the Hamming distance (the difference hash between the current video frame and the extracted previous video frame) can be calculated. If the distance is short, the two frames can be considered to be too similar, and the detection of the current video frame can be abandoned. That is, the target video frame is determined from multiple candidate video frames based on the difference information between any candidate video frame and the previous candidate video frame of any candidate video frame. The filtered video frames can be stored in the CFS file system for later use.

[0078] In step S103, audio data is extracted from the music video according to a preset audio format;

[0079] In practical applications, such as Figure 2a As shown, for audio extraction, in order to use the singing ASR model, the audio stream needs to be extracted into a WAV file (i.e., the preset audio format) and saved to the CFS file system (which can share files between different servers) for later use; for video frame extraction, a uniform frame extraction strategy (i.e., preset frame extraction information) can be adopted, and different frame extraction densities can be set according to the video length (e.g., a video within 1 minute can be extracted at 3 frames per second, and a video exceeding 1 minute can be extracted at 1 frame per second).

[0080] For example, a music video can be obtained and processed by frame extraction. By uniformly extracting frames from the music video, video frames whose difference hash value is less than the distance threshold with the previous video frame can be filtered out. The filtered video frames can then be saved to the CFS (Cloud File Storage) file system (i.e., the video frame set is extracted) for further processing. The audio in the music video can also be extracted into a WAV file and saved to the CFS file system (i.e., the audio data is extracted) for later use.

[0081] In step S104, the lyrics data of each video frame in the video frame set are extracted to obtain the first lyrics set, and the lyrics data of the extracted audio data are extracted to obtain the second lyrics set;

[0082] After obtaining the video frame set and audio data, video subtitle lyrics can be extracted for each video frame in the video frame set. Then, a first lyrics set can be obtained based on the extracted lyrics data of each video frame, and audio lyrics can be extracted from the audio data to obtain a second lyrics set.

[0083] In one example, such as Figure 2b As shown, in the process of extracting lyrics from video subtitles, OCR (optical character recognition) technology can be used to identify the text information in each video frame. Then, non-lyric text information, such as identifier text, singer names, song titles, etc., can be filtered out. Furthermore, the lyrics of subtitles in adjacent areas within a video frame can be extracted and their union taken to obtain the subtitle lyrics for each video frame. Then, identical lyrics between video frames can be merged to obtain the first set of lyrics (e.g., ...). Figure 2b (Chinese subtitle lyrics candidate list).

[0084] In yet another example, such as Figure 2cAs shown, during the process of extracting audio lyrics, audio files (i.e., audio data) stored in the CFS file system can be loaded into memory. Then, the audio lyrics can be detected. For example, an Automatic Speech Recognition (ASR) model can be used to identify the lyrics information in the audio file, and then the lyrics can be saved line by line to the audio lyric candidate set (i.e., the second lyric set). For example, the extracted audio lyrics can be saved in the following format:

[0085] [(start0, end1, duration0, audio_content),...]

[0086] Here, start is the start timestamp, end is the end timestamp, duration is the time interval, and the audio lyrics within that time interval are also included.

[0087] In step S105, the lyrics data corresponding to the timestamp of any lyrics data in the first lyrics set is determined from the second lyrics set and used as the target lyrics data;

[0088] The target lyrics data can be lyrics data in the second lyrics set whose presentation time matches the presentation time of any lyrics data in the first lyrics set, and the presentation time can be determined based on a timestamp.

[0089] In step S106, any lyrics data and target lyrics data are obtained to obtain a first similarity. If the first similarity is greater than a first threshold, any lyrics data is added to the target lyrics set.

[0090] In its implementation, based on a first set of lyrics, the lyrics data within each video frame of this set can be traversed according to timestamp information. The similarity between any given lyric data and the corresponding audio lyrics (i.e., the target lyric data) at the specified timestamp can be calculated. This allows for the selection of the target lyric set based on the first set of lyrics. Consequently, subtitle lyrics without corresponding detection results can be effectively filtered out based on the audio lyrics, further improving the accuracy of the selected lyrics and effectively avoiding false detections.

[0091] In one example, if the similarity is greater than the threshold (i.e., the first similarity is greater than the first threshold), it can be considered that both the OCR detection and the singing ASR detection have obtained good results, and this subtitle lyric is very likely to be the target lyric to be recognized. In the case where OCR detection or singing ASR detection may bring typos, such as the OCR detection result being "resolutely like iron" and the singing ASR detection result being "cut off like iron", considering the user's own accent and the special singing scenario in the music video, ASR may recognize "resolutely" as "cut off", but overall it is correct. Since the result of OCR is more accurate, the subtitle lyric "resolutely like iron" detected by OCR can be added to the target lyric set.

[0092] In another example, if the similarity is less than the threshold, since the subtitle lyric in the video frame can contain multiple complete lyrics in the upper and lower lines, that is, the subtitle can be in a multi-line mode, based on the singing ASR detection result, the OCR detection result can be divided more finely in the time dimension, and then the similarity can be recalculated. If the recalculated similarity is still less than the threshold, it can be directly ignored. For example, for the OCR detection result "add me to the fan club", since there is no corresponding detection result in the audio lyric, it can be directly filtered. If there is no result corresponding to the singing ASR detection for all the subtitle lyrics within this timestamp, an error can be directly reported.

[0093] In an optional embodiment, during the process of screening the target lyrics, if the partitioning granularity of the lyric data in the first lyric set is greater than that of the lyric data in the second lyric set, the lyric data in the second lyric set can be used for a finer partitioning to effectively determine the target lyric set.

[0094] In step S107, a reference track lyric set determined based on the target lyric set is used to calculate the second similarity between the target lyric set and the reference track lyric set, and the content of the lyrics in the target lyric set is adjusted to obtain the lyric file corresponding to the music video.

[0095] After obtaining the target lyric set, by introducing the ability to search for songs based on lyrics, the original lyrics of the searched song can be matched from the song library as the reference lyrics to obtain the reference track lyric set. Then, by calculating the second similarity between the target lyric set and the reference track lyric set, the content of the lyrics in the target lyric set can be adjusted using this reference track lyric set. For example, typos can be corrected, or other adjustment operations can be performed, which are not specifically limited in this embodiment. Furthermore, based on the adjusted result, the lyric file corresponding to the music video, such as an LRC lyric, can be obtained and displayed to the user.

[0096] Compared to traditional methods, the technical solution in this embodiment can generate LRC lyrics for short music videos based on a multi-modal approach. By recognizing and storing the corresponding LRC lyrics generated from user-uploaded short music videos on the server side, corresponding LRC lyrics can be automatically generated for user-made music videos, avoiding missed detections and false detections, and improving the accuracy of generated lyrics.

[0097] In the above-described lyric file generation method, multiple candidate video frames are extracted from the music video according to a preset frame extraction rule. Based on the difference information between any candidate video frame and its previous candidate video frame, a target video frame is determined from the multiple candidate video frames, resulting in a video frame set. Audio data is extracted from the music video according to a preset audio format. Then, lyric data is extracted from each video frame in the video frame set to obtain a first lyric set. The extracted audio data and lyric data form a second lyric set. From the second lyric set, the lyric data corresponding to the timestamp of any lyric data in the first lyric set is determined as the target lyric data. This process is repeated for each lyric data point. The first similarity score is obtained by comparing the target lyrics data with the target lyrics data. If the first similarity score is greater than a first threshold, any lyrics data is added to the target lyrics set. Then, a second similarity score is calculated between the target lyrics set and the reference track lyrics set, which is determined based on the target lyrics set. The content of the lyrics in the target lyrics set is adjusted to obtain the lyrics file corresponding to the music video. This optimizes the generation of lyrics files. Based on the similarity between the subtitle lyrics extracted from the music video and the audio lyrics, the target lyrics set is selected and determined. The content of the lyrics is adjusted in combination with the reference track lyrics set to obtain the lyrics file. This can avoid the situation of missed or false detection of lyrics and effectively improve the accuracy of lyrics file generation.

[0098] In one embodiment, extracting lyrics data for each video frame in the video frame set in step S104 may include the following steps:

[0099] For any given video frame, extract the text information contained in that video frame to obtain multiple extracted texts; filter the multiple extracted texts according to preset text removal information to obtain filtered extracted texts; the preset text removal information is used to remove non-lyric text content; aggregate the filtered extracted texts in any given video frame to obtain the lyrics data of that video frame.

[0100] In one example, the video frame of any video frame can be as follows: Figure 3 As shown, it can display multiple logo text types, multi-line horizontal or vertical subtitle lyrics (i.e., lyrics data in the first lyrics set), and can also display singer names and song titles.

[0101] During the process of extracting lyrics from video subtitles, such as Figure 2b As shown, video frame files (i.e., a collection of video frames) stored in the CFS file system can be loaded into memory. Then, lyrics data in each video frame can be detected. For example, all video frames can be traversed sequentially, and the following operations can be performed on each video frame:

[0102] For any given video frame, OCR can be used to detect all text information contained in that video frame (i.e., multiple extracted texts), such as... Figure 3 As shown, video frames can include identifying text, such as text-based logos, one or more lines of subtitle lyrics, and music video-related keywords such as singer names and song titles. Based on a preset set of logo keywords, all identifying text can be excluded, and singer names and song titles can be excluded based on a preset recognition format for specific keywords. For example, multiple extracted texts can be filtered according to preset text removal information, based on specified symbols (such as 《》), specified words (such as "original singer", "cover", "singing", "adaptation", "cover", or modified keywords "Cover", "c0ver", "C0ver"), to obtain the filtered extracted text.

[0103] Since one or more lines of subtitle lyrics in the same video frame appear in adjacent areas, lyrics can be aggregated based on the distance between the detected text information areas in the same video frame. That is, the filtered and extracted text in any video frame can be aggregated to obtain the subtitle lyrics of any video frame. Then, the subtitle lyrics obtained by the union of adjacent areas in the frame can be added to the initial subtitle set. If there are two or more lines of subtitles, the detected text information areas can be appropriately enlarged horizontally and vertically.

[0104] In this embodiment, by extracting text information contained in any video frame, multiple extracted texts are obtained. Then, the multiple extracted texts are filtered according to preset text removal information to obtain filtered extracted texts. The filtered extracted texts in any video frame are then aggregated to obtain the lyrics data of any video frame. This effectively removes non-lyrics text content extracted from the video frame, improves extraction accuracy, and provides data support for subsequent processing.

[0105] In one embodiment, obtaining the first lyrics set in step S104 may include the following steps:

[0106] If a video frame is found to meet the inter-frame merging condition with a consecutive video frame adjacent to it, the lyrics data of the video frame are merged with the lyrics data of the consecutive video frames to obtain merged lyrics, and the merged lyrics are added to the first lyrics set; if a video frame is found not to meet the inter-frame merging condition, the lyrics data corresponding to the video frame are added to the first lyrics set.

[0107] In practical applications, such as Figure 2b As shown, after obtaining the initial subtitle set by extracting the lyrics data of each video frame in the video frame set, the same lyrics between frames can be merged. By detecting the subtitle extraction area and the similarity of subtitle lyrics text between two adjacent video frames, the same lyrics between frames that meet the inter-frame merging conditions can be merged, and the lyrics data that do not meet the inter-frame merging conditions can be retained to obtain the subtitle lyrics candidate set (i.e., the first lyrics set).

[0108] In an optional embodiment, timestamps can be calculated based on the frame number and frame rate of the video frames in the video frame set, and the first lyrics set can store data in the following format:

[0109] [(start0, end0, duration0, [subtitle0, subtitle1, ...]), ...]

[0110] Here, start is the start timestamp, end is the end timestamp, duration is the time interval, and a list of all subtitle lyrics within that time interval.

[0111] In this embodiment, when it is detected that a certain video frame meets the inter-frame merging condition with a consecutive video frame adjacent to it, the lyrics data of the certain video frame are merged with the lyrics data of the consecutive video frames to obtain merged lyrics, and the merged lyrics are added to the first lyrics set. When it is detected that a certain video frame does not meet the inter-frame merging condition, the lyrics data corresponding to that video frame are added to the first lyrics set, so that the same lyrics between frames can be merged, reducing the amount of duplicate data calculation and effectively improving the efficiency of lyrics file generation.

[0112] In one embodiment, each lyric data in the first lyric set corresponds to a subtitle extraction region. Detecting that consecutive video frames adjacent to a certain video frame meet the inter-frame merging condition may include the following steps:

[0113] If the subtitle extraction area between a certain video frame and the next adjacent video frame is within the preset coordinate range; obtain the third similarity of the lyrics data between a certain video frame and the next video frame; if the third similarity meets the preset threshold, determine that the inter-frame merging condition is met between a certain video frame and the consecutive video frames adjacent to a certain video frame.

[0114] In one example, for consecutive video frames, since the display area of ​​subtitle lyrics is continuous, the subtitle extraction area between two adjacent video frames can be detected separately. If the coordinates of the subtitle lyrics display area of ​​the two video frames are the same (i.e., within the preset coordinate range), the similarity between the lyrics data can be calculated. If the detected similarity is higher than the threshold (i.e., the third similarity meets the preset threshold), it can be considered that the two video frames contain the same lyrics. The search can continue to the next adjacent video frame. If the lyrics appear in the next video frame, the result of the video frame that appears more often can be taken; otherwise, the result of the first video frame can be taken. If the detected similarity is lower than the threshold, it can be considered that the two video frames contain different lyrics, and the different lyrics in the two video frames can be retained.

[0115] For example, if "again, but in another way" and "on the contrary, in another way" (lyrics data between a video frame and the next video frame) are detected between two adjacent video frames, it can be determined that the lyrics data in the two video frames have a high degree of text similarity, and at the same time, the subtitle extraction areas in the two video frames basically overlap, so it can be determined that they are the same lyric.

[0116] In this embodiment, if the subtitle extraction area between a certain video frame and the next adjacent video frame is within a preset coordinate range, the third similarity of the lyrics data between the certain video frame and the next video frame is obtained. If the third similarity meets a preset threshold, it is determined that the consecutive video frames adjacent to a certain video frame meet the inter-frame merging condition, which can accurately merge and process the same lyrics between frames and effectively reduce duplicate data.

[0117] In one embodiment, the following steps may be included before step S107:

[0118] Determine the reference tracks corresponding to each target lyric; the reference tracks include multiple reference songs; based on the reference lyrics of the reference songs that meet the song selection criteria, obtain the reference track lyric set.

[0119] After obtaining the target lyrics set, for each target lyric, a reverse search can be performed on the song ID in the music library to obtain the n songs with the highest similarity to each target lyric (n is a constant, such as 5), which are the reference tracks. The number of times each song ID appears can be recorded, and then it can be calculated whether the number of times each song ID appears is greater than n * number of subtitle sentences * ratio (such as the ratio can be 40%). The IDs of the songs that meet the conditions (i.e., the reference songs that meet the song selection conditions) can be retained as a candidate song set. By traversing all the songs in the candidate song set, the complete lyrics can be retrieved according to each song ID, thus obtaining the reference track lyrics set.

[0120] In this embodiment, by determining the reference tracks corresponding to each target lyric, and then obtaining the reference track lyric set based on the reference lyrics of the reference songs that meet the song selection criteria, the reference track lyric set can be accurately obtained, providing data support for further lyric adjustments.

[0121] In one embodiment, step S107, using a reference track lyrics set determined based on the target lyrics set, calculating a second similarity between the target lyrics set and the reference track lyrics set, and adjusting the content of the lyrics in the target lyrics set to obtain the lyrics file corresponding to the music video, may include the following steps:

[0122] For any target lyrics, determine the second similarity between the target lyrics and any reference lyrics in the reference song lyrics set, the first offset value of the target lyrics in the target lyrics set, and the second offset value of the reference lyrics in the reference song lyrics set; take the first offset value and the second offset value corresponding to the largest second similarity as the baseline offset value; adjust the content of any target lyrics according to the baseline offset value to obtain the adjusted target lyrics, and generate a lyrics file based on the adjusted target lyrics.

[0123] In one example, such as Figure 4 As shown, for each line of lyrics in the target lyrics set (i.e., any target lyric), the reference track lyrics set can be traversed line by line. This is achieved by calculating the similarity between the current subtitle lyrics and the current reference track lyrics (i.e., the second similarity) and the offset value of the current subtitle lyrics within the complete subtitle lyrics (i.e., the first offset value). Figure 4 The offset value of "Lyrics Content 2" in the complete subtitle lyrics is 2, and the offset value of the current reference track lyrics in the complete reference track lyrics (i.e., the second offset value) is as follows: Figure 4The offset value of “lyric content c” in the complete reference song lyrics is 10. After all the lyrics are traversed, the maximum similarity (i.e. the second largest similarity) can be found, as well as its corresponding subtitle lyrics offset value and reference song lyrics offset value (i.e. the first offset value and the second offset value corresponding to the second largest similarity). If multiple identical maximum similarities occur, the initial lyrics position can be retained.

[0124] In this embodiment, for any target lyric, a second similarity between the target lyric and any reference lyric in the reference song lyric set, a first offset value of the target lyric in the target lyric set, and a second offset value of the reference lyric in the reference song lyric set are determined. Then, the first offset value and the second offset value corresponding to the largest second similarity are used as the baseline offset value. The content of any target lyric is then adjusted according to the baseline offset value to obtain the adjusted target lyric. A lyric file is generated based on the adjusted target lyric, which can accurately obtain the baseline offset value for lyric adjustment.

[0125] In one embodiment, adjusting the content of any target lyrics according to a reference offset value to obtain the adjusted target lyrics may include the following steps:

[0126] Based on the baseline offset value, the matching region is determined from the set of lyrics of the reference track by shifting the first offset value and the second offset value up and down respectively; using the reference lyrics in the matching region, the typos in any target lyrics are corrected to obtain the adjusted target lyrics.

[0127] In one example, a verification process can be performed. By using the offset value of the subtitle lyrics corresponding to the maximum similarity and the offset value of the reference track lyrics as the reference points (i.e., the reference offset value), the subtitle lyrics offset value and the reference track lyrics offset value can be shifted up and down respectively to find the most matching area in the reference track lyrics. The reference track lyrics can be used to correct typos in the subtitle lyrics (i.e., any target lyrics). It can also be verified whether the duration difference between the subtitle lyrics and the reference track lyrics is within the threshold.

[0128] For example, validation rules can be represented as follows:

[0129] max_similarity_offset subtitle >= max_similarity_offset official

[0130] len subtitle -max_similarity_offset subtitle <=len official -max_similarity_offset official

[0131] Where max_similarity is the maximum third similarity, and offset subtitle The first offset value, offset officiol This is the second offset value.

[0132] In this embodiment, by offsetting the first offset value and the second offset value up and down respectively according to the reference offset value, the matching area is determined from the set of lyrics of the reference track. Then, the reference lyrics in the matching area are used to correct the typos in any target lyrics, and the adjusted target lyrics are obtained. This can effectively correct the content of the target lyrics and improve the accuracy of the generated lyrics file.

[0133] In one embodiment, the following steps may be included before step S101:

[0134] Obtain preset video filtering information; the video filtering information includes video duration threshold and reference parameter conditions; if the video duration of the input music video does not exceed the video duration threshold and the video parameters of the input music video meet the reference parameter conditions, the input music video will be used as a music video.

[0135] In one example, such as Figure 2a As shown, before extracting downloaded videos (i.e., music videos), video information can be obtained for verification, such as the duration of the video. Since the duration of a single song is generally within 5 minutes, videos exceeding 5 minutes can be discarded directly. Videos with durations exceeding the threshold (i.e., video duration threshold) or videos with abnormal parameters can be filtered out, and then videos that meet the requirements can be downloaded to the local disk.

[0136] In this embodiment, by obtaining preset video filtering information, and then, if the video duration of the input music video does not exceed the video duration threshold and the video parameters of the input music video meet the reference parameter conditions, the input music video is used as a music video, which can effectively filter out music videos that meet the requirements and provide data support for subsequent processing.

[0137] In one embodiment, such as Figure 5 The diagram illustrates another method for generating lyrics files. In this embodiment, the method includes the following steps:

[0138] In step 501, preset video filtering information is obtained. If the duration of the input music video does not exceed the video duration threshold and the video parameters of the input music video meet the reference parameter conditions, the input music video is selected as the music video. In step 502, multiple candidate video frames are extracted from the music video according to preset frame extraction rules. Based on the difference information between any candidate video frame and its previous candidate video frame, a target video frame is determined from the multiple candidate video frames, resulting in a video frame set. In step 503, audio data is extracted from the music video according to a preset audio format. In step 504, text information contained in any video frame is extracted, resulting in multiple extracted texts. These extracted texts are filtered according to preset text removal information, resulting in filtered extracted texts. The filtered extracted texts within any video frame are aggregated to determine the lyrics data of any video frame, resulting in a first lyrics set. The lyrics data of the audio data are then extracted, resulting in a second lyrics set. In step 505, lyrics data corresponding to the timestamp of any lyrics data in the first lyrics set are determined from the second lyrics set and used as the target lyrics data. In step 506, any lyrics data and target lyrics data are obtained to obtain a first similarity. If the first similarity is greater than a first threshold, any lyrics data is added to the target lyrics set. In step 507, the reference tracks corresponding to each target lyric are determined. Based on the reference lyrics of the reference songs that meet the song selection criteria, a reference track lyrics set is obtained. In step 508, using the reference track lyrics set determined based on the target lyrics set, a second similarity is calculated between the target lyrics set and the reference track lyrics set. The content of the lyrics in the target lyrics set is adjusted to obtain the lyrics file corresponding to the music video. It should be noted that the specific limitations of the above steps can be found in the specific limitations of a page rendering method described above, and will not be repeated here.

[0139] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0140] Based on the same inventive concept, this application also provides a page rendering apparatus for implementing the lyrics file generation method described above. The solution provided by this apparatus is similar to the implementation described in the above method; therefore, the specific limitations in one or more embodiments of the lyrics file generation apparatus provided below can be found in the limitations of the lyrics file generation method described above, and will not be repeated here.

[0141] In one embodiment, such as Figure 6 As shown, a lyric file generation apparatus is provided, comprising:

[0142] The frame extraction module 601 is used to extract multiple candidate video frames from the music video according to a preset frame extraction rule.

[0143] The video frame set acquisition module 602 is used to determine the target video frame from the plurality of candidate video frames based on the difference information between any candidate video frame and the previous candidate video frame of any candidate video frame, thereby obtaining a video frame set;

[0144] The audio data extraction module 603 is used to extract audio data from the music video according to a preset audio format;

[0145] The lyrics set acquisition module 604 is used to extract the lyrics data of each video frame in the video frame set to obtain a first lyrics set, and to extract the lyrics data of the audio data to obtain a second lyrics set;

[0146] The target lyrics data determination module 605 is used to determine, from the second lyrics set, the lyrics data corresponding to the timestamp of any lyrics data in the first lyrics set, as the target lyrics data;

[0147] The target lyrics set acquisition module 606 is used to obtain any lyrics data and the target lyrics data to obtain a first similarity. If the first similarity is greater than a first threshold, the any lyrics data is added to the target lyrics set.

[0148] The lyrics file generation module 607 is used to calculate the second similarity between the target lyrics set and the reference track lyrics set based on the target lyrics set, adjust the content of the lyrics in the target lyrics set, and obtain the lyrics file corresponding to the music video.

[0149] In one embodiment, the lyrics set acquisition module 604 includes:

[0150] The text extraction submodule is used to extract text information contained in any given video frame, resulting in multiple extracted texts.

[0151] The text filtering submodule is used to filter the multiple extracted texts according to preset text removal information to obtain filtered extracted text; the preset text removal information is used to remove non-lyric text content.

[0152] The lyrics aggregation submodule is used to aggregate the filtered and extracted text within any video frame to obtain the lyrics data for that video frame.

[0153] In one embodiment, the lyrics set acquisition module 604 includes:

[0154] The inter-frame lyrics merging submodule is used to merge the lyrics data of a certain video frame with the lyrics data of the consecutive video frames when it is detected that the inter-frame merging conditions are met between a certain video frame and the consecutive video frames adjacent to the certain video frame, to obtain merged lyrics, and to add the merged lyrics to the first lyrics set.

[0155] The lyrics retention submodule is used to add the lyrics data corresponding to a certain video frame to the first lyrics set when it is detected that a certain video frame does not meet the inter-frame merging condition.

[0156] In one embodiment, each lyric data in the first lyric set corresponds to a subtitle extraction region, and the inter-frame lyric merging submodule includes:

[0157] The subtitle extraction region detection unit is used to detect if the subtitle extraction region between a certain video frame and the next adjacent video frame is within a preset coordinate range.

[0158] The third similarity acquisition unit is used to acquire the third similarity of lyrics data between a certain video frame and the next video frame.

[0159] The inter-frame merging determination unit is used to determine, if the third similarity satisfies a preset threshold, that the inter-frame merging condition is met between a certain video frame and consecutive video frames adjacent to the certain video frame.

[0160] In one embodiment, the apparatus further includes:

[0161] A reference track determination module is used to determine the reference track corresponding to each of the target lyrics; the reference track includes multiple reference songs.

[0162] The module for obtaining the reference song lyrics set is used to obtain the reference song lyrics set based on the reference lyrics of the reference songs that meet the song selection criteria.

[0163] In one embodiment, the lyrics file generation module 607 includes:

[0164] The calculation submodule is used to determine, for any target lyrics, a second similarity between the target lyrics and any reference lyrics in the reference song lyrics set, a first offset value of the target lyrics in the target lyrics set, and a second offset value of the reference lyrics in the reference song lyrics set;

[0165] The baseline offset value determination submodule is used to take the first offset value and the second offset value corresponding to the largest second similarity as the baseline offset value;

[0166] The adjustment submodule is used to adjust the content of any target lyrics according to the reference offset value to obtain the adjusted target lyrics, and generate the lyrics file based on the adjusted target lyrics.

[0167] In one embodiment, the adjustment submodule includes:

[0168] The matching region determination unit is used to determine the matching region from the reference song lyrics set by offsetting the first offset value and the second offset value up and down respectively, according to the reference offset value;

[0169] The lyrics correction unit is used to correct typos in any of the target lyrics by using reference lyrics in the matching area, so as to obtain the adjusted target lyrics.

[0170] In one embodiment, the apparatus further includes:

[0171] The video filtering information acquisition module is used to acquire preset video filtering information; the video filtering information includes video duration thresholds and reference parameter conditions.

[0172] The video filtering module is used to select an input music video as the music video if the video duration of the input music video does not exceed the video duration threshold and the video parameters of the input music video meet the reference parameter conditions.

[0173] Each module in the aforementioned lyrics file generation device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0174] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 7As shown, this computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores lyric file generation data. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements a lyric file generation method.

[0175] Those skilled in the art will understand that Figure 7 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0176] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0177] According to the preset frame extraction rules, multiple candidate video frames are extracted from the music video;

[0178] Based on the difference information between any candidate video frame and the previous candidate video frame of any candidate video frame, the target video frame is determined from the plurality of candidate video frames to obtain a video frame set;

[0179] Audio data is extracted from the music video according to a preset audio format.

[0180] The lyrics data of each video frame in the video frame set are extracted to obtain a first lyrics set, and the lyrics data of the audio data are extracted to obtain a second lyrics set;

[0181] From the second set of lyrics, determine the lyrics data that corresponds to the timestamp of any lyrics data in the first set of lyrics, and use it as the target lyrics data;

[0182] Obtain any one of the lyrics data and the target lyrics data to get a first similarity. If the first similarity is greater than a first threshold, add the any one of the lyrics data to the target lyrics set.

[0183] Using a reference track lyrics set determined based on the target lyrics set, a second similarity is calculated between the target lyrics set and the reference track lyrics set. The content of the lyrics in the target lyrics set is then adjusted to obtain the lyrics file corresponding to the music video.

[0184] In one embodiment, the processor also performs the steps described in the other embodiments when executing the computer program.

[0185] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0186] According to the preset frame extraction rules, multiple candidate video frames are extracted from the music video;

[0187] Based on the difference information between any candidate video frame and the previous candidate video frame of any candidate video frame, the target video frame is determined from the plurality of candidate video frames to obtain a video frame set;

[0188] Audio data is extracted from the music video according to a preset audio format.

[0189] The lyrics data of each video frame in the video frame set are extracted to obtain a first lyrics set, and the lyrics data of the audio data are extracted to obtain a second lyrics set;

[0190] From the second set of lyrics, determine the lyrics data that corresponds to the timestamp of any lyrics data in the first set of lyrics, and use it as the target lyrics data;

[0191] Obtain any one of the lyrics data and the target lyrics data to get a first similarity. If the first similarity is greater than a first threshold, add the any one of the lyrics data to the target lyrics set.

[0192] Using a reference track lyrics set determined based on the target lyrics set, a second similarity is calculated between the target lyrics set and the reference track lyrics set. The content of the lyrics in the target lyrics set is then adjusted to obtain the lyrics file corresponding to the music video.

[0193] In one embodiment, the computer program, when executed by a processor, also implements the steps described in the other embodiments above.

[0194] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, performs the following steps:

[0195] According to the preset frame extraction rules, multiple candidate video frames are extracted from the music video;

[0196] Based on the difference information between any candidate video frame and the previous candidate video frame of any candidate video frame, the target video frame is determined from the plurality of candidate video frames to obtain a video frame set;

[0197] Audio data is extracted from the music video according to a preset audio format.

[0198] The lyrics data of each video frame in the video frame set are extracted to obtain a first lyrics set, and the lyrics data of the audio data are extracted to obtain a second lyrics set;

[0199] From the second set of lyrics, determine the lyrics data that corresponds to the timestamp of any lyrics data in the first set of lyrics, and use it as the target lyrics data;

[0200] Obtain any one of the lyrics data and the target lyrics data to get a first similarity. If the first similarity is greater than a first threshold, add the any one of the lyrics data to the target lyrics set.

[0201] Using a reference track lyrics set determined based on the target lyrics set, a second similarity is calculated between the target lyrics set and the reference track lyrics set. The content of the lyrics in the target lyrics set is then adjusted to obtain the lyrics file corresponding to the music video.

[0202] In one embodiment, the computer program, when executed by a processor, also implements the steps described in the other embodiments above.

[0203] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data shall comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0204] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0205] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0206] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for generating lyrics files, characterized in that, The method includes: According to the preset frame extraction rules, multiple candidate video frames are extracted from the music video; Based on the difference information between any candidate video frame and the previous candidate video frame of any candidate video frame, the target video frame is determined from the plurality of candidate video frames to obtain a video frame set; Audio data is extracted from the music video according to a preset audio format. The lyrics data of each video frame in the video frame set are extracted to obtain a first lyrics set, and the lyrics data of the audio data are extracted to obtain a second lyrics set; From the second set of lyrics, determine the lyrics data that corresponds to the timestamp of any lyrics data in the first set of lyrics, and use it as the target lyrics data; Obtain any one of the lyrics data and the target lyrics data to get a first similarity. If the first similarity is greater than a first threshold, add the any one of the lyrics data to the target lyrics set. Using a reference track lyrics set determined based on the target lyrics set, a second similarity is calculated between the target lyrics set and the reference track lyrics set. The content of the lyrics in the target lyrics set is then adjusted to obtain the lyrics file corresponding to the music video.

2. The method according to claim 1, characterized in that, The step of extracting lyrics data from each video frame in the video frame set includes: For any given video frame, extract the text information contained in that video frame to obtain multiple extracted texts; The multiple extracted texts are filtered according to preset text removal information to obtain filtered extracted text; the preset text removal information is used to remove non-lyric text content. By aggregating the filtered and extracted text within any video frame, the lyrics data for that video frame are obtained.

3. The method according to claim 1 or 2, characterized in that, The process of obtaining the first set of lyrics includes: If it is detected that a certain video frame meets the inter-frame merging condition with a consecutive video frame adjacent to the certain video frame, the lyrics data of the certain video frame and the lyrics data of the consecutive video frames are merged to obtain merged lyrics, and the merged lyrics are added to the first lyrics set. If a video frame is detected as not meeting the inter-frame merging condition, the lyrics data corresponding to that video frame is added to the first lyrics set.

4. The method according to claim 3, characterized in that, Each lyric data in the first lyric set corresponds to a subtitle extraction region. The step of detecting that a certain video frame and consecutive video frames adjacent to that video frame satisfy the inter-frame merging condition includes: If the subtitle extraction area between a certain video frame and the next adjacent video frame is within a preset coordinate range; Obtain the third similarity of lyrics data between a certain video frame and the next video frame; If the third similarity satisfies a preset threshold, it is determined that the inter-frame merging condition is met between a certain video frame and consecutive video frames adjacent to the certain video frame.

5. The method according to claim 1, characterized in that, Before the steps of using a reference track lyrics set determined based on the target lyrics set, calculating a second similarity between the target lyrics set and the reference track lyrics set, adjusting the content of the lyrics in the target lyrics set, and obtaining the lyrics file corresponding to the music video, the method further includes: Determine the reference tracks corresponding to each of the target lyrics; the reference tracks include multiple reference songs. The set of lyrics for the reference tracks is obtained based on the reference lyrics of the reference songs that meet the song selection criteria.

6. The method according to claim 1, characterized in that, The step involves using a reference track lyrics set determined based on the target lyrics set, calculating a second similarity between the target lyrics set and the reference track lyrics set, adjusting the content of the lyrics in the target lyrics set, and obtaining the lyrics file corresponding to the music video, including: For any target lyrics, determine the second similarity between the target lyrics and any reference lyrics in the reference song lyrics set, the first offset value of the target lyrics in the target lyrics set, and the second offset value of the reference lyrics in the reference song lyrics set; The first offset value and the second offset value corresponding to the largest second similarity are used as the reference offset value; Adjust the content of any target lyrics according to the reference offset value to obtain the adjusted target lyrics, and generate the lyrics file based on the adjusted target lyrics.

7. The method according to claim 6, characterized in that, The step of adjusting the content of any target lyrics according to the reference offset value to obtain the adjusted target lyrics includes: Based on the aforementioned baseline offset value, the matching region is determined from the reference track lyrics set by shifting the first offset value and the second offset value upwards and downwards respectively; Using the reference lyrics in the matching area, typos in any of the target lyrics are corrected to obtain the adjusted target lyrics.

8. The method according to claim 1, characterized in that, Before the step of extracting multiple candidate video frames from the music video according to a preset frame extraction rule, the method further includes: Obtain preset video filtering information; the video filtering information includes video duration thresholds and reference parameter conditions; If the duration of the input music video does not exceed the video duration threshold and the video parameters of the input music video meet the reference parameter conditions, the input music video will be used as the music video.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 8.

Images