Lyrics extraction method, device, system and computer-readable storage medium
Through interval extraction and clustering technology, the problem of lyric text redundancy in MV videos is solved, efficient lyric extraction is achieved, redundancy is reduced and recognition accuracy is improved, and the lyric text repetition and redundancy in traditional methods is solved. question.
Patent Information
- Application Number
- CN202111529212.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-12-14
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2041-12-14
AI Technical Summary
In the KTV environment, many MV videos lack corresponding lyric text files. Traditional subtitle recognition methods lead to duplication and redundancy in lyric texts. The existing technology has failed to effectively solve the problem of redundancy in the text information of the same lyrics.
By extracting video frames at intervals, text extraction and clustering are performed, optimal standard text information is selected as lyric text, and text similarity calculation and clustering are used to reduce redundancy and improve recognition accuracy.
It effectively reduces the redundancy and data volume of lyric text, improves the recognition accuracy and the quality of lyric text files, and improves data processing efficiency.
Smart Images

Figure CN114120207B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of data processing, and in particular to a lyrics extraction method, device, system and computer-readable storage medium. Background Art
[0002] In a KTV environment, many MV videos with subtitles do not have corresponding lyrics text files, wherein the lyrics text files contain lyrics information of the singing order.
[0003] Traditional MV subtitle recognition methods usually extract each frame of the MV and then extract text frame by frame, which will cause many repeated lyrics in the recognized lyrics text. In addition, because the length of time each singer sings a line of lyrics is different, the same lyrics text after recognition will be redundant.
[0004] There is currently no effective solution to the problem of redundant information of the same lyrics text in the prior art. Summary of the invention
[0005] To solve the above problems, the present invention provides a lyrics extraction method, device, system and computer-readable storage medium, which extract video frames at intervals, perform text extraction on the extracted video frames, perform text clustering on all extracted text information to obtain a set with the same text information, and then extract an optimal standard text information from each set as the lyrics text in a lyrics text file, and obtain a lyrics text file containing the singing order through the corresponding timestamp.
[0006] To achieve the above-mentioned purpose, the present invention provides a lyric extraction method, comprising: extracting frames in a target video as target frames at predetermined time intervals; identifying the timestamp of the current target frame and the standard text information set of the current target frame; calculating the similarity between one of the standard text information in the standard text information set and the reference text information in each subset of the current clustered text information set, respectively, to obtain the text similarity corresponding to each subset; when the text similarity is not less than a first preset similarity threshold of the corresponding subset, adding the standard text information to the corresponding subset to obtain the latest clustered text information set; when all the text similarities are less than the first preset similarity threshold of the corresponding subset, using the standard text information as the reference text information of the latest subset, deleting the head subset in the current clustered text information set, adding the latest subset to the tail of the current clustered text information set, to obtain the latest clustered text information set, and using the best standard text information in the head subset as the lyric text information corresponding to the timestamp.
[0007] Further optionally, the method of using the optimal standard text information in the header subset as the lyric text information of the corresponding timestamp includes: calculating the text similarity of any two standard text information in the header subset; forming a set of multiple standard text information whose text similarity is greater than a second preset similarity threshold to obtain at least one identical text information set; counting the number of standard text information in all identical text information sets, and arranging the identical text information sets in descending order according to the number of standard text information; and randomly selecting any standard text information from the identical text information set ranked first as the optimal standard text information.
[0008] Further optionally, the method includes: if the current target frame is the initial target frame, using the standard text information set in the initial target frame as the latest clustered text information set, wherein each standard text information is the reference text information of the corresponding subset.
[0009] Further optionally, the identifying the timestamp of the current target frame and the standard text information set of the current target frame includes: binarizing the current target frame; and extracting text from the binarized current target frame to obtain a standard text information set including at least one standard text information.
[0010] On the other hand, the present invention provides a lyric extraction device, comprising: a data acquisition module, used to extract frames in a target video as target frames at predetermined time intervals; an identification module, used to identify the timestamp of the current target frame and the standard text information set of the current target frame; a calculation module, used to calculate the similarity between one of the standard text information in the standard text information set and the reference text information in each subset of the current cluster text information set, so as to obtain the text similarity corresponding to each subset; a first cluster text information set generation module, used to add the standard text information to the corresponding subset when the text similarity is not less than a first preset similarity threshold of the corresponding subset, so as to obtain the latest cluster text information set; a second cluster text information set generation module, used to use the standard text information as the reference text information of the latest subset, delete the head subset in the current cluster text information set, add the latest subset to the tail of the current cluster text information set, so as to obtain the latest cluster text information set, and use the best standard text information in the head subset as the lyric text information of the corresponding timestamp.
[0011] Further optionally, the second clustering text information set generation module includes: a text similarity calculation submodule, used to calculate the text similarity of any two standard text information in the head subset; a same text information set generation submodule, used to form a set of multiple standard text information with text similarity greater than a second preset similarity threshold into a same text information set to obtain at least one same text information set; a sorting submodule, used to count the number of standard text information in all same text information sets, and arrange the same text information sets in descending order according to the number of standard text information; a selection submodule, used to randomly select any standard text information from the same text information set ranked first as the optimal standard text information.
[0012] Further optionally, the device includes: if the current target frame is the initial target frame, using the standard text information set in the initial target frame as the latest clustered text information set, wherein each standard text information is the reference text information of the corresponding subset.
[0013] Further optionally, the recognition module includes: a binarization submodule, used to binarize the current target frame; and a text extraction submodule, used to extract text from the binarized current target frame to obtain a standard text information set including at least one standard text information.
[0014] On the other hand, the present invention also provides a lyrics extraction system, including the above-mentioned lyrics extraction device.
[0015] On the other hand, the present invention further provides a computer-readable storage medium having a computer program stored thereon, wherein the program implements the above-mentioned lyrics extraction method when executed by a processor.
[0016] The above technical scheme has the following beneficial effects: extracting target frames from the target video at predetermined time intervals, and extracting lyrics text from the target frames, reducing the repetition rate of text information and the amount of data, and increasing data processing efficiency while ensuring recognition accuracy; extracting the corresponding standard text information set from each target frame, clustering each standard text information in the standard text information set with the text information in the current clustering text information set, and obtaining multiple identical text information with similar text similarities, and selecting the best text information from these text information as the lyrics text corresponding to the final timestamp, thereby achieving the purpose of selecting a lyrics text from multiple identical lyrics texts, reducing the redundancy of the lyrics text, and improving the recognition accuracy and the quality of the final lyrics text file. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0018] Figure 1 is a flow chart of a lyrics extraction method provided by an embodiment of the present invention;
[0019] Figure 2 is a flow chart of an optimal standard text information selection method provided by an embodiment of the present invention;
[0020] Figure 3 is a flow chart of a method for generating a standard text information set provided by an embodiment of the present invention;
[0021] Figure 4 is a structural schematic diagram of a lyrics extraction device provided by an embodiment of the present invention;
[0022] Figure 5 It is a structural schematic diagram of a second clustering text information set generating module provided in an embodiment of the present invention;
[0023] Figure 6 It is a structural diagram of the identification module provided by an embodiment of the present invention.
[0024] Reference numerals: 100 - data acquisition module 200 - identification module 2001 - binarization submodule 2002 - text extraction submodule 300 - calculation module 400 - first clustering text information set generation module 500 - second clustering text information set generation module 5001 - text similarity calculation submodule 5002 - identical text information set generation submodule 5003 - sorting submodule 5004 - selection submodule 600 - initial text clustering information set generation module DETAILED DESCRIPTION
[0025] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0026] In order to solve the problem of redundancy of lyrics text in the lyrics text file generated after text extraction of MV subtitles, an embodiment of the present invention provides a lyrics extraction method. Figure 1 is a flowchart of a lyrics extraction method provided by an embodiment of the present invention, such as Figure 1As shown, the method includes:
[0027] S101, extracting frames in a target video as target frames at predetermined time intervals;
[0028] In the target video, each lyric text has a certain time length, that is, each lyric text will appear in multiple consecutive frames of the target video. If the target video is extracted frame by frame, multiple repeated lyric texts will inevitably be extracted. Therefore, this embodiment extracts frames of the target video at predetermined intervals, reduces the repetition rate of the extracted lyric texts, and improves the efficiency of subsequent data processing. Among them, the setting of the predetermined time interval needs to ensure that each lyric text can be extracted. Preferably, in this embodiment, the target video is iterated from 0S, and the predetermined time interval is 0.5S.
[0029] S102, identifying the timestamp of the current target frame and the standard text information set of the current target frame;
[0030] Text extraction is performed on each target frame to obtain a standard text information set in each target frame. Each standard text information set contains at least one standard text information. In this embodiment, there are two standard text information in each standard text information set, corresponding to two lines of subtitles in the MV video, and the two standard text information are two lines of character strings in the subtitles. This embodiment is described with a target frame as the main body, and the standard text information set in the current target frame collected at the current moment and the timestamp corresponding to the target frame are extracted.
[0031] S103, calculating similarity between one of the standard text information in the standard text information set and the reference text information in each subset of the current clustered text information set, to obtain text similarity corresponding to each subset;
[0032] The current clustering text information set is used for clustering standard text information, which includes at least one subset, each subset includes at least one text information, and the current clustering text information set is obtained by clustering the previous standard text information, that is, the text information in each subset is the standard text information that was successfully clustered in the forward step.
[0033] In this embodiment, each clustered text information set contains two subsets, corresponding to two lines of subtitles in the MV video, and each subset represents a cluster set of the same lyrics text.
[0034] This embodiment calculates the similarity between each standard text information in the standard text information set corresponding to the current target frame and the reference text information in each subset of the current clustered text information to obtain the text similarity corresponding to each subset, that is, to determine whether the standard text information currently performing similarity calculation and the reference text information are the same lyrics text.
[0035] In this embodiment, the text similarity is obtained by calculating the Hamming distance of two text information. The Hamming distance is a special case of the edit distance, which is only used to calculate the number of inconsistent characters in two strings. Therefore, the Hamming distance does not need to consider addition and deletion, but only needs to compare differences, so it is relatively simple to implement.
[0036] S104, when the text similarity is not less than the first preset similarity threshold of the corresponding subset, adding the standard text information to the corresponding subset to obtain the latest clustered text information set;
[0037] When the text similarity is not less than the first preset similarity threshold corresponding to itself, the standard text information and the reference text information in the subset are deemed to be the same lyrics text. At this time, the standard text information is added to the corresponding subset to update the latest clustered text information set.
[0038] S105. When all text similarities are less than the first preset similarity threshold of the corresponding subset, the standard text information is used as the reference text information of the latest subset, the head subset in the current clustered text information set is deleted, the latest subset is added to the end of the current clustered text information set to obtain the latest clustered text information set, and the optimal standard text information in the head subset is used as the lyrics text information of the corresponding timestamp.
[0039] When the text similarity corresponding to each subset is less than the first preset similarity threshold, it is considered that the standard text information is different from the reference text information in any subset and is a newly appeared lyric text. At this time, the standard text information is used as the reference text information of the latest subset, and the head subset in the current clustered text information is deleted, and the latest subset is added to the end of the current clustered text information set to update the latest clustered text information set. Among them, the subsets in each clustered text information set are arranged in order of appearance, and the head subset is the data subset with the longest storage time.
[0040] When new lyrics text appears, the head subset at this time is identified as the data set that completes the clustering of all identical lyrics texts, from which an optimal standard information set is selected as the representative lyrics text, and it is added to the lyrics text file according to its timestamp.
[0041] As an optional implementation, Figure 2 is a flow chart of the optimal standard text information selection method provided by an embodiment of the present invention, such as Figure 2 As shown, S105, using the optimal standard text information in the header subset as the lyrics text information corresponding to the timestamp, including:
[0042] S1051, calculating the text similarity of any two standard text information in the header subset;
[0043] S1052, grouping a plurality of standard text information whose text similarity is greater than a second preset similarity threshold into a set, to obtain at least one identical text information set;
[0044] S1053, counting the number of standard text information in all identical text information sets, and arranging the identical text information sets in descending order according to the number of standard text information;
[0045] S1054: Randomly select any standard text information from the first-ranked identical text information set as the optimal standard text information.
[0046] When recognizing each target frame, there may be a recognition error rate, which will cause some lyrics texts to appear in the head subset that have a certain gap in similarity with other text information.
[0047] To solve this problem, this embodiment traverses all elements in the header subset and selects the standard text information with the most occurrences as the optimal text information, that is, the standard text information with the most occurrences is used as the lyrics text that is finally added to the lyrics text file.
[0048] Specifically, the similarity of every two standard text information in the head subset is calculated, and the standard text information whose text similarity is greater than the second preset threshold is grouped into a set, which is called the same text information set. The second preset threshold is greater than the first preset similarity threshold, and the standard text information with identification problems is eliminated from the same text information set, and the standard text information in the same text information set is identified as the same lyrics text.
[0049] Since there may be multiple identical text information sets after clustering, these identical text information sets are sorted in descending order according to the number of standard text information in the set, and one standard text information is selected from the identical text information set at the head as the optimal standard text information.
[0050] In this embodiment, the text similarity is obtained by calculating the Hamming distance between two text information.
[0051] As an optional implementation, the method further includes S100: if the current target frame is the initial target frame, using the standard text information set in the initial target frame as the latest clustered text information set, wherein each standard text information is the reference text information of the corresponding subset.
[0052] In the initial state, the clustering text information set is an empty set. When the initial target frame is clustered with the initial clustering text information set, since there is no corresponding reference standard text information, the standard text information set is used as the latest clustering text information set, and each standard text information is used as a reference text information of a subset to facilitate subsequent data processing.
[0053] As an optional implementation, Figure 3 is a flow chart of a method for generating a standard text information set provided by an embodiment of the present invention. Figure 3 As shown, S102, identifying the timestamp of the current target frame and the standard text information set of the current target frame includes:
[0054] S1021, binarizing the current target frame;
[0055] Binarize the current target frame to highlight the lyrics text area.
[0056] S1022: Perform text extraction on the binarized current target frame to obtain a standard text information set including at least one standard text information.
[0057] The binarized image is called Tesseract for OCR recognition. Each recognized string is the standard text information, and all the standard text information of the frame is combined into a standard text information set.
[0058] OCR (optical character recognition) is the process by which an electronic device (such as a scanner or digital camera) examines characters printed on paper and then uses character recognition methods to translate the shapes into computer text; that is, the text data is scanned and then the image file is analyzed and processed.
[0059] Tesseract is an open source OCR (Optical Character Recognition) engine developed by HP Labs and maintained by Google.
[0060] The embodiment of the present invention also provides a lyrics extraction device, Figure 4 is a schematic diagram of the structure of a lyrics extraction device provided by an embodiment of the present invention. Figure 4 As shown, the device comprises:
[0061] The data acquisition module 100 is used to extract frames in the target video as target frames at predetermined time intervals;
[0062] In the target video, each lyric text has a certain time length, that is, each lyric text will appear in multiple consecutive frames of the target video. If the target video is extracted frame by frame, multiple repeated lyric texts will inevitably be extracted. Therefore, this embodiment extracts frames of the target video at predetermined intervals, reduces the repetition rate of the extracted lyric texts, and improves the efficiency of subsequent data processing. Among them, the setting of the predetermined time interval needs to ensure that each lyric text can be extracted. Preferably, in this embodiment, the target video is iterated from 0S, and the predetermined time interval is 0.5S.
[0063] An identification module 200 is used to identify the timestamp of the current target frame and the standard text information set of the current target frame;
[0064] Text extraction is performed on each target frame to obtain a standard text information set in each target frame. Each standard text information set contains at least one standard text information. In this embodiment, there are two standard text information in each standard text information set, corresponding to two lines of subtitles in the MV video, and the two standard text information are two lines of character strings in the subtitles. This embodiment is described with a target frame as the main body, and the standard text information set in the current target frame collected at the current moment and the timestamp corresponding to the target frame are extracted.
[0065] The calculation module 300 is used to calculate the similarity between one of the standard text information in the standard text information set and the reference text information in each subset of the current clustered text information set, so as to obtain the text similarity corresponding to each subset;
[0066] The current clustering text information set is used for clustering standard text information, which includes at least one subset, each subset includes at least one text information, and the current clustering text information set is obtained by clustering the previous standard text information, that is, the text information in each subset is the standard text information that was successfully clustered in the forward step.
[0067] In this embodiment, each clustered text information set contains two subsets, corresponding to two lines of subtitles in the MV video, and each subset represents a cluster set of the same lyrics text.
[0068] This embodiment calculates the similarity between each standard text information in the standard text information set corresponding to the current target frame and the reference text information in each subset of the current clustered text information to obtain the text similarity corresponding to each subset, that is, to determine whether the standard text information currently performing similarity calculation and the reference text information are the same lyrics text.
[0069] In this embodiment, the text similarity is obtained by calculating the Hamming distance of two text information. The Hamming distance is a special case of the edit distance, which is only used to calculate the number of inconsistent characters in two strings. Therefore, the Hamming distance does not need to consider addition and deletion, but only needs to compare differences, so it is relatively simple to implement.
[0070] A first clustered text information set generation module 400 is used to add the standard text information to the corresponding subset to obtain the latest clustered text information set when the text similarity is not less than a first preset similarity threshold of the corresponding subset;
[0071] When the text similarity is not less than the first preset similarity threshold corresponding to itself, the standard text information and the reference text information in the subset are deemed to be the same lyrics text. At this time, the standard text information is added to the corresponding subset to update the latest clustered text information set.
[0072] The second clustering text information set generation module 500 is used for the second clustering text information set generation module. When all text similarities are less than the first preset similarity threshold of the corresponding subset, the standard text information is used as the reference text information of the latest subset, the head subset in the current clustering text information set is deleted, the latest subset is added to the end of the current clustering text information set to obtain the latest clustering text information set, and the optimal standard text information in the head subset is used as the lyrics text information of the corresponding timestamp.
[0073] When the text similarity corresponding to each subset is less than the first preset similarity threshold, it is considered that the standard text information is different from the reference text information in any subset and is a newly appeared lyric text. At this time, the standard text information is used as the reference text information of the latest subset, and the head subset in the current clustered text information is deleted, and the latest subset is added to the end of the current clustered text information set to update the latest clustered text information set. Among them, the subsets in each clustered text information set are arranged in order of appearance, and the head subset is the data subset with the longest storage time.
[0074] When new lyrics text appears, the head subset at this time is identified as the data set that completes the clustering of all identical lyrics texts, from which an optimal standard information set is selected as the representative lyrics text, and it is added to the lyrics text file according to its timestamp.
[0075] As an optional implementation, Figure 5 is a schematic diagram of the structure of a second clustering text information set generation module provided in an embodiment of the present invention, such as Figure 5 As shown, the second clustering text information set generation module 500 includes:
[0076] The text similarity calculation submodule 5001 is used to calculate the text similarity of any two standard text information in the header subset;
[0077] The same text information set generating submodule 5002 is used for the same text information set to form a set of multiple standard text information with text similarity greater than a second preset similarity threshold to obtain at least one same text information set;
[0078] The sorting submodule 5003 is used to count the number of standard text information in all identical text information sets, and sort the identical text information sets in descending order according to the number of standard text information;
[0079] The selection submodule 5004 is used to randomly select any standard text information from the first-ranked identical text information set as the optimal standard text information.
[0080] When recognizing each target frame, there may be a recognition error rate, which will cause some lyrics texts to appear in the head subset that have a certain gap in similarity with other text information.
[0081] To solve this problem, this embodiment traverses all elements in the header subset and selects the standard text information with the most occurrences as the optimal text information, that is, the standard text information with the most occurrences is used as the lyrics text that is finally added to the lyrics text file.
[0082] Specifically, the similarity of every two standard text information in the head subset is calculated, and the standard text information whose text similarity is greater than the second preset threshold is grouped into a set, which is called the same text information set. The second preset threshold is greater than the first preset similarity threshold, and the standard text information with identification problems is eliminated from the same text information set, and the standard text information in the same text information set is identified as the same lyrics text.
[0083] Since there may be multiple identical text information sets after clustering, these identical text information sets are sorted in descending order according to the number of standard text information in the set, and one standard text information is selected from the identical text information set at the head as the optimal standard text information.
[0084] In this embodiment, the text similarity is obtained by calculating the Hamming distance between two text information.
[0085] As an optional implementation, the device also includes an initial text clustering information set generation module 600, which is used to use the standard text information set in the initial target frame as the latest clustering text information set if the current target frame is the initial target frame, wherein each standard text information is the reference text information of the corresponding subset.
[0086] In the initial state, the clustering text information set is an empty set. When the initial target frame is clustered with the initial clustering text information set, since there is no corresponding reference standard text information, the standard text information set is used as the latest clustering text information set, and each standard text information is used as a reference text information of a subset to facilitate subsequent data processing.
[0087] As an optional implementation, Figure 6 is a schematic diagram of the structure of the identification module provided by an embodiment of the present invention, such as Figure 6 As shown, the identification module 200 includes:
[0088] Binarization submodule 2001, used for binarizing the current target frame;
[0089] Binarize the current target frame to highlight the lyrics text area.
[0090] The text extraction submodule 2002 is used to extract text from the current target frame after binarization to obtain a standard text information set including at least one standard text information.
[0091] The binarized image is called Tesseract for OCR recognition. Each recognized string is the standard text information, and all the standard text information of the frame is combined into a standard text information set.
[0092] OCR (optical character recognition) is the process by which an electronic device (such as a scanner or digital camera) examines characters printed on paper and then uses character recognition methods to translate the shapes into computer text; that is, the text data is scanned and then the image file is analyzed and processed.
[0093] Tesseract is an open source OCR (Optical Character Recognition) engine developed by HP Labs and maintained by Google. Specific embodiment 1:
[0095] Prepare MV video files. Iterate the video from 0 seconds, extract a frame of MV video image every 0.5 seconds, perform binarization and highlight the lyrics text area. Call Tesseract to perform OCR recognition on the binarized image. Save the OCR recognition results. Iterate until the end of the video. You can get the lyrics dataset of this song. Each sample in the dataset contains two elements, picture frame-frame timestamp-recognized string text.
[0096] Iterate the data set. Because the same lyrics may appear in multiple video frames, lyrics clustering and deduplication are required. The clustering method is to iterate the data set and get element i. Because most mv subtitles are two lines, an empty priority queue L1 with only two set elements is established. The two set elements are W1 and W2 to traverse the recognition results of element i. The Hamming distance of the string is calculated with the set elements in L1 for each line of recognition results. It is determined whether the two strings are similar to each subset of the set elements in L1. If at least one subset is similar, it is considered to be a lyric, and the lyrics are clustered into the set Wx. Finally, each element in L1 will be a list of sets of indefinite length, and the set items in the set are all similar lyrics texts. If the element currently traversing i is not clustered into any element in L1. It is considered to be the appearance of a new lyric. Then the first set element W1 in L1 is removed from L1, and the new element i is inserted into the tail of L1 to generate W3. Generate a new L1 with only two elements W2 and W3.
[0097] Traverse all the lyrics elements in W1 and select the lyrics with the most occurrences as the final lyrics line, and return to iterate until all elements in the data set have been iterated, and sort all the lyrics by the time in L1 that pops up to generate a complete lyrics text.
[0098] The scheme extracts MV video frames for OCR recognition at fixed intervals, adds strings with high similarity to queue set elements for clustering lyrics, and pops up the first element of the queue until a dissimilar lyric line appears. Finally, the queue exit time of all pop-up elements is arranged to form a complete lyric text. This method can generate lyric text in the singing order based on the video without lyric files, including local and Internet ones.
[0099] On the other hand, the present invention also provides a lyrics extraction system, including the above-mentioned lyrics extraction device.
[0100] On the other hand, the present invention also provides a computer-readable storage medium having a computer program stored thereon, and the program implements the above-mentioned lyrics extraction method when executed by a processor.
[0101] The above-mentioned software is stored in the above-mentioned storage medium, which includes but is not limited to: a CD, a floppy disk, a hard disk, an erasable memory, etc.
[0102] The above technical scheme has the following beneficial effects: extracting target frames from the target video at predetermined time intervals, and extracting lyrics text from the target frames, reducing the repetition rate of text information and the amount of data, and increasing data processing efficiency while ensuring recognition accuracy; extracting the corresponding standard text information set from each target frame, clustering each standard text information in the standard text information set with the text information in the current clustering text information set, and obtaining multiple identical text information with similar text similarities, and selecting the best text information from these text information as the lyrics text corresponding to the final timestamp, thereby achieving the purpose of selecting a lyrics text from multiple identical lyrics texts, reducing the redundancy of the lyrics text, and improving the recognition accuracy and the quality of the final lyrics text file.
[0103] The specific implementation methods of the above invention further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above invention is only a specific implementation method of the present invention and is not intended to limit the protection scope of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.
Claims
1. A lyrics extraction method, characterized in that: include: Extracting frames in the target video as target frames at predetermined time intervals; Identify the timestamp of the current target frame and the standard text information set of the current target frame; Calculating the similarity between one of the standard text information in the standard text information set and the reference text information in each subset of the current clustered text information set to obtain the text similarity corresponding to each subset; When the text similarity is not less than a first preset similarity threshold of the corresponding subset, adding the standard text information to the corresponding subset to obtain a latest clustered text information set; When all the text similarities are less than the first preset similarity threshold of the corresponding subset, the standard text information is used as the reference text information of the latest subset, the head subset in the current clustered text information set is deleted, the latest subset is added to the tail of the current clustered text information set to obtain the latest clustered text information set, and the optimal standard text information in the head subset is used as the lyrics text information of the corresponding timestamp; The step of using the optimal standard text information in the header subset as the lyrics text information corresponding to the timestamp includes: Calculate the text similarity of any two standard text information in the header subset; Grouping a plurality of standard text information whose text similarity is greater than a second preset similarity threshold into a set to obtain at least one identical text information set; Counting the number of standard text information in all identical text information sets, and sorting the identical text information sets in descending order according to the number of standard text information; Any standard text information is randomly selected from the first-ranked identical text information set as the optimal standard text information.
2. The lyrics extraction method according to claim 1, characterized in that: include: If the current target frame is the initial target frame, the standard text information set in the initial target frame is used as the latest clustered text information set, wherein each standard text information is the reference text information of the corresponding subset.
3. The lyrics extraction method according to claim 1, characterized in that: The timestamp for identifying the current target frame and the standard text information set for the current target frame include: Binarizing the current target frame; Text extraction is performed on the binarized current target frame to obtain a standard text information set including at least one standard text information.
4. A lyrics extraction device, characterized in that: include: A data acquisition module, used for extracting frames in a target video as target frames at predetermined time intervals; An identification module, used for identifying the timestamp of the current target frame and the standard text information set of the current target frame; A calculation module, used for calculating the similarity between one of the standard text information in the standard text information set and the reference text information in each subset of the current clustered text information set, to obtain the text similarity corresponding to each subset; A first clustered text information set generating module, configured to add the standard text information to the corresponding subset to obtain a latest clustered text information set when the text similarity is not less than a first preset similarity threshold of the corresponding subset; A second clustering text information set generation module is used to, when all the text similarities are less than a first preset similarity threshold of the corresponding subset, use the standard text information as the reference text information of the latest subset, delete the head subset in the current clustering text information set, add the latest subset to the end of the current clustering text information set to obtain the latest clustering text information set, and use the optimal standard text information in the head subset as the lyrics text information of the corresponding timestamp; The second clustering text information set generating module comprises: A text similarity calculation submodule, used to calculate the text similarity of any two standard text information in the header subset; The same text information set generation submodule is used for the same text information set to form a set of multiple standard text information with text similarity greater than a second preset similarity threshold to obtain at least one same text information set; A sorting submodule is used to count the number of standard text information in all identical text information sets, and sort the identical text information sets in descending order according to the number of standard text information; The selection submodule is used to randomly select any standard text information from the first-ranked identical text information set as the optimal standard text information.
5. The lyrics extraction device according to claim 4, characterized in that: include: The initial text clustering information set generation module is used to use the standard text information set in the initial target frame as the latest clustering text information set if the current target frame is the initial target frame, wherein each standard text information is the reference text information of the corresponding subset.
6. The lyrics extraction device according to claim 4, characterized in that: The identification module comprises: A binarization submodule, used for binarizing the current target frame; The text extraction submodule is used to extract text from the current target frame after binarization to obtain a standard text information set including at least one standard text information.
7. A lyrics extraction system, comprising the lyrics extraction device according to any one of claims 4 to 6.
8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the lyrics extraction method as described in any one of claims 1 to 3 is implemented.
