Audio data processing method and apparatus, device, and medium

Abstract

Embodiments of the present disclosure provide an audio data processing method, device, equipment and medium. The present disclosure obtains a first lyric string of each original singing audio segment and a second lyric string of each alternative cover singing audio data in at least one alternative cover singing audio data, so as to match each second lyric string based on the first lyric string corresponding to each original singing audio segment, to determine a cover singing audio segment matched with the original singing audio segment from the at least one alternative cover singing audio data, so as to realize the matching of the original singing audio segment and the cover singing audio segment. The matched original singing audio segment and cover singing audio segment can be used as training data for constructing feature extraction rules subsequently, so as to realize the expansion of the training data.

Description

TECHNICAL FIELD

[0001] Embodiments of the present disclosure relate to the field of multimedia technology, and more particularly, to an audio data processing method, device, equipment and medium. BACKGROUND

[0002] This section is intended to provide background or context to the embodiments of the present disclosure. The description herein is not admitted to be prior art merely by inclusion in this section.

[0003] With the popularity of short videos, variety shows and live broadcasts, the number of cover songs is increasing, and the scenario of song recognition is becoming more and more complex. The traditional song recognition function cannot meet the diversified and personalized recognition needs of users, and therefore, a cover recognition function emerges as the times require. Through the cover recognition function, users can match the original song fragments and cover song fragments, making the song search process more accurate and convenient.

[0004] When matching the original song and the cover song through the cover recognition function, according to a predetermined rule, features sufficient to represent the attributes of the song fragments are extracted, and then the similarity matching of the original song fragments and the cover song fragments is completed based on the extracted features, so as to find the original song corresponding to the cover song fragment. However, a large amount of training data is required in the process of constructing the rule, and therefore, an audio data processing method is needed to find the cover song corresponding to the original song fragment in the original song, and then construct the original song fragment and the corresponding cover song fragment as a cover pair, so as to use the cover pair as the training data for constructing the rule. SUMMARY

[0005] However, due to the insufficient number of original songs and cover songs as training data in the related art, the feature extraction rule constructed cannot extract features that can effectively represent the attributes of the song fragments, and thus the accuracy of matching the cover song and the original song is poor.

[0006] Based on this situation, in the present context, the embodiments of the present disclosure aim to provide an audio data processing method to obtain original song fragments and corresponding cover song fragments based on a limited number of original songs and cover songs, so as to expand the training data for constructing the rule.

[0007] In a first aspect of the embodiments of the present disclosure, an audio data processing method is provided, which comprises:

[0008] segmenting the original audio data to obtain a plurality of original audio fragments corresponding to the original audio data;

[0009] obtaining a first lyric string of each original audio clip and a second lyric string of each of the at least one alternative audio data;

[0010] For any original audio clip, based on the first lyric string, matching is performed in each second lyric string to determine a cover audio clip matching the original audio clip from the at least one alternative audio data.

[0011] In an embodiment of the present disclosure, based on the first lyric string, matching is performed in each second lyric string to determine a cover audio clip matching the original audio clip from the at least one alternative audio data, comprising:

[0012] Based on the first lyric string, matching is performed in each second lyric string to determine a target cover lyric string matching the first lyric string from the at least one second lyric string;

[0013] According to the time stamp corresponding to the start and end positions of the target cover lyric string, the alternative audio data corresponding to the target cover lyric string is segmented to obtain a cover audio clip matching the original audio clip.

[0014] In an embodiment of the present disclosure, based on the first lyric string, matching is performed in each second lyric string to determine a target cover lyric string matching the first lyric string from the at least one second lyric string, comprising:

[0015] Based on part of the characters in the first lyric string, matching is performed in each second lyric string to determine at least one alternative cover lyric string from the at least one second lyric string;

[0016] According to the string length of the first lyric string, the at least one alternative cover lyric string is extended to obtain at least one to-be-matched lyric string;

[0017] Based on the similarity between each to-be-matched lyric string and the first lyric string, a target cover lyric string matching the first lyric string is determined from the at least one to-be-matched lyric string.

[0018] In an embodiment of the present disclosure, based on part of the characters in the first lyric string, matching is performed in each second lyric string to determine at least one alternative cover lyric string from the at least one second lyric string, comprising:

[0019] For any second lyric string, the second lyric string is traversed to determine the similarity between the plurality of string segments corresponding to the second lyric string and the part of the characters in the first lyric string;

[0020] The string fragments in the plurality of string fragments that satisfy the first preset condition are determined as candidate cover song lyric strings.

[0021] In an embodiment of the present disclosure, the at least one candidate cover song lyric string is extended according to the string length of the first lyric string to obtain at least one to-be-matched lyric string, including:

[0022] For any candidate cover song lyric string, in the second lyric string corresponding to the candidate cover song lyric string, a lyric string centered on the candidate cover song lyric string and having the same length as the first lyric string is obtained as the to-be-matched lyric string corresponding to the candidate cover song lyric string.

[0023] In an embodiment of the present disclosure, before determining the target cover song lyric string matching the first lyric string from the at least one to-be-matched lyric string based on the similarity between each to-be-matched lyric string and the first lyric string, the method further includes:

[0024] The similarity between each to-be-matched lyric string and the first lyric string is determined.

[0025] In an embodiment of the present disclosure, the similarity between each to-be-matched lyric string and the first lyric string is determined, including:

[0026] For any to-be-matched lyric string, the similarity between the to-be-matched lyric string and the first lyric string is determined based on the edit distance between the to-be-matched lyric string and the first lyric string.

[0027] In an embodiment of the present disclosure, the similarity between the to-be-matched lyric string and the first lyric string is determined based on the edit distance between the to-be-matched lyric string and the first lyric string, including:

[0028] The difference between the edit distance and the difference in string length between the first lyric string and the to-be-matched lyric string is determined as the similarity between the to-be-matched lyric string and the first lyric string.

[0029] In an embodiment of the present disclosure, the target cover song lyric string matching the first lyric string is determined from the at least one to-be-matched lyric string based on the similarity between each to-be-matched lyric string and the first lyric string, including:

[0030] The to-be-matched lyric string in the at least one to-be-matched lyric string that satisfies the second preset condition with the first lyric string is determined as the target cover song lyric string.

[0031] In one embodiment of the present disclosure, the partial character is a plurality of characters located at the center of the first lyric string and having a set length.

[0032] In one embodiment of the present disclosure, the original singing audio data is segmented to obtain a plurality of original singing audio segments corresponding to the original singing audio data, including:

[0033] The original singing audio data is divided into a plurality of original singing audio segments with a preset time length based on the timestamp of each lyric in the original singing audio data.

[0034] In one embodiment of the present disclosure, for any original singing audio segment, the process of obtaining the first lyric string of the original singing audio segment includes:

[0035] Obtaining the lyric text of the original singing audio segment;

[0036] Removing the punctuation in the lyric text of the original singing audio segment, and converting the characters in the lyric text of the original singing audio segment into characters in a set format to obtain the first lyric string of the original singing audio segment.

[0037] In one embodiment of the present disclosure, for any alternative cover audio data, the process of obtaining the second lyric string of the alternative cover audio data includes:

[0038] Obtaining the lyric text of the alternative cover audio data;

[0039] Removing the punctuation in the lyric text of the alternative cover audio data, and converting the characters in the lyric text of the alternative cover audio data into characters in a set format to obtain the second lyric string of the alternative cover audio data.

[0040] In a second aspect of the embodiments of the present disclosure, an audio data processing device is provided, which includes:

[0041] The segmentation module is configured to segment the original singing audio data to obtain a plurality of original singing audio segments corresponding to the original singing audio data.

[0042] The obtaining module is configured to obtain the first lyric string of each original singing audio segment and the second lyric string of each alternative cover audio data in the at least one alternative cover audio data.

[0043] The determination module is configured to, for any original singing audio segment, based on the first lyric string, match in each second lyric string to determine a cover audio segment matching the original singing audio segment from the at least one alternative cover audio data.

[0044] In a third aspect of the embodiments of the present disclosure, a computing device is provided, which comprises a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements the operations performed by the audio data processing method provided in the first aspect and any of the embodiments of the first aspect when executing the program.

[0045] In a fourth aspect of the embodiments of the present disclosure, a computer readable storage medium is provided, which stores a program, and the program is executed by a processor to implement the operations performed by the audio data processing method provided in the first aspect and any of the embodiments of the first aspect.

[0046] In a fifth aspect of the embodiments of the present disclosure, a computer program product is provided, which comprises a computer program, and the program is executed by a processor to implement the operations performed by the audio data processing method provided in the first aspect and any of the embodiments of the first aspect.

[0047] The present disclosure achieves the matching of the original audio data segment and the cover audio data segment by, in the case that the original audio data is segmented to obtain a plurality of original audio data segments corresponding to the original audio data, obtaining a first lyric string of each original audio data segment and a second lyric string of each cover audio data in at least one alternative cover audio data, matching each second lyric string based on the first lyric string corresponding to each original audio data segment to determine a cover audio segment matching the original audio data segment from the at least one alternative cover audio data, so as to implement the matching of the original audio data segment and the cover audio data segment. The matched original audio data segment and cover audio data segment can be subsequently used as training data for constructing a feature extraction rule, so as to achieve the expansion of the training data. BRIEF DESCRIPTION OF DRAWINGS

[0048] The above and other objects, features and advantages of the exemplary embodiments of the present disclosure will be more apparent from the following detailed description taken in conjunction with the accompanying drawings, in which:

[0049] Figure 1 is a flowchart of an audio data processing method according to an exemplary embodiment of the present disclosure;

[0050] Figure 2 is a flowchart of an audio data processing method according to an exemplary embodiment of the present disclosure;

[0051] Figure 3 is a block diagram of an audio data processing device according to an exemplary embodiment of the present disclosure;

[0052] Figure 4is a schematic diagram of a computer-readable storage medium according to an exemplary embodiment of the present disclosure;

[0053] Figure 5 is a structural schematic diagram of a computing device according to an exemplary embodiment of the present disclosure;

[0054] In the drawings, identical or corresponding reference signs refer to identical or corresponding parts. DETAILED DESCRIPTION

[0055] The principles and spirits of the present disclosure will be described below with reference to several exemplary embodiments. It should be understood that these embodiments are given only to enable those skilled in the art to better understand and implement the present disclosure, and in no way limit the scope of the present disclosure. On the contrary, these embodiments are provided to make the present disclosure more thorough and complete, and to fully convey the scope of the present disclosure to those skilled in the art.

[0056] Those skilled in the art know that the embodiments of the present disclosure can be implemented as a system, device, apparatus, method or computer program product. Therefore, the present disclosure can be embodied in the form of a complete hardware, a complete software (including firmware, resident software, microcode, etc.), or a combination of hardware and software.

[0057] According to the embodiments of the present disclosure, an audio data processing method is proposed, which is used to identify cover audio segments matched with each original audio segment, so as to generate a cover data pair of the original audio segment and the cover audio segment. The generated cover data pair can be used as training data to obtain more training data to train the feature extraction rule, so that the feature extraction rule obtained by training can extract more reliable features representing the attributes of the song segment.

[0058] The above-mentioned audio data processing method can be executed by a computing device. The computing device can be a terminal device, such as a desktop computer, a portable computer, a notebook computer, a smart phone, a tablet computer, a smart watch, etc. Alternatively, the computing device can be a server, such as a single server, multiple servers, a server cluster, a cloud computing platform, etc. The present disclosure does not limit the device type of the computing device.

[0059] It should be noted that the above-mentioned application scenarios are only shown for the purpose of facilitating the understanding of the spirit and principles of the present disclosure, and the embodiments of the present disclosure are not limited in this respect. On the contrary, the embodiments of the present disclosure can be applied to any applicable scenario.

[0060] The audio data processing method provided by the present disclosure will be described in detail below in combination with the above introduction of the application scenarios. In order to facilitate the understanding of the method provided by the present disclosure, the related terms involved in the present disclosure will be introduced first.

[0061] Cover song recognition: Given a cover song, the singer, duration, tempo, arrangement of the cover song can be different from any original song, but the main melody of the cover song is the same as a certain original song. By extracting specific features (such as features that can represent the main melody of the cover song), the extracted features are matched with the corresponding features of each original song in the original song library to determine the original song with the highest similarity to the cover song from the original song library.

[0062] Fragment cover recognition: Given a short cover version of a music fragment (such as a music fragment of about 15 seconds), extract the fragment representation of the music fragment, match it with the existing music fragment representation in the original song library, and return the song identification (ID) of the original song with the highest similarity to the music fragment, to find the corresponding original song through a cover song fragment.

[0063] Edit distance: A quantitative measure of the difference between two strings, which measures how many operations are needed to change one string to another. A commonly used edit distance definition is the Levenshtein distance, which refers to the minimum number of editing operations required to change one string to another. Allowed editing operations include replacing a character with another character, inserting a character, and deleting a character.

[0064] After introducing the related terms involved in the present disclosure, the audio data processing method provided according to an exemplary embodiment of the present disclosure will be described below with reference to Figure 1 .

[0065] Referring to Figure 1 , Figure 1 is a flowchart of an audio data processing method according to an exemplary embodiment of the present disclosure, as shown in Figure 1 , the method comprises:

[0066] S101, segmenting the original song audio data to obtain a plurality of original song audio fragments corresponding to the original song audio data.

[0067] It should be noted that the computing device can maintain an original song audio database, which can be used to store a plurality of original song audio data. For any original song audio data in the original song audio database, the cover audio fragments corresponding to each original song audio fragment in the original song audio data can be obtained by the audio data processing method provided by the present disclosure.

[0068] Optionally, the original singing audio database can also store audio information of the original singing audio data, such as singing singer, lyricist, composer, song duration, song release time, style, rhythm, and various types of audio information, which are not limited by the present disclosure.

[0069] S102, obtain a first lyric string of each original singing audio segment and a second lyric string of each of the at least one alternative cover audio data.

[0070] It should be noted that the computing device can also maintain a cover audio database, which can be used to store a plurality of alternative cover audio data, so that the original singing audio data in the original singing audio database can be matched based on the plurality of alternative cover audio data stored in the cover audio database.

[0071] Optionally, the cover audio database can also store audio information of the cover audio data, such as singing singer, lyricist, composer, song duration, song release time, style, rhythm, and various types of audio information, which are not limited by the present disclosure.

[0072] S103, for any original singing audio segment, based on the first lyric string, matching in each second lyric string to determine a cover audio segment matched with the original singing audio segment from the at least one alternative cover audio data.

[0073] It should be noted that for each original singing audio segment, the first lyric string of the original singing audio segment can be matched with the second lyric string of the plurality of alternative cover audio data to find a string segment matched with the first lyric string from the plurality of second lyric strings, so as to obtain the cover audio segment based on the matched string segment.

[0074] The present disclosure obtains the first lyric string of each original singing audio segment and the second lyric string of each of the at least one alternative cover audio data when the original singing audio data is segmented to obtain a plurality of original singing audio segments corresponding to the original singing audio data, so as to match in each second lyric string based on the first lyric string corresponding to each original singing audio segment to determine a cover audio segment matched with the original singing audio segment from the at least one alternative cover audio data, so as to realize the matching of the original singing audio segment and the cover audio segment. The matched original singing audio segment and cover audio segment can be used as training data for constructing feature extraction rules subsequently, so as to realize the expansion of the training data to obtain more training data to train the feature extraction rules, so that the feature extraction rules obtained by training can extract more reliable features representing the song segment attributes, and improve the accuracy of the feature extraction process.

[0075] After introducing the basic principles of the present disclosure, the various non-limiting embodiments of the present disclosure will be specifically introduced below.

[0076] In some embodiments, for step S101, when the original singing audio data is segmented into multiple original singing audio segments corresponding to the original singing audio data, the following method can be used:

[0077] Based on the timestamp of each lyric in the original singing audio data, the original singing audio data is divided into multiple original singing audio segments with a preset duration.

[0078] The preset duration can be any duration, for example, the preset duration can be 15 seconds, or the preset duration can also be other values, and the present disclosure does not limit the specific value of the preset duration. The timestamp can represent the singing time corresponding to the first character in the lyric, or the timestamp can also be other types, which are not limited by the present disclosure.

[0079] It should be noted that when the original singing audio data is segmented, the last part may not have a duration of the preset duration. At this time, the last part with a duration less than the preset duration can be directly used as an original singing audio segment, and it is not necessary to ensure that the duration of each original singing audio segment meets the preset duration.

[0080] Optionally, after the segmentation of the original singing audio data is completed, the start and end timestamps of each original singing audio segment obtained by segmentation can also be recorded, for example, for any original singing audio segment, the timestamp of the first lyric in the original singing audio segment can be used as the start timestamp of the original singing audio segment, and the timestamp of the last lyric in the original singing audio segment can be used as the end timestamp of the original singing audio segment.

[0081] After the original singing audio data is segmented into multiple original singing audio segments through the above process, the first lyric string of each original singing audio segment can be obtained through step S102, and the second lyric string of each alternative cover audio data in at least one alternative cover audio data can also be obtained through step S102.

[0082] In some embodiments, for step S102, when the first lyric string of each original singing audio segment is obtained, the following method can be used:

[0083] For any original singing audio segment, the lyric text of the original singing audio segment is obtained; the punctuation in the lyric text of the original singing audio segment is removed, and the characters in the lyric text of the original singing audio segment are converted into characters in a specified format, to obtain the first lyric string of the original singing audio segment.

[0084] wherein, for English text in the lyric text, the character in the set format can be a lowercase letter character, and for Chinese characters in the lyric text, the character in the set format can be a simplified Chinese character.

[0085] Taking the lyric text as English text as an example, for example, the lyric text can be “We make it happen, cause we believe it. Look who we are, we are the dreamers”, the punctuation marks in the lyric text can be removed, and the capital letters in the lyric text can be converted into lowercase letter characters, to obtain a first lyric string as shown in “wemakeithappencausewebelieveitlookwhowe arewearethedreamers”.

[0086] It should be noted that the above embodiment is described by taking the acquisition process of the first lyric string of one original audio segment as an example, and the acquisition process of the first lyric string of other original audio segments is the same as the above, which will not be described herein.

[0087] In some embodiments, for step S102, when the second lyric string of each of the at least one candidate cover audio data is acquired, the following method can be used:

[0088] For any candidate cover audio data, the lyric text of the candidate cover audio data is acquired; the punctuation marks in the lyric text of the candidate cover audio data are removed, and the characters in the lyric text of the candidate cover audio data are converted into characters in a set format, to obtain the second lyric string of the candidate cover audio data.

[0089] wherein, for English text in the lyric text, the character in the set format can be a lowercase letter character, and for Chinese characters in the lyric text, the character in the set format can be a simplified Chinese character.

[0090] Taking the lyric text as English text as an example, when the second string of any candidate cover audio data is acquired, all punctuation marks in the lyric text of the candidate cover audio data can be removed, and all letters in the lyric text can be changed into lowercase letters.

[0091] Optionally, after the second lyric string of the candidate cover audio data is acquired, the time stamp corresponding to each character in the second lyric string can be set, and for any character, the time stamp of the sentence to which the character belongs can be set as the time stamp of the character.

[0092] It should be noted that the above embodiment is described by taking the acquisition process of a second lyric string of an alternative cover audio data as an example, and the acquisition process of a second lyric string of other alternative cover audio data is the same, which will not be repeated here.

[0093] After the first lyric string and the second lyric string are acquired through the above process, for any original singing audio segment, the first lyric string of the original singing audio segment can be matched in each second lyric string based on the first lyric string to determine a cover audio segment matched with the original singing audio segment from at least one alternative cover audio data through step S103.

[0094] In some embodiments, for step S103, when the first lyric string is matched in each second lyric string to determine a cover audio segment matched with the original singing audio segment from at least one alternative cover audio data based on the first lyric string, the step can include the following steps:

[0095] S1031, match the first lyric string in each second lyric string to determine a target cover lyric string matched with the first lyric string from at least one second lyric string.

[0096] In a possible implementation, the step S1031 can include the following steps:

[0097] S1031-1, match the first lyric string in each second lyric string based on part of characters in the first lyric string to determine at least one alternative cover lyric string from at least one second lyric string.

[0098] Wherein, the part of characters can be a plurality of characters located at the center of the first lyric string and having a set length. For example, the part of characters can be 10 characters as the center of the string.

[0099] Optionally, for any second lyric string, the second lyric string can be traversed to determine the similarity between the plurality of string segments corresponding to the second lyric string and the part of characters in the first lyric string; and the string segment having a similarity satisfying a first preset condition from the plurality of string segments is determined as an alternative cover lyric string.

[0100] That is, the second lyric string can be traversed based on the part of characters in the first lyric string, and when any second lyric string is traversed, each character in the second string traversed at present can be traversed to compare the part of characters in the first lyric string with the plurality of string segments corresponding to the second lyric string to find a string segment having a higher similarity with the part of characters in the first lyric string from the second lyric string.

[0101] Optionally, the similarity between the plurality of string segments corresponding to the second lyric string and the part of characters in the first lyric string can be calculated by calculating the edit distance, and the smaller the calculated edit distance is, the higher the similarity between the two strings is, and vice versa.

[0102] It should be noted that after the edit distance between the part of characters in the first lyric string and the plurality of string segments corresponding to the plurality of second lyric strings is calculated, the string segment with an edit distance greater than a set threshold value can be taken as a candidate cover song lyric string based on the calculated edit distance, or the plurality of string segments can be sorted in ascending order of edit distance based on the calculated edit distance, and the string segment before the preset position in the sorting result can be taken as a candidate cover song lyric string.

[0103] Referring to Table 1 below, taking the part of characters in the first lyric string as an example, the plurality of candidate cover song lyric strings corresponding to the string are shown in Table 1:

[0104] Table 1

[0105]

[0106] Optionally, for the candidate cover song lyric strings corresponding to the edit distance values in different value intervals, the candidate cover song lyric strings corresponding to the edit distances in different value intervals can be recorded in different tables, for example, the candidate cover song lyric strings with an edit distance less than 2 can be recorded in a candidate list 1, and the candidate cover song lyric strings with an edit distance greater than or equal to 2 and less than 4 can be recorded in a candidate list 2, so that different processing methods can be used for the candidate cover song lyric strings recorded in different candidate lists in the subsequent processing.

[0107] It should be noted that the storage of the candidate cover song lyrics string can be completed in the traversal process. Taking the candidate cover song lyrics string as a string segment with an edit distance of less than 4 from the partial character in the first song lyrics string as an example, in the process of traversing the second song lyrics string corresponding to the cover audio data, if a string segment with an edit distance of less than 2 is traversed, the string segment can be recorded as a candidate cover song lyrics string in the candidate list 1, and then the next string segment is continuously traversed; if a string segment with an edit distance of greater than or equal to 2 and less than 4 is traversed, the string segment can be recorded as a candidate cover song lyrics string in the candidate list 2, and then the next string segment is continuously traversed; if a string segment with an edit distance of greater than or equal to 4 is traversed, it is considered that the string segment does not meet the similarity requirement, and the string segment does not need to be recorded, but the next string segment is directly traversed. In the case where the traversal of the second song lyrics string corresponding to one candidate cover audio data is completed, the second song lyrics string corresponding to the next candidate cover audio data is continuously traversed, and the traversal of the second song lyrics string corresponding to the next candidate cover audio data is continuously performed until the traversal of the second song lyrics string corresponding to the multiple candidate cover audio data is completed. At this time, the acquisition of the candidate cover song lyrics string is completed.

[0108] It should be noted that the calculation of the similarity between two strings by using the edit distance is only an example, and in more possible implementation manners, other manners can be used to calculate the similarity between two strings, for example, the Difflib fuzzy matching manner can be used to calculate the similarity between two strings, and the disclosure does not limit the specific manner.

[0109] S1031-2, according to the string length of the first song lyrics string, the at least one candidate cover song lyrics string is expanded to obtain at least one to-be-matched song lyrics string.

[0110] In a possible implementation manner, for any candidate cover song lyrics string, a song lyrics string with the same length as the first song lyrics string and centered on the candidate cover song lyrics string can be obtained in the second song lyrics string corresponding to the candidate cover song lyrics string as the to-be-matched song lyrics string corresponding to the candidate cover song lyrics string.

[0111] That is, in the second song lyrics string corresponding to the candidate cover song lyrics string, the character expansion can be performed forward and backward respectively with the candidate cover song lyrics string as the center to obtain the to-be-matched song lyrics string with the same length as the first song lyrics string.

[0112] It should be noted that, in the process of string expansion based on the alternative cover song lyrics string, the length of the first song lyrics string may not be matched even if the string is expanded to the limit, at this time, the string is only expanded to the limit of the string. For example, for a first song lyrics string with a length of 30 characters, if the length of the alternative cover song lyrics string is 10 characters, and the alternative cover song lyrics string is located at the tail part of the second song lyrics string, only 7 characters away from the end of the second song lyrics string, then in the process of string expansion based on the alternative cover song lyrics string, only 10 characters need to be expanded forward and 7 characters need to be expanded backward based on the alternative cover song lyrics string. The length of the final expanded matching song lyrics string is 27 characters.

[0113] Alternatively, if the alternative cover song lyrics strings corresponding to the editing distances with different value intervals are recorded in different tables in step S1031-1, then only the alternative cover song lyrics strings recorded in the table corresponding to the smaller editing distance value can be used for string expansion.

[0114] Still taking the example that the alternative cover song lyrics strings with an editing distance less than 2 are recorded in the candidate list 1, and the alternative cover song lyrics strings with an editing distance greater than or equal to 2 and less than 4 are recorded in the candidate list 2, then only the alternative cover song lyrics strings in the candidate list 1 can be used for string expansion.

[0115] It should be noted that after the above process is used to obtain the matching song lyrics string, the similarity between each matching song lyrics string and the first song lyrics string can be determined.

[0116] In a possible implementation, for any matching song lyrics string, the similarity between the matching song lyrics string and the first song lyrics string can be determined based on the editing distance between the matching song lyrics string and the first song lyrics string.

[0117] It should be noted that, in order to avoid the increase of the editing distance caused by the difference in the length of the matching song lyrics string and the first song lyrics string, which affects the process of determining the similarity between the matching song lyrics string and the first song lyrics string, the difference in the length between the matching song lyrics string and the first song lyrics string needs to be subtracted, so as to ensure the accuracy of the determined similarity between the matching song lyrics string and the first song lyrics string.

[0118] Alternatively, the difference between the editing distance and the difference in the length of the string between the first song lyrics string and the matching song lyrics string can be determined as the similarity between the matching song lyrics string and the first song lyrics string. That is, the similarity between the matching song lyrics string and the first song lyrics string can be determined by the following formula (1):

[0119] (1)

[0120] wherein, D Ls represents the edit distance between the to-be-matched lyric string and the first lyric string, L cover represents the length of the first lyric string, L trans represents the length of the to-be-matched lyric string, D ls represents the corrected edit distance, i.e., the similarity between the to-be-matched lyric string and the first lyric string.

[0121] S1031-3, determining, from the at least one to-be-matched lyric string, a target cover lyric string that matches the first lyric string based on the similarity between each to-be-matched lyric string and the first lyric string.

[0122] In a possible implementation, a to-be-matched lyric string in the at least one to-be-matched lyric string that satisfies a second preset condition with the similarity to the first lyric string can be determined as the target cover lyric string.

[0123] For example, a to-be-matched lyric string with a similarity (i.e., the corrected edit distance) less than one third of the length of the first lyric string can be determined as the target cover lyric string.

[0124] The above process of matching the first lyric string and the target cover lyric string adopts a two-step matching manner, i.e., first, a coarse positioning of similar strings is implemented based on partial characters through center matching, and then a refined matching is implemented based on the result of the coarse positioning through segment matching and segmentation of the second lyric string, which relaxes the matching condition, so that the original audio segment can be matched with the adapted part of the lyric in the cover audio segment. In addition, the non-strict matching requirement on the time stamp also allows the original audio segment to be matched with the possible multiple choruses in the cover audio segment, further increasing the amount of training data. Experiments show that, through the above matching manner, for 2000 original audio data and 6000 cover audio data, a total of 150,000 cover data pairs can be generated, which is nearly 20 times larger than the number of original audio data and cover audio data, greatly expanding the amount of training data for training the feature extraction rule.

[0125] S1032, segmenting the candidate cover audio data corresponding to the target cover lyric string according to the time stamp corresponding to the start and end positions of the target cover lyric string, to obtain a cover audio segment that matches the original audio segment.

[0126] It should be noted that since each character in the second lyric string has recorded a timestamp, and the target cover song lyric string is part of the second lyric string, the timestamp of each character in the target cover song lyric string is also known, so the timestamp corresponding to the start and end positions of the target cover song lyric string can be obtained, so that the cover audio segment matching the original audio segment can be obtained according to the obtained timestamp.

[0127] It should be noted that after obtaining the cover audio segment corresponding to an original audio segment, the cover audio segment corresponding to the next original audio segment can be obtained through the steps similar to the above process. The above operation is repeated in the plurality of candidate cover audio data to obtain the cover audio segment corresponding to each original audio segment. Each original audio segment and its corresponding cover audio segment can form a cover data pair, which can be used as training data for training feature extraction rules.

[0128] Referring to Figure 2 , Figure 2 is a flowchart of an audio data processing method according to an exemplary embodiment of the present disclosure, as shown in Figure 2 After the original song segment is cut based on the original audio data and the cover song lyrics are obtained based on the cover audio data, the center matching positioning method can be used to roughly position the string similar to the character of the original audio segment, so that the cover audio segment matching the original audio segment can be obtained by cutting the cover song lyric string based on the rough positioning result. The cover data pair can be obtained as training data.

[0129] Through the scheme provided by the present disclosure, for a batch of original audio data and cover audio data, only the lyrics text containing the timestamp of each sentence of lyrics needs to be provided, and the cover data pair can be automatically matched and generated. The matching lyrics do not need to be strictly required to be consistent with the duration of the original song, and the DJ fast version or emotional slow version of the original audio data existing in the cover audio data can be better utilized, which is closer to the actual business needs.

[0130] After introducing the audio data processing method of the exemplary embodiment of the present disclosure, the structure of the audio data processing device of the exemplary embodiment of the present disclosure and the computing device for implementing the audio data processing method will be described.

[0131] Referring to Figure 3 , Figure 3 is a block diagram of an audio data processing device according to an exemplary embodiment of the present disclosure, which comprises:

[0132] The segmentation module 301 is configured to segment the original audio data to obtain a plurality of original audio segments corresponding to the original audio data.

[0133] The acquisition module 302 is configured to acquire a first lyric string of each original audio segment and a second lyric string of each of the at least one cover audio data.

[0134] The determination module 303 is configured to, for any original audio segment, match each second lyric string based on the first lyric string to determine a cover audio segment matching the original audio segment from the at least one cover audio data.

[0135] In an embodiment of the present disclosure, the determination module 303, when used to match each second lyric string based on the first lyric string to determine a cover audio segment matching the original audio segment from the at least one cover audio data, is configured to:

[0136] match each second lyric string based on the first lyric string to determine a target cover lyric string matching the first lyric string from the at least one second lyric string;

[0137] segment the cover audio data corresponding to the target cover lyric string into a cover audio segment matching the original audio segment according to the time stamp corresponding to the start and end positions of the target cover lyric string.

[0138] In an embodiment of the present disclosure, the determination module 303, when used to match each second lyric string based on the first lyric string to determine a target cover lyric string matching the first lyric string from the at least one second lyric string, is configured to:

[0139] match each second lyric string based on part of the characters in the first lyric string to determine at least one cover lyric string from the at least one second lyric string;

[0140] extend the at least one cover lyric string according to the string length of the first lyric string to obtain at least one to-be-matched lyric string;

[0141] determine a target cover lyric string matching the first lyric string from the at least one to-be-matched lyric string based on the similarity between each to-be-matched lyric string and the first lyric string.

[0142] In an embodiment of the present disclosure, the determining module 303, when determining at least one candidate cover song lyric string from the at least one second song lyric string based on the partial characters in the first song lyric string and matching in each second song lyric string, is configured to:

[0143] For any second song lyric string, the second song lyric string is traversed to determine the similarity of the plurality of character segments corresponding to the second song lyric string and the partial characters in the first song lyric string.

[0144] The character segment in the plurality of character segments that satisfies the first preset condition is determined as the candidate cover song lyric string.

[0145] In an embodiment of the present disclosure, the determining module 303, when expanding the at least one candidate cover song lyric string to obtain at least one to-be-matched song lyric string according to the character length of the first song lyric string, is configured to:

[0146] For any candidate cover song lyric string, a song lyric string centered on the candidate cover song lyric string and having the same length as the first song lyric string is obtained in the second song lyric string corresponding to the candidate cover song lyric string as the to-be-matched song lyric string corresponding to the candidate cover song lyric string.

[0147] In an embodiment of the present disclosure, the determining module 303 is further configured to determine the similarity of each to-be-matched song lyric string and the first song lyric string.

[0148] In an embodiment of the present disclosure, the determining module 303, when determining the similarity of each to-be-matched song lyric string and the first song lyric string, is configured to:

[0149] For any to-be-matched song lyric string, the similarity of the to-be-matched song lyric string and the first song lyric string is determined based on the edit distance between the to-be-matched song lyric string and the first song lyric string.

[0150] In an embodiment of the present disclosure, the determining module 303, when determining the similarity of the to-be-matched song lyric string and the first song lyric string based on the edit distance between the to-be-matched song lyric string and the first song lyric string, is configured to:

[0151] The difference between the edit distance and the difference in character length between the first song lyric string and the to-be-matched song lyric string is determined as the similarity of the to-be-matched song lyric string and the first song lyric string.

[0152] In an embodiment of the present disclosure, the determining module 303, when determining a target cover song lyric string that matches the first song lyric string from the at least one to-be-matched song lyric string based on a similarity of each to-be-matched song lyric string to the first song lyric string, is configured to:

[0153] determine a to-be-matched song lyric string that has a similarity to the first song lyric string satisfying a second preset condition as the target cover song lyric string.

[0154] In an embodiment of the present disclosure, the partial characters are a plurality of characters located at a center of the first song lyric string and having a set length.

[0155] In an embodiment of the present disclosure, the segmenting module 301, when segmenting the original song audio data to obtain a plurality of original song audio segments corresponding to the original song audio data, is configured to:

[0156] divide the original song audio data into a plurality of original song audio segments each having a preset time length based on a timestamp of each lyric in the original song audio data.

[0157] In an embodiment of the present disclosure, the obtaining module 302, when obtaining the first song lyric string of any original song audio segment, is configured to:

[0158] obtain a lyric text of the original song audio segment;

[0159] remove punctuation marks in the lyric text of the original song audio segment, and convert characters in the lyric text of the original song audio segment into characters in a set format, to obtain the first song lyric string of the original song audio segment.

[0160] In an embodiment of the present disclosure, the obtaining module 302, when obtaining the second song lyric string of any candidate cover song audio data, includes:

[0161] obtaining a lyric text of the candidate cover song audio data;

[0162] removing punctuation marks in the lyric text of the candidate cover song audio data, and converting characters in the lyric text of the candidate cover song audio data into characters in a set format, to obtain the second song lyric string of the candidate cover song audio data.

[0163] It should be noted that although several modules of the data processing apparatus are mentioned in the foregoing detailed description, such division is merely exemplary and not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules described above can be embodied in one module. Conversely, the features and functions of one module described above can be further divided into a plurality of modules.

[0164] The embodiments of the present disclosure further provide a computer readable storage medium. Figure 4 FIG. 1 is a schematic diagram of a computer readable storage medium according to an exemplary embodiment of the present disclosure, as shown, the storage medium stores a computer program 101, the computer program 101 can execute the audio data processing method provided by any of the embodiments of the present disclosure when executed by a processor. Figure 4 The embodiments of the present disclosure further provide a computer readable storage medium.

[0165] The embodiments of the present disclosure further provide a computing device, which can include a memory for storing computer instructions executable on a processor, and the processor for implementing the audio data processing method provided by any of the embodiments of the present disclosure when executing the computer instructions. Figure 5 , Figure 5 FIG. 5 is a structural schematic diagram of a computing device according to an exemplary embodiment of the present disclosure, the computing device 500 can include but not limited to: a processor 510, a memory 520, a bus 530 connecting different system components including the memory 520 and the processor 510.

[0166] The memory 520 stores computer instructions executable by the processor 510, so that the processor 510 can execute the audio data processing method provided by any of the embodiments of the present disclosure. The memory 520 can include a random access memory unit RAM 521, a cache memory unit 522 and / or a read only memory unit ROM 523. The memory 520 can further include a program tool 525 having a set of program modules 524, the program modules 524 include but not limited to: an operating system, one or more application programs, other program modules and program data, one or more combinations of which can include the implementation of a network environment.

[0167] The bus 530 can include a data bus, an address bus and a control bus, etc. The computing device 500 can further communicate with an external device 550, for example, a keyboard, a Bluetooth device, etc. through an I / O interface 540. The computing device 500 can further communicate with one or more networks, for example, a local area network, a wide area network, a public network, etc. through a network adapter 560. As shown in FIG. 5, the network adapter 560 can also communicate with other modules of the computing device 500 through the bus 530. Figure 5

[0168] The embodiments of the present disclosure further provide a computer program product, the computer program product includes a computer program, the program is executed by the processor 510 of the computing device 500, and the audio data processing method provided by any of the embodiments of the present disclosure can be implemented.

[0169] ​Furthermore, although the operations of the methods of the present disclosure are described in a particular, sequential order, this should not be understood as a requirement or implied that the operations be performed in anything less than the order described, and / or that all be performed, to achieve desirable results. Additionally or alternatively, certain steps can be omitted, combined, performed simultaneously, and / or performed separately from other steps disclosed.

[0170] While the spirit and principles of the present disclosure have been described with reference to several specific implementations, it is to be understood that the present disclosure is not limited to the specific implementations disclosed and that the division into aspects is not meant to imply that features from one aspect cannot be combined with features from another aspect to benefit, but is merely for ease of presentation. The present disclosure is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the claims appended hereto.

Claims

1. An audio data processing method, characterized by, The method comprises: segmenting the original audio data to obtain a plurality of original audio segments corresponding to the original audio data; obtaining a first lyric string of each original audio segment and a second lyric string of each of the at least one alternative cover audio data; for any original audio segment, matching each second lyric string based on the first lyric string to determine a cover audio segment matching the original audio segment from the at least one alternative cover audio data; wherein the matching each second lyric string based on the first lyric string comprises: matching each second lyric string based on part of the characters in the first lyric string to determine at least one alternative cover lyric string from the at least one second lyric string; extending the at least one alternative cover lyric string according to the string length of the first lyric string to obtain at least one matching lyric string; determining a target cover lyric string matching the first lyric string from the at least one matching lyric string based on the similarity between each matching lyric string and the first lyric string.

2. The method of claim 1, wherein, The determination of the cover audio segment matching the original audio segment from the at least one alternative cover audio data comprises: segmenting the alternative cover audio data corresponding to the target cover lyric string according to the time stamp corresponding to the start and end positions of the target cover lyric string to obtain the cover audio segment matching the original audio segment.

3. The method of claim 1, wherein, The matching each second lyric string based on part of the characters in the first lyric string to determine at least one alternative cover lyric string from the at least one second lyric string comprises: for any second lyric string, traversing the second lyric string to determine the similarity between the plurality of string segments corresponding to the second lyric string and part of the characters in the first lyric string; determining the string segment in the plurality of string segments satisfying the first preset condition as the alternative cover lyric string.

4. The method of claim 1, wherein, The extension of the at least one alternative cover lyric string according to the string length of the first lyric string to obtain at least one matching lyric string comprises: for any alternative cover lyric string, obtaining a lyric string centered on the alternative cover lyric string and having the same length as the first lyric string in the second lyric string corresponding to the alternative cover lyric string as the matching lyric string corresponding to the alternative cover lyric string.

5. The method of claim 1, wherein, Before the determination of the target cover lyric string matching the first lyric string from the at least one matching lyric string based on the similarity between each matching lyric string and the first lyric string, the method further comprises: determining the similarity between each matching lyric string and the first lyric string.

6. The method of claim 5, wherein, The determining of the similarity between each to-be-matched lyric string and the first lyric string comprises: For any to-be-matched lyric string, the similarity between the to-be-matched lyric string and the first lyric string is determined based on an edit distance between the to-be-matched lyric string and the first lyric string.

7. The method of claim 6, wherein, The determining of the similarity between the to-be-matched lyric string and the first lyric string based on the edit distance between the to-be-matched lyric string and the first lyric string comprises: The difference between the edit distance and the string length difference value is determined as the similarity between the to-be-matched lyric string and the first lyric string.

8. The method of claim 1, wherein, The determining of the target cover song lyric string matched with the first lyric string from the at least one to-be-matched lyric string based on the similarity between each to-be-matched lyric string and the first lyric string comprises: The to-be-matched lyric string in the at least one to-be-matched lyric string that satisfies a second preset condition with the similarity to the first lyric string is determined as the target cover song lyric string.

9. The method of claim 1, wherein, The partial characters are a plurality of characters located at the center of the first lyric string and having a set length.

10. The method of claim 1, wherein, The segmenting of the original song audio data to obtain a plurality of original song audio segments corresponding to the original song audio data comprises: The original song audio data is divided into a plurality of original song audio segments with a preset segment length based on the time stamp of each lyric in the original song audio data.

11. The method of claim 1, wherein, For any original song audio segment, the obtaining process of the first lyric string of the original song audio segment comprises: Obtaining the lyric text of the original song audio segment; Removing the punctuation in the lyric text of the original song audio segment, and converting the characters in the lyric text of the original song audio segment into characters in a set format to obtain the first lyric string of the original song audio segment.

12. The method of claim 1, wherein, For any alternative cover song audio data, the obtaining process of the second lyric string of the alternative cover song audio data comprises: Obtaining the lyric text of the alternative cover song audio data; Removing the punctuation in the lyric text of the alternative cover song audio data, and converting the characters in the lyric text of the alternative cover song audio data into characters in a set format to obtain the second lyric string of the alternative cover song audio data.

13. An audio data processing apparatus, characterized by comprising: The device comprises: A segmenting module configured to segment original song audio data to obtain a plurality of original song audio segments corresponding to the original song audio data; An obtaining module configured to obtain a first lyric string of each original song audio segment and a second lyric string of each alternative cover song audio data in at least one alternative cover song audio data; A determining module configured to, for any original song audio segment, match in each second lyric string based on the first lyric string to determine a cover song audio segment matched with the original song audio segment from the at least one alternative cover song audio data; and The determining module is configured to: match each second lyric string based on the first lyric string to determine at least one candidate cover lyric string from the at least one second lyric string; extend the at least one candidate cover lyric string according to a string length of the first lyric string to obtain at least one to-be-matched lyric string; determine a target cover lyric string matching the first lyric string from the at least one to-be-matched lyric string based on a similarity between each to-be-matched lyric string and the first lyric string.

14. The apparatus of claim 13, wherein, The determining module is configured to: segment the candidate cover audio data corresponding to the target cover lyric string according to time stamps corresponding to start and end positions of the target cover lyric string to obtain a cover audio segment matching the original audio segment.

15. The apparatus of claim 13, wherein, The determining module is configured to: for any second lyric string, traverse the second lyric string to determine similarities between a plurality of string segments of the second lyric string and the partial characters in the first lyric string; determine, as the candidate cover lyric string, a string segment from the plurality of string segments that satisfies a first preset condition.

16. The apparatus of claim 13, wherein, The determining module is configured to: for any candidate cover lyric string, obtain, in the second lyric string corresponding to the candidate cover lyric string, a lyric string centered on the candidate cover lyric string and having a length consistent with that of the first lyric string as a to-be-matched lyric string corresponding to the candidate cover lyric string.

17. The apparatus of claim 13, wherein, The determining module is further configured to determine the similarity between each to-be-matched lyric string and the first lyric string.

18. The apparatus of claim 17, wherein, The determining module is configured to: for any to-be-matched lyric string, determine the similarity between the to-be-matched lyric string and the first lyric string based on an edit distance between the to-be-matched lyric string and the first lyric string.

19. The apparatus of claim 18, wherein, The determining module is configured to: for any to-be-matched lyric string, determine the similarity between the to-be-matched lyric string and the first lyric string based on an edit distance between the to-be-matched lyric string and the first lyric string. The difference between the edit distance and the difference between the string lengths of the first lyric string and the to-be-matched lyric string is determined as a similarity between the to-be-matched lyric string and the first lyric string.

20. The apparatus of claim 13, wherein, The determining module, when determining a target cover song lyric string that matches the first lyric string from the at least one to-be-matched lyric string based on the similarity between each to-be-matched lyric string and the first lyric string, is configured to: determine a to-be-matched lyric string that satisfies a second preset condition in terms of the similarity to the first lyric string as the target cover song lyric string.

21. The apparatus of claim 13, wherein, The partial characters are a plurality of characters located at the center of the first lyric string and having a set length.

22. The apparatus of claim 13, wherein, The splitting module, when splitting the original song audio data into a plurality of original song audio segments, is configured to: divide the original song audio data into a plurality of original song audio segments each having a preset time length based on the time stamp of each lyric in the original song audio data.

23. The apparatus of claim 13, wherein, The obtaining module, when obtaining the first lyric string of any original song audio segment, is configured to: obtain the lyric text of the original song audio segment; remove punctuation in the lyric text of the original song audio segment, and convert characters in the lyric text of the original song audio segment into characters in a set format to obtain the first lyric string of the original song audio segment.

24. The apparatus of claim 13, wherein, The obtaining module, when obtaining the second lyric string of any candidate cover song audio data, includes the following steps: obtain the lyric text of the candidate cover song audio data; remove punctuation in the lyric text of the candidate cover song audio data, and convert characters in the lyric text of the candidate cover song audio data into characters in a set format to obtain the second lyric string of the candidate cover song audio data.

25. A computing device, comprising: The computing device includes a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements operations performed by the audio data processing method according to any one of claims 1 to 12 when executing the program.

26. A computer readable storage medium, characterized in that, The computer readable storage medium stores a program, and the program is executed by the processor to perform operations performed by the audio data processing method according to any one of claims 1 to 12.

Images