Cover song recognition method, cover song recognition device and storage medium

Abstract

This application discloses a cover song recognition method, a cover song recognition device, and a storage medium, for use in the field of audio recognition technology. The method includes acquiring an audio file to be recognized; determining the audio features of multiple time-shifted audio segments in the audio file to be recognized; based on the feature similarity between the audio features of each audio segment to be recognized and the audio features of each original audio segment, determining candidate original audio segments matching each audio segment to be recognized from the multiple original audio segments; among the candidate original audio segments matched by multiple consecutive audio segments to be recognized in terms of segment offset order, determining a target original audio segment belonging to the same original audio and having a consecutive segment offset order; and based on the number of segments of the target original audio segment, determining the cover song corresponding to the audio file to be recognized from the original audio; effectively improving the accuracy of cover song recognition.

Description

Technical Field

[0001] This application relates to the field of audio recognition technology, and in particular to a cover song recognition method, a cover song recognition device, and a storage medium. Background Technology

[0002] Cover song recognition technology is a music retrieval technology based on audio data. It can identify the original song being covered by audio data and is commonly used in music applications such as song recognition and humming recognition. Existing cover song recognition technology involves acquiring a song retrieval database and the audio data to be detected. The song retrieval database contains multiple audio features of the original song. Audio features are extracted from the audio data to be detected, and these features are compared with the audio features in the song retrieval database to determine the original song being covered by the audio data within that database based on feature similarity.

[0003] However, the song retrieval database may contain multiple audio segments with similar audio features, and these similar audio segments may belong to different original songs. When using the song retrieval database to compare feature similarity, these multiple similar audio segments can easily cause misleading results, leading to errors in the cover song recognition. Summary of the Invention

[0004] This application provides a cover song recognition method, a cover song recognition device, and a storage medium, which can improve the accuracy of cover song recognition.

[0005] This application provides a cover song recognition method, which is applied to a retrieval database. The retrieval database includes audio features of multiple time-shifted original audio slices, including:

[0006] Obtain the audio to be recognized;

[0007] Determine the audio features of multiple time-shifted audio slices in the audio to be identified;

[0008] Based on the feature similarity between the audio features of each audio slice to be identified and the audio features of each original audio slice, candidate original audio slices matching each audio slice to be identified are determined from the multiple original audio slices.

[0009] Among the candidate original audio slices that are matched by multiple consecutive audio slices to be identified in the slice offset order, the target original audio slice that belongs to the same original audio and has a consecutive slice offset order is determined.

[0010] Based on the number of slices of the target original audio slice, the cover audio corresponding to the audio to be identified is determined from the original audio.

[0011] Furthermore, determining the audio features of multiple time-shifted audio slices in the audio to be identified includes:

[0012] A pre-trained cover song recognition model is obtained, which is trained based on a preset cover song audio and a preset original song audio, wherein the preset cover song audio is an audio obtained by covering the preset original song audio;

[0013] Each audio slice to be identified in the audio to be identified is input into the cover song recognition model to obtain the audio features of each audio slice to be identified in the audio to be identified output by the cover song recognition model.

[0014] Furthermore, the step of determining candidate original audio slices matching each audio slice to be identified from multiple original audio slices based on the feature similarity between the audio features of each audio slice to be identified and the audio features of each original audio slice includes:

[0015] Determine the feature similarity between the audio features of any audio slice in the audio slice to be identified and the audio features of each original audio slice;

[0016] For each audio slice to be identified, the original audio slice that meets the preset similarity threshold is used as the candidate original audio slice to be matched with the audio slice to be identified.

[0017] Furthermore, determining the target original audio slices that belong to the same original audio and whose slice offset order is continuous includes:

[0018] Obtain the audio slice sequence corresponding to the audio slice to be identified, wherein each audio slice to be identified in the audio slice sequence has a corresponding slice offset order;

[0019] Based on the slice offset order of each candidate original audio slice in the same original audio, each candidate original audio slice in the same original audio is arranged on the audio slice sequence;

[0020] The target original audio slice is determined based on the candidate original audio slices arranged consecutively on the audio slice sequence.

[0021] Furthermore, determining the target original vocal audio slice based on the consecutively arranged candidate original vocal audio slices in the audio slice sequence includes:

[0022] Determine whether the candidate original audio slices in the audio slice sequence are consecutively arranged audio slices;

[0023] If the candidate original audio slices are consecutive audio slices, then determine whether the consecutive candidate original audio slices are consecutive audio slices in the corresponding original audio slice offset order.

[0024] If the audio slices are consecutive, then the consecutively arranged candidate original audio slices will be used as the target original audio slice.

[0025] Furthermore, determining the cover audio corresponding to the audio to be identified from the original audio based on the number of slices of the target original audio slice includes:

[0026] The target original audio slice with the largest number of slices is determined from the number of slices of the target original audio slice;

[0027] The original audio corresponding to the target original audio slice with the largest number of slices is taken as the cover audio corresponding to the audio to be identified.

[0028] Furthermore, determining the cover audio corresponding to the audio to be identified from the original audio based on the number of slices of the target original audio slice includes:

[0029] Determine the similarity score of each candidate original audio slice matched by the audio slice to be identified, wherein the similarity score indicates the magnitude of feature similarity between the candidate original audio slice and the audio slice to be identified corresponding to the candidate original audio slice;

[0030] Determine the continuous density score of each candidate original audio slice matched by the audio slice to be identified, wherein the continuous density score is proportional to the number of slices of the target original audio slice corresponding to the candidate original audio slice;

[0031] The similarity score is added to the continuous density score to obtain the cover score of each candidate original audio slice matched by the audio slice to be identified;

[0032] Determine the maximum score among the cover song scores, and use the original audio corresponding to the candidate original audio slice with the maximum score as the cover audio corresponding to the audio to be identified.

[0033] Furthermore, the method also includes;

[0034] Based on a preset slice offset rule, the audio to be identified is sliced ​​to obtain multiple audio slices to be identified.

[0035] Based on the preset slice offset rules, the original audio is sliced ​​to obtain multiple original audio slices.

[0036] This application also provides a cover song recognition device, including:

[0037] Central processing unit, memory, input / output interface, wired or wireless network interface, power supply;

[0038] The memory is either a short-term storage memory or a persistent storage memory;

[0039] The central processing unit is configured to communicate with the memory and execute instructions in the memory on a control plane functional entity to perform the methods described above.

[0040] This application also provides a computer-readable storage medium including instructions that, when executed on a computer, cause the computer to perform the method described above.

[0041] As can be seen from the above technical solutions, the embodiments of this application have the following advantages:

[0042] In this embodiment, the following steps are taken: First, the audio to be identified is acquired. Then, the audio features of multiple time-shifted audio segments within the audio to be identified are determined. Based on the feature similarity between the audio features of each audio segment to be identified and the audio features of each original audio segment, candidate original audio segments matching each audio segment to be identified are determined from the multiple original audio segments. From the candidate original audio segments matched by multiple consecutive audio segments to be identified in terms of segment offset order, target original audio segments belonging to the same original audio and having consecutive segment offset orders are determined. Based on the number of target original audio segments, the cover song corresponding to the audio to be identified is determined from the original audio. By determining the number of target audio segments belonging to the same original audio and having consecutive segment offset orders, the probability of the audio to be identified corresponding to each original audio is determined, thereby improving the accuracy of cover song recognition. Attached Figure Description

[0043] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings.

[0044] Figure 1 This is a communication network architecture diagram for cover song recognition disclosed in an embodiment of this application;

[0045] Figure 2 This is a flowchart of a cover song recognition method disclosed in an embodiment of this application;

[0046] Figure 3 This is another flowchart for cover song recognition disclosed in an embodiment of this application;

[0047] Figure 4This is a schematic diagram of a cover song recognition model disclosed in an embodiment of this application;

[0048] Figure 5 This is a schematic diagram of a cover song recognition device disclosed in an embodiment of this application;

[0049] Figure 6 This is a schematic diagram of another cover song recognition device disclosed in an embodiment of this application. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0051] In the description of the embodiments of this application, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing the embodiments of this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the embodiments of this application.

[0052] In the description of the embodiments of this application, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in the embodiments of this application based on the specific circumstances.

[0053] Existing cover song recognition such as Figure 1As shown, the cover song recognition device 101 can acquire the audio to be recognized from the audio receiver 102 and identify the original song covered by the audio. The audio to be recognized can be manually entered into the audio receiver 102 or the audio receiver 102 can acquire the audio via a network connection; the specific method is not limited here. The audio receiver 102 can be an electronic device with audio collection capabilities, such as a recorder or a mobile phone; the specific method is not limited here. The cover song recognition device 101 can be connected to one or more audio receivers 102 to identify whether the audio to be recognized in one or more audio receivers 102 is a cover song and the original song covered. The cover song recognition device 101 includes a song retrieval library containing the audio features of multiple audio segments of the original song. It can compare the audio features of the audio segments of the audio to be recognized with the audio features of the audio segments in the song retrieval library to identify the original song covered by the audio. However, song retrieval databases may contain multiple audio segments with similar audio features, and these similar audio segments may belong to different original songs. When comparing feature similarity using the song retrieval database, these similar audio segments can easily mislead, leading to errors in cover song recognition results. Therefore, this application provides a cover song recognition method that can effectively improve the accuracy of cover song recognition, such as... Figure 2 As shown, it specifically includes steps 201 to 205.

[0054] 201. Obtain the audio to be recognized.

[0055] The cover song recognition method in this application includes a retrieval library. This retrieval library comprises audio features of multiple time-shifted original audio segments. Specifically, the original audio can be sliced ​​based on a certain time shift to obtain multiple original audio segments, and the retrieval library is constructed based on the audio features of these multiple original audio segments. The original audio refers to the audio corresponding to the original song. The original audio can be the entire original song or a portion of it; specificity is not limited here. Multiple original audio segments can be obtained from the music library of a music application or through online searches; specificity is not limited here.

[0056] The cover song recognition device can acquire the audio to be recognized, either by manually recording the audio or by obtaining it through a network connection; the specific method is not limited here. It is understood that in this embodiment, the main focus is on determining the cover song corresponding to the audio to be recognized from the multiple original audio recordings, i.e., identifying the original audio recording covered by the audio to be recognized. Therefore, preferably, the search library contains as many audio features as possible of the original audio segments from multiple original audio recordings.

[0057] 202. Determine the audio features of multiple time-shifted audio slices in the audio to be identified.

[0058] Once the audio to be identified is obtained, the audio features of multiple audio slices with different time offsets can be determined. Specifically, the audio to be identified is sliced ​​based on a certain time offset, resulting in multiple audio slices with different time offsets. This time offset can be 0.2 seconds or 0.3 seconds, and the specific offset is not limited here. The time offsets in each slice can be the same, or the time offsets in each slice can be slightly different, and the specific offset is not limited here.

[0059] After obtaining the audio slices to be identified, the audio features of each slice can be extracted. These audio features can be audio vectors or Mel-spectral features; the specific method is not limited here. The Liborosa extraction tool or the Timbral_models extraction tool can be used to extract the audio features of the audio slices; the specific method is not limited here.

[0060] 203. Determine the candidate original audio slices that match each audio slice to be identified from multiple original audio slices.

[0061] After determining the audio features of each audio segment to be identified in the audio to be identified, candidate original audio segments matching each audio segment to be identified can be determined from multiple original audio segments based on the feature similarity between the audio features of each audio segment to be identified and the audio features of each original audio segment. Specifically, the feature similarity of the audio features of each audio segment to be identified can be compared with the audio features of all original audio segments, and the original audio segments with a similarity reaching a preset threshold can be identified as candidate original audio segments matching that audio segment to be identified.

[0062] It is understandable that each audio slice to be identified can be matched with multiple candidate original audio slices. In general, the multiple candidate original audio slices matched by the same audio slice to be identified belong to different original audios. That is, each original audio to be identified generally only matches one original audio of the same original audio. The original audio to be identified can be matched with multiple original audios of the same original audio for feature similarity, and then the original audio with the highest feature similarity is selected as the candidate original audio of the original audio to be identified for matching in that original audio.

[0063] 204. Among the candidate original audio slices that are matched by multiple audio slices to be identified with consecutive slice offset order, determine the target original audio slice that belongs to the same original audio and has consecutive slice offset order.

[0064] It is understandable that each audio segment to be identified has a corresponding segment offset order in the audio to be identified. This segment offset order refers to the time sequence of the audio segments to be identified. Based on the time sequence of the segments, the segment offset order of each audio segment to be identified can be obtained, and thus, multiple audio segments to be identified with consecutive segment offset orders can be determined. For example, if the audio to be identified is segmented with a segment offset of 0.2 seconds and a segment duration of 2 seconds, the audio segments to be identified can be ordered according to the time sequence as the first audio segment to be identified, the second audio segment to be identified, and so on, up to the Nth audio segment to be identified. The audio segments to be identified that are consecutive in the order are multiple audio segments to be identified with consecutive segment offset orders.

[0065] From multiple candidate original audio slices that are matched separately in a continuous slice offset order, a target original audio slice belonging to the same original audio and having a continuous slice offset order can be determined. It is understood that the multiple candidate audio slices with a continuous slice offset order can be two or four consecutive candidate audio slices; the specifics are not limited here. From the candidate original audio slices that are matched separately in a continuous slice offset order, candidate original audio slices belonging to the same original audio can be determined. Then, it is determined whether the slice offset order of these candidate original audio slices belonging to the same original audio is continuous, thus obtaining the target original audio slice with a continuous slice offset order. Understandably, the continuous slice offset order of the candidate original audio slices means that there is no break in the slice offset order. For example, if the slice offset order of the original audio slices in the original audio is the first original audio slice, the second original audio slice, ..., the Nth original audio slice, if the first audio slice to be identified matches the second original audio slice, and the second audio slice to be identified matches the third original audio slice, then the second and third original audio slices are determined to be target original audio slices belonging to the same original audio and with a continuous slice offset order. If the third audio slice to be identified matches the fourth original audio slice, then the fourth original audio slice is also the target original audio slice. However, if the third audio slice to be identified matches the fifth original audio slice or does not match any original audio slice, then there is a break in the slice offset order.

[0066] It is understandable that different candidate original audio slices can be used to obtain corresponding target original audio slices with continuous slice offset order.

[0067] 205. Based on the number of slices in the target original audio slice, determine the cover audio corresponding to the audio to be identified from the original audio.

[0068] Once a target original audio segment belonging to the same original audio and with a continuous segment offset order has been identified, the cover audio corresponding to the audio to be identified can be determined from the original audio based on the number of segments in the target original audio segment. The number of segments in the target original audio segment can be used as the cover probability of the original audio corresponding to the target original audio segment (i.e., the probability that the audio to be identified will cover the original audio). Based on the magnitude of this cover probability, the cover audio corresponding to the audio to be identified can be determined.

[0069] Specifically, the target original audio slice with the most slices can be identified from the total number of slices in the target original audio slice. The original audio corresponding to the target original audio slice with the most slices is taken as the cover audio corresponding to the audio to be identified. That is, determine the number of candidate original audio slices that belong to the same original audio and have consecutive slice offsets in the slice offset order of the audio slice to be identified, find the candidate original audio slice with the most slices, and take the original audio corresponding to the candidate original audio slice as the cover audio corresponding to the audio to be identified.

[0070] In this embodiment, the following steps are taken: First, the audio to be identified is acquired. Then, the audio features of multiple time-shifted audio segments within the audio to be identified are determined. Based on the feature similarity between the audio features of each audio segment to be identified and the audio features of each original audio segment, candidate original audio segments matching each audio segment to be identified are determined from the multiple original audio segments. From the candidate original audio segments matched by multiple consecutive audio segments to be identified in terms of segment offset order, target original audio segments belonging to the same original audio and having consecutive segment offset orders are determined. Based on the number of target original audio segments, the cover song corresponding to the audio to be identified is determined from the original audio. By determining the number of target audio segments belonging to the same original audio and having consecutive segment offset orders, the probability of the audio to be identified corresponding to each original audio is determined, thereby improving the accuracy of cover song recognition.

[0071] Furthermore, the process of cover song recognition will be described in detail below, such as... Figure 3 As shown, it specifically includes steps 301 to 306.

[0072] 301. Obtain the audio to be recognized.

[0073] It is understandable that step 301 is similar to step 201 above, and will not be described in detail here.

[0074] 302. Based on the preset slice offset rules, slice the audio to be recognized to obtain multiple audio slices to be recognized.

[0075] In this embodiment, the audio to be identified and the original audio can be sliced ​​based on a preset slice offset rule, resulting in multiple audio slices to be identified and multiple original audio slices. To facilitate feature similarity comparison of the audio slices, the same slice rule is used when building the retrieval database. That is, the audio to be identified is sliced ​​based on the preset slice offset rule to obtain multiple audio slices to be identified; the original audio is sliced ​​based on the same preset slice offset rule to obtain multiple original audio slices. The preset slice rule can be that the duration of each audio slice is 2 seconds and the slice offset duration is 32 milliseconds. For example, if the duration of the audio to be identified is 15 seconds, then after slicing using the preset slice offset rule, 468 audio slices to be identified can be obtained, and each original audio is sliced ​​according to the same preset slice offset rule.

[0076] 303. Input the audio slice to be identified into the cover song recognition model to obtain the audio features of the audio slice to be identified.

[0077] After obtaining the audio slice to be identified, the audio slice can be input into the cover song recognition model to obtain the audio features of the audio slice to be identified.

[0078] Specifically, a pre-trained cover song recognition model can be obtained. This model is trained based on preset cover audio and preset original audio. The preset cover audio is the audio obtained by covering the preset original audio. When the audio data input to the cover song recognition model is a cover audio, the feature similarity between the audio features output by the cover song recognition model and the audio features of the original audio is greater than a preset value. When the audio input to the cover song recognition model is not a cover audio, the feature similarity between the audio features output by the cover song recognition model and the audio features of the original audio is less than a preset value.

[0079] The training process of the cover song recognition model is as follows: Figure 4As shown, an initial cover song recognition model (residual neural network), original audio (original song), cover audio (cover song), and non-cover audio (other songs) can be obtained. The original audio, cover audio, and non-cover audio are sliced ​​to obtain the original audio slice (anchor segment), the cover audio slice (positive segment), and the non-cover audio slice (negative segment), which form a triple. The audio features of the original audio segments, the cover audio segments, and the non-cover audio segments are input into the initial cover song recognition model, respectively. This yields the original audio features of the original audio segments, the cover audio features of the cover audio segments, and the non-cover audio features of the non-cover audio segments output by the initial cover song recognition model. These audio features can be audio vectors (embedding features). A triplet loss function is constructed based on the feature similarity between the original and cover audio features, and between the original and non-cover audio features: L = max(d(a,p) - d(a,n) + margin,0), where d(a,p) is the cosine distance between the anchor and the positive segment, d(a,n) is the cosine distance between the anchor and other negative segments, and margin is an adjustable coefficient. The initial cover song recognition model is considered complete when the loss function converges. Understandably, the purpose of training is to make the original audio features output by the cover song recognition model as similar as possible to the cover song audio features, and as dissimilar as possible to the non-cover song audio features.

[0080] Each audio slice to be identified in the audio to be identified is input into the cover song recognition model to obtain the audio features of each audio slice in the audio to be identified, as output by the cover song recognition model. Similarly, each original audio slice in each original audio is input into the cover song recognition model to obtain the audio features of each original audio slice in each original audio, as output by the cover song recognition model. It is understandable that a retrieval library can be built using the audio features of all original audio slices, and the Mel-frequency spectral characteristics (MFCC features) of the original audio slices can be extracted and input into the cover song recognition model. The audio features of these original audio slices can then be used to extract the audio features of the original audio slices, and the set of these audio features constitutes the retrieval library. It is also understandable that the audio features of the audio slice to be identified output by the cover song recognition model are quite similar to the audio features of the original audio slice that the audio slice to be identified is covering.

[0081] 304. Select original audio slices that meet the preset similarity threshold as candidate original audio slices for matching the audio slice to be identified.

[0082] After obtaining the audio features of the audio slice to be identified, the feature similarity between the audio features of any audio slice in the audio segment to be identified and the audio features of each original audio slice can be determined. This involves comparing the audio features of any audio slice in the audio segment to be identified with the audio features of all original audio slices in the search database. For each audio slice to be identified, original audio slices that meet a preset similarity threshold are selected as candidate original audio slices for matching. The audio features can be audio vectors, and the feature similarity can be cosine distance. The preset similarity threshold can also be a cosine distance threshold. TopK results, i.e., matrix results, can be retained for each audio slice to be identified.

[0083] x11...x1K,x21...x2K,x31...x3K,,,xn1...xnK;

[0084] Where x represents a candidate original audio slice, n represents the number of audio slices to be identified, and K represents the number of the first K results to be retained. This allows us to obtain the candidate original audio slices that match each audio slice to be identified.

[0085] 305. Determine the candidate original audio slices that are arranged continuously and whose slice offset order is continuous, and obtain the target original audio slice.

[0086] After obtaining the candidate original audio slices matching each audio slice to be identified, a series of consecutive candidate original audio slices with continuous slice offset order can be determined, thus obtaining the target original audio slice. Specifically, an audio slice sequence corresponding to the audio slice to be identified can be obtained, where each audio segment to be identified has a corresponding slice offset order; that is, the audio slice sequence is arranged according to the order of the slice offset order corresponding to the audio slices to be identified. Based on the slice offset order of each candidate original audio slice in the same original audio, each candidate original audio slice in the same original audio is arranged on the audio slice sequence; based on the consecutively arranged candidate original audio slices on the audio slice sequence, the target original audio slice is determined.

[0087] The target original audio slice can be determined by considering consecutive candidate original audio slices in an audio slice sequence. This can be achieved by: determining whether the candidate original audio slices are consecutively arranged; if so, determining whether they are consecutive in the corresponding original audio slice offset order; and if so, using the consecutively arranged candidate original audio slices as the target original audio slice. It's understandable that because songs are continuous and have sequential relationships (i.e., audio slices are related), in the TopK results, for a given audio slice x to be identified... iIf the original audio slice y in the original audio A is matched j And the audio slice to be identified x i The next audio slice to be identified, x, corresponds to the slice offset order of the audio to be identified. i+1 Also, the original audio slice y in the original audio A. j The next original audio slice y j+1 Matching allows us to determine the corresponding original audio slice y. j And the next original audio slice y j+1 The target original audio slices are arranged continuously with a continuous slice offset order.

[0088] 306. Based on the similarity score and continuous density score of the candidate original audio slices, determine the cover audio corresponding to the audio to be identified.

[0089] In this embodiment, the cover audio corresponding to the audio to be identified can be determined based on the similarity score and continuous density score of the candidate original audio slices. Specifically, the similarity score of each candidate original audio slice matched with the audio slice to be identified can be determined. The similarity score indicates the degree of feature similarity between the candidate original audio slice and the audio slice to be identified corresponding to the candidate original audio slice. This similarity score can be represented by S1, and the similarity scores of each candidate original audio slice are somewhat different. Next, the continuous density score of each candidate original audio slice matched with the audio slice to be identified is determined. The continuous density score is proportional to the number of slices of the target original audio slice corresponding to the candidate original audio slice, that is, the continuous density score is proportional to the number of consecutive slices in the slice offset order of the candidate original audio slice. This continuous density score can be represented by S2, S2 = c * K, where c represents the accumulation coefficient, defined as c = e s The reason for defining it as an exponential score is that the more consecutively matched segments there are, the more reliable the result. For example, in the TopK results, for a given audio segment x to be identified... i If the original audio slice y in the original audio A is matched j And the audio slice to be identified x i The next audio slice to be identified, x, corresponds to the slice offset order of the audio to be identified. i+1 Also, the original audio slice y in the original audio A. j The next original audio slice y j+1 Matching means that if two consecutive audio segments of the same original vocal track are successfully matched, then S2 = e*K. If three consecutive audio segments of the same original vocal track are successfully matched, then S2 = e*K. 2 *K.

[0090] The similarity score is added to the continuous density score to obtain the cover score for each candidate original audio segment matched by the audio segment to be identified, i.e., the cover score S = S1 + S2 for each candidate original audio segment matched by the audio segment to be identified. The maximum score among the cover scores is determined, and the original audio corresponding to the candidate original audio segment with the maximum score is taken as the cover audio corresponding to the audio segment to be identified.

[0091] As can be seen, in this embodiment, considering the issues of continuity in cover song recognition and instability of single results, a continuous density cover song recognition method is proposed, which can obtain globally better recognition results. This further improves the performance of the segment cover song recognition model and brings a better user experience to song recognition.

[0092] This application also provides a cover song recognition device, such as... Figure 5 As shown, it includes:

[0093] Acquisition unit 501 is used to acquire the audio to be recognized;

[0094] The first determining unit 502 is used to determine the audio features of multiple time-shifted audio slices in the audio to be identified.

[0095] The matching unit 503 is used to determine, from the plurality of original audio slices, a candidate original audio slice to be matched for each audio slice to be identified based on the feature similarity between the audio features of each audio slice to be identified and the audio features of each original audio slice.

[0096] The second determining unit 504 is used to determine, from among the candidate original audio slices that are matched by multiple audio slices to be identified in a continuous slice offset order, a target original audio slice that belongs to the same original audio and has a continuous slice offset order.

[0097] The identification unit 505 is used to determine the cover audio corresponding to the audio to be identified from the original audio based on the number of slices of the target original audio slice.

[0098] This application embodiment also provides a cover song recognition device 600, such as... Figure 6 As shown, the cover song recognition device 600 of this application embodiment may include one or more central processing units (CPUs) 601 and a memory 602, wherein the memory 602 stores one or more applications or data.

[0099] The memory 602 can be volatile or persistent storage. The program stored in the memory 602 can include one or more modules, each module including a series of instruction operations on the electronic device. Furthermore, the central processing unit 601 can be configured to communicate with the memory 602 and execute the series of instruction operations in the memory 602 on the cover song recognition device 600.

[0100] The cover song recognition device 600 may also include one or more power supplies 605, one or more wired or wireless network interfaces 604, one or more input / output interfaces 603, and / or one or more operating systems, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0101] The central processing unit 601 can perform the operations performed by the first aspect or any specific method embodiment of the first aspect, which will not be described in detail here.

[0102] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0103] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between apparatuses or units through some interfaces, and may be electrical, mechanical, or other forms.

[0104] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0105] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0106] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A cover song recognition method, wherein the method is applied to a retrieval database, the retrieval database comprising audio features of multiple time-shifted original audio slices, characterized in that, include: Obtain the audio to be recognized; Determine the audio features of multiple time-shifted audio slices in the audio to be identified; Based on the feature similarity between the audio features of each audio slice to be identified and the audio features of each original audio slice, candidate original audio slices matching each audio slice to be identified are determined from the multiple original audio slices. Among the candidate original audio slices that are matched by multiple consecutive audio slices to be identified in the slice offset order, the target original audio slice that belongs to the same original audio and has a consecutive slice offset order is determined. Based on the number of slices of the target original audio slice, the cover audio corresponding to the audio to be identified is determined from the original audio; The determination of target original audio slices that belong to the same original audio and whose slice offset order is continuous includes: Obtain the audio slice sequence corresponding to the audio slice to be identified, wherein each audio slice to be identified in the audio slice sequence has a corresponding slice offset order; Based on the slice offset order of each candidate original audio slice in the same original audio, each candidate original audio slice in the same original audio is arranged on the audio slice sequence; The target original audio slice is determined based on the candidate original audio slices arranged consecutively on the audio slice sequence.

2. The cover song recognition method according to claim 1, characterized in that, The audio features for determining multiple time-shifted audio slices in the audio to be identified include: A pre-trained cover song recognition model is obtained, which is trained based on a preset cover song audio and a preset original song audio, wherein the preset cover song audio is an audio obtained by covering the preset original song audio; Each audio slice to be identified in the audio to be identified is input into the cover song recognition model to obtain the audio features of each audio slice to be identified in the audio to be identified output by the cover song recognition model.

3. The cover song recognition method according to claim 1, characterized in that, The step of determining candidate original audio slices matching each audio slice to be identified from multiple original audio slices, based on the feature similarity between the audio features of each audio slice to be identified and the audio features of each original audio slice, includes: Determine the feature similarity between the audio features of any audio slice in the audio slice to be identified and the audio features of each original audio slice; For each audio slice to be identified, the original audio slice that meets the preset similarity threshold is used as the candidate original audio slice to be matched with the audio slice to be identified.

4. The cover song recognition method according to claim 1, characterized in that, The determination of the target original vocal audio slice based on consecutively arranged candidate original vocal audio slices in the audio slice sequence includes: Determine whether the candidate original audio slices in the audio slice sequence are consecutively arranged audio slices; If the candidate original audio slices are consecutive audio slices, then determine whether the consecutive candidate original audio slices are consecutive audio slices in the corresponding original audio slice offset order. If it is a continuous audio slice, then the consecutively arranged candidate original audio slices will be used as the target original audio slice.

5. The cover song recognition method according to claim 1, characterized in that, The step of determining the cover audio corresponding to the audio to be identified from the original audio based on the number of slices of the target original audio slice includes: The target original audio slice with the largest number of slices is determined from the number of slices of the target original audio slice; The original audio corresponding to the target original audio slice with the largest number of slices is taken as the cover audio corresponding to the audio to be identified.

6. The cover song recognition method according to claim 1, characterized in that, The step of determining the cover audio corresponding to the audio to be identified from the original audio based on the number of slices of the target original audio slice includes: Determine the similarity score of each candidate original audio slice matched by the audio slice to be identified, wherein the similarity score indicates the magnitude of feature similarity between the candidate original audio slice and the audio slice to be identified corresponding to the candidate original audio slice; Determine the continuous density score of each candidate original audio slice matched by the audio slice to be identified, wherein the continuous density score is proportional to the number of slices of the target original audio slice corresponding to the candidate original audio slice; The similarity score is added to the continuous density score to obtain the cover score of each candidate original audio slice matched by the audio slice to be identified; Determine the maximum score among the cover song scores, and use the original audio corresponding to the candidate original audio slice with the maximum score as the cover audio corresponding to the audio to be identified.

7. The cover song recognition method according to claim 1, characterized in that, The method further includes: Based on a preset slice offset rule, the audio to be identified is sliced ​​to obtain multiple audio slices to be identified. Based on the preset slice offset rules, the original audio is sliced ​​to obtain multiple original audio slices.

8. A cover song recognition device, characterized in that, include: Central processing unit, memory, input / output interface, wired or wireless network interface, power supply; The memory is either a short-term storage memory or a persistent storage memory; The central processing unit is configured to communicate with the memory and execute instructions in the memory on a control plane functional entity to perform the method described in any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes instructions that, when executed on a computer, cause the computer to perform the method as described in any one of claims 1 to 7.

Images