Song recognition method, device, electronic device and storage medium

Abstract

The present application relates to the field of computer technology, and discloses a song recognition method, device, electronic device and storage medium. By adopting a pre-built classification model to extract features of an audio segment to be detected, after obtaining feature information of the audio segment to be detected, the feature information of the audio segment to be detected is compared with the feature information of the original audio; if the similarity between the feature information of the audio segment to be detected and the feature information of the original audio is greater than a preset similarity, it is determined that the audio segment to be detected is a cover audio, so that the audio can be accurately identified as the original audio or the cover audio.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a song recognition method, device, electronic device and storage medium. Background Art

[0002] With the rapid development of the Internet, mobile terminals can realize more and more functions. In particular, the popularity of smart mobile terminals such as smartphones and tablets has enabled music lovers to record songs simply through applications on smart mobile terminals.

[0003] Since the user base spans all age groups and musical backgrounds, a singer's original song may have multiple cover versions. However, there is currently no technical solution for identifying cover versions. Summary of the Invention

[0004] The embodiments of the present application provide a song recognition method, device, electronic device, and storage medium that can accurately identify whether audio is original audio or cover audio.

[0005] On the one hand, an embodiment of the present application provides a song recognition method, comprising:

[0006] Using a pre-built classification model to perform feature extraction on the audio segment to be detected, to obtain feature information of the audio segment to be detected;

[0007] Comparing the feature information of the audio segment to be detected with the feature information of the original singing audio;

[0008] If the similarity between the feature information of the audio segment to be detected and the feature information of the original audio is greater than a preset similarity, determining that the audio segment to be detected is a cover audio;

[0009] The classification model is used to extract features of the audio sample and identify the original audio identifier corresponding to the audio sample based on the features of the audio sample.

[0010] An optional implementation manner is that the extracted feature information is the bottleneck feature of the classification model;

[0011] The classification model is constructed according to the following method:

[0012] Obtaining the audio sample and its corresponding annotation label, wherein the annotation label includes an identifier of the original audio corresponding to the audio sample;

[0013] Inputting the audio sample into the classification model so that the classification model outputs a predicted label of the audio sample;

[0014] The classification model is trained based on the loss between the predicted labels and the labeled labels.

[0015] In an optional implementation manner, before extracting features from the audio segment to be detected, the method further includes:

[0016] Acquire a plurality of verification samples, wherein the verification samples include audio segments other than the audio samples;

[0017] Inputting the plurality of verification samples into the pre-trained classification model to obtain feature information of each verification sample;

[0018] A first type of sample pair is formed by a verification sample corresponding to the original singer's audio and a verification sample corresponding to the cover version of the original singer's audio, and a second type of sample pair is formed by verification samples of different music, and the similarity between two samples in multiple first type sample pairs and the similarity between two samples in multiple second type sample pairs are obtained;

[0019] The similarities between two samples in the plurality of first-category sample pairs and the similarities between two samples in the plurality of second-category sample pairs are classified and processed to obtain a dividing point capable of distinguishing the first-category sample pairs from the second-category sample pairs as the preset similarity.

[0020] In an optional implementation, the feature extraction of the audio segment to be detected using a pre-built classification model includes:

[0021] Obtaining Mel-frequency cepstral coefficients of the audio segment to be detected;

[0022] The classification model is used to perform feature extraction on the Mel-frequency cepstral coefficients.

[0023] In an optional implementation manner, before extracting features from the audio segment to be detected using the pre-built classification model, the method further includes:

[0024] The audio to be detected is divided into multiple audio segments of specified duration at equal intervals to obtain the audio segment to be detected, wherein the audio segment to be detected is any one of the multiple audio segments.

[0025] An optional implementation manner is to obtain the Mel-frequency cepstral coefficients of the audio segment to be detected by the following method:

[0026] Based on a preset window length, a preset moving distance per frame, and a preset dimension of the Mel-cepstral coefficient, the audio segment to be detected is framed and windowed to determine the Mel-cepstral coefficient of the audio segment to be detected.

[0027] An optional implementation manner is that the feature extraction of the audio segment to be detected using a pre-built classification model includes:

[0028] Inputting the audio segment to be detected into the classification model, performing feature extraction, and outputting a predicted label of the audio segment to be detected and a bottleneck feature of the classification model;

[0029] Before comparing the feature information of the audio segment to be detected with the feature information of the original singing audio, the method further includes:

[0030] Based on the predicted label of the audio sample, the original singing audio whose correlation with the predicted label of the audio segment to be detected is greater than the preset label correlation is selected from the original singing audio as the original singing audio to be compared with the feature information of the audio segment to be detected.

[0031] In one aspect, an embodiment of the present application provides a song recognition device, comprising:

[0032] a feature extraction unit configured to perform feature extraction on the audio segment to be detected using a pre-built classification model to obtain feature information of the audio segment to be detected;

[0033] a comparing unit configured to compare the feature information of the audio segment to be detected with the feature information of the original singing audio;

[0034] a determining unit configured to determine that the audio segment to be detected is a cover audio segment if the similarity between the feature information of the audio segment to be detected and the feature information of the original audio segment is greater than a preset similarity;

[0035] The classification model is used to extract features of the audio sample and identify the original audio identifier corresponding to the audio sample based on the features of the audio sample.

[0036] An optional implementation manner is that the extracted feature information is the bottleneck feature of the classification model;

[0037] The classification model is constructed according to the following method:

[0038] Obtaining the audio sample and its corresponding annotation label, wherein the annotation label includes an identifier of the original audio corresponding to the audio sample;

[0039] Inputting the audio sample into the classification model so that the classification model outputs a predicted label of the audio sample;

[0040] The classification model is trained based on the loss between the predicted labels and the labeled labels.

[0041] In an optional implementation manner, before the feature extraction unit, the device further includes:

[0042] an acquiring unit, configured to acquire a plurality of verification samples, wherein the verification samples include audio segments other than the audio samples;

[0043] A first determining unit is configured to input the plurality of verification samples into the classification model to obtain feature information of each verification sample;

[0044] The second determining unit is configured to form a first type of sample pair using a verification sample corresponding to the original singer's audio and a verification sample corresponding to the cover version of the original singer's audio, and form a second type of sample pair using verification samples of different music, and obtain similarities between two samples in a plurality of first type sample pairs and similarities between two samples in a plurality of second type sample pairs;

[0045] The third determining unit is configured to perform classification processing on the similarities between two samples in multiple first-category sample pairs and the similarities between two samples in multiple second-category sample pairs, and obtain a dividing point that can distinguish between the first-category sample pairs and the second-category sample pairs as the preset similarity.

[0046] In an optional implementation manner, the feature extraction unit is configured to perform:

[0047] Obtaining Mel-frequency cepstral coefficients of the audio segment to be detected;

[0048] The classification model is used to perform feature extraction on the Mel-frequency cepstral coefficients.

[0049] In an optional implementation manner, before the feature extraction unit extracts features from the audio segment to be detected, the device further includes:

[0050] The audio to be detected is divided into multiple audio segments of specified duration at equal intervals to obtain the audio segment to be detected, wherein the audio segment to be detected is any one of the multiple audio segments.

[0051] In an optional implementation manner, the feature extraction unit is configured to perform:

[0052] Based on a preset window length, a preset moving distance per frame, and a preset dimension of the Mel-cepstral coefficient, the audio segment to be detected is framed and windowed to determine the Mel-cepstral coefficient of the audio segment to be detected.

[0053] In an optional implementation manner, the feature extraction unit is configured to perform:

[0054] Inputting the audio segment to be detected into the classification model, performing feature extraction, and outputting a predicted label of the audio segment to be detected and a bottleneck feature of the classification model;

[0055] Before the comparison unit, the device further includes:

[0056] The original singing audio acquisition unit is configured to execute the predicted label based on the audio sample, and screen the original singing audio whose correlation with the predicted label of the audio segment to be detected is greater than the preset label correlation as the original singing audio to be compared with the feature information of the audio segment to be detected.

[0057] According to a third aspect of the embodiments of the present application, there is provided an electronic device, including:

[0058] processor;

[0059] a memory for storing instructions executable by the processor;

[0060] The processor is configured to execute the instructions to implement the song recognition method as described in any one of the first aspects.

[0061] According to a fourth aspect of an embodiment of the present application, a computer-readable storage medium is provided. When instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform the song recognition method as described in any one of the first aspects.

[0062] According to a fifth aspect of an embodiment of the present application, a computer program product is provided, comprising a computer program / instruction, which, when executed by a processor, implements the song recognition method described in any one of the first aspects.

[0063] The technical solutions provided by the embodiments of this application bring at least the following beneficial effects:

[0064] By using a pre-built classification model to extract features from the audio clip to be detected, after obtaining the feature information of the audio clip to be detected, the feature information of the audio clip to be detected is compared with the feature information of the original audio; if the similarity between the feature information of the audio clip to be detected and the feature information of the original audio is greater than the preset similarity, it is determined that the audio clip to be detected is a cover audio, so that the audio can be accurately identified as the original audio or the cover audio. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. Obviously, the drawings introduced below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0066] Figure 1 A schematic diagram of an application scenario of the song recognition method provided in an embodiment of the present application;

[0067] Figure 2 A flowchart of a song recognition method provided in one embodiment of the present application;

[0068] Figure 3 A schematic diagram of the classification model training process provided in one embodiment of the present application;

[0069] Figure 4 A flowchart of a method for determining similarity provided in one embodiment of the present application;

[0070] Figure 5 A schematic diagram of the structure of a song recognition device provided in one embodiment of the present application;

[0071] Figure 6 A schematic diagram of the structure of an electronic device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0072] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0073] It should be noted that the terms "first," "second," and the like in the specification and claims of this application and the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate, so that the embodiments of the application described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Instead, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.

[0074] Below, some terms used in the embodiments of the present application are explained to facilitate understanding by those skilled in the art.

[0075] (1) In the embodiments of the present application, the term "plurality" refers to two or more than two, and other quantifiers are similar to it.

[0076] (2) “And / or” describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The character “ / ” generally indicates that the related objects are in an “or” relationship.

[0077] (3) Server: It serves the terminal. The service content includes providing resources to the terminal and saving terminal data. The server corresponds to the application installed on the terminal and runs in conjunction with the application on the terminal.

[0078] (4) Terminal device, which can refer to software APP (Application) or client. It has a visual display interface and can interact with users; it corresponds to the server and provides local services to customers. For software applications, except for some applications that only run locally, they are generally installed on ordinary client terminals and need to cooperate with the server to run. After the development of the Internet, more commonly used applications include email clients for sending and receiving emails, and instant messaging clients. For this type of application, there needs to be corresponding servers and service programs in the network to provide corresponding services, such as database services, configuration parameter services, etc., so a specific communication connection needs to be established between the client terminal and the server to ensure the normal operation of the application.

[0079] In practice, since the user base spans all age groups and musical levels, there may be multiple different cover versions of an original audio track released by a singer. A cover version can be a non-original singer's cover version using the accompaniment in the original audio track, or a non-original singer's re-performance of the original audio track. There are various types of cover versions of the original audio track. In the prior art, after extracting features from the audio clip to be detected and the original audio track, a manually designed algorithm is used to determine the similarity of the feature information and determine whether the audio clip to be detected is a cover version. However, this method makes it difficult to accurately identify whether an audio segment is an original audio track or a cover version.

[0080] To this end, the present application provides a song recognition method, which uses a pre-constructed classification model to extract features from an audio clip to be detected. After obtaining the feature information of the audio clip to be detected, the feature information of the audio clip to be detected is compared with the feature information of the original audio; if the similarity between the feature information of the audio clip to be detected and the feature information of the original audio is greater than a preset similarity, it is determined that the audio clip to be detected is a cover audio, thereby improving the accuracy of identifying the audio as the original audio or the cover audio.

[0081] After introducing the design concepts of the embodiments of this application, the following briefly introduces the application scenarios to which the technical solutions of the embodiments of this application can be applied. It should be noted that the application scenarios introduced below are only used to illustrate the embodiments of this application and are not limiting. In specific implementations, the technical solutions provided by the embodiments of this application can be flexibly applied according to actual needs.

[0082] refer to Figure 1, which is a schematic diagram of an application scenario of the song recognition method provided in an embodiment of the present application. The application scenario includes multiple terminal devices 101 (including terminal device 101-1, terminal device 101-2, ... terminal device 101-n) and a server 102. Among them, the terminal device 101 and the server 102 are connected via a wireless or wired network, and the terminal device 101 includes but is not limited to electronic devices such as desktop computers, mobile phones, mobile computers, tablet computers, media players, smart wearable devices, smart TVs, etc. The server 102 can be a single server, a server cluster consisting of several servers, or a cloud computing center. The server 102 can be an independent physical server, or a server cluster or distributed system consisting of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms.

[0083] Different users send the audio clip to be detected to the server 102 through their respective corresponding terminal devices 101. The server 102 extracts features of the audio clip to be detected through a pre-built classification model. After obtaining the feature information of the audio clip to be detected, the feature information of the audio clip to be detected is compared with the feature information of the original audio. If the similarity between the feature information of the audio clip to be detected and the feature information of the original audio is greater than the preset similarity, the audio clip to be detected is determined to be a cover audio, and the determination result is sent to its corresponding terminal device 101, and the user is prompted with the detection result through the terminal device 101.

[0084] Of course, the method provided in the embodiment of the present application is not limited to Figure 1 The application scenarios shown can also be used in other possible application scenarios, and the embodiments of the present application are not limited thereto. Figure 1 The functions that can be implemented by each device in the application scenario shown will be described in subsequent method embodiments and will not be described in detail here.

[0085] To further illustrate the technical solutions provided by the embodiments of the present application, the following is a detailed description of the technical solutions in conjunction with the accompanying drawings and specific implementation methods. Although the embodiments of the present application provide method operation steps as shown in the following embodiments or drawings, more or fewer operation steps may be included in the method based on routine or no creative work. In steps where there is no necessary causal relationship logically, the execution order of these steps is not limited to the execution order provided in the embodiments of the present application.

[0086] The following combination Figure 1 The application scenario shown illustrates the technical solution provided by the embodiment of this application.

[0087] refer to Figure 2 , an embodiment of the present application provides a song recognition method, comprising the following steps:

[0088] During implementation, some covers are performed at the climax of the audio, and a complete audio segment is often quite long. Therefore, to improve the accuracy and timeliness of cover detection, the audio to be detected can be divided into multiple audio segments of a specified duration at equal intervals to obtain audio segments to be detected, where the audio segment to be detected is any of the multiple audio segments. For example, if the audio to be detected is 2 minutes long, the audio to be detected can be divided into 12 audio segments of a specified duration of 10 seconds at equal intervals to obtain 12 audio segments. Any of the 12 audio segments can be used as the audio segment to be detected.

[0089] After obtaining the audio to be detected, in step S201 , a pre-built classification model may be used to perform feature extraction on the audio segment to be detected to obtain feature information of the audio segment to be detected.

[0090] Optionally, by obtaining the Mel-frequency cepstral coefficients of the audio segment to be detected and then performing feature extraction on the Mel-frequency cepstral coefficients, feature information of the audio segment to be detected can be obtained, thereby better sorting out the features of the audio to facilitate accurate and convenient cover recognition.

[0091] Specifically, during implementation, the audio segment to be detected may be framed and windowed based on a preset window length, a preset moving distance per frame, and a preset dimension of Mel-cepstral coefficients to determine the Mel-cepstral coefficients of the audio segment to be detected.

[0092] Mel-scale frequency cepstral coefficients (MFCCs) are cepstral parameters extracted in the mel-scale frequency domain. The mel scale describes the nonlinear frequency characteristics of the human ear. Furthermore, the features extracted using mel-scale cepstral coefficients are independent of the properties of the audio signal being detected, making no assumptions or restrictions on the input signal, and leveraging research findings in auditory models. Therefore, determining the mel-scale cepstral coefficients of the audio segment being detected is more robust, more consistent with the human auditory characteristics, and maintains good recognition performance even when the signal-to-noise ratio decreases.

[0093] In an embodiment of the present application, a pre-built classification model may be used to extract features from the audio segment to be detected; the extracted feature information is used as the bottleneck feature of the classification model.

[0094] Here, we use a trained neural network model for feature extraction to extract features that effectively describe the differences between different audio segments. Because bottleneck features can effectively improve speech recognition efficiency, the bottleneck features of the classification model are used as features for comparing the similarity of the audio segments to be tested.

[0095] Among them, the classification model is constructed according to the following method:

[0096] Obtain an audio sample and its corresponding annotated label, where the annotated label includes an identifier of the original audio corresponding to the audio sample; input the audio sample into a classification model so that the classification model outputs a predicted label for the audio sample; and train the classification model based on the loss between the predicted label and the annotated label.

[0097] During implementation, the Mel-cepstral coefficients (number of frames * dimension) of the audio sample are used as input to the classification model. For example, a Mel-cepstral coefficient feature map can be constructed based on the Mel-cepstral coefficients. The number of frames is obtained by windowing the audio sample based on a preset window length, a preset movement distance per frame, and a preset dimension of the Mel-cepstral coefficients. The dimension is the dimension of the preset Mel-cepstral coefficients.

[0098] In implementation, the feature map can be frame number*1*dimension, where frame number is the length of the feature map, 1 is the width of the feature map, and dimension is the height of the feature map, such as Figure 3 As shown in the figure, the feature map is input into the classification model. After passing through the convolutional and activation layers, pooling layers, fully connected layers, and activation layers, the bottleneck features of the audio sample are obtained. The bottleneck features of the audio sample are then passed through the classifier to obtain the predicted label of the audio sample. Among them, 301, 303, 304, 306, 307, and 308 are convolutional and activation layers, 302, 305, and 309 are pooling layers, 310 is a fully connected layer and activation layer, and 311 is a classifier.

[0099] For example, as shown in Table 1, it is assumed that the audio samples include: Audio 1-Audio Name: Little Lucky (Original Audio 1, Cover Audio 11, Cover Audio 12, Cover Audio 13), Audio 2-Audio Name: Little Dimples (Original Audio 2, Cover Audio 21, Cover Audio 22, Cover Audio 23), etc. The annotation label corresponding to audio sample 1 is the identifier of its original audio, that is, original audio 1, and the annotation label corresponding to audio sample 2 is the identifier of its original audio, that is, original audio 2. The audio samples are input into the classification model for sample training to obtain the corresponding predicted labels. The classification model is trained by calculating the loss between the predicted labels and the annotation labels.

[0100] Table 1

[0101] Audio Audio Name Original audio Cover Audio Audio 1 A Little Luck Original Audio 1 Cover Audio 11, Cover Audio 12, Cover Audio 13 Audio 2 dimples Original Audio 2 Cover Audio 21, Cover Audio 22, Cover Audio 23 ... ... ... ... Audio Sky Original audio Cover audio n1, cover audio n2, cover audio n3

[0102] After extracting the feature information of the audio segment to be detected, in step S202 , the feature information of the audio segment to be detected can be compared with the feature information of the original audio to obtain the similarity between the two audios.

[0103] During implementation, the feature information of the audio segment to be detected can be compared one by one with the feature information of the original singing audio, or the original singing audio can be screened before comparison with the audio segment to be detected.

[0104] Specifically, the classification model can output the confidence level of the audio segment to be detected classified into each original audio. In principle, the more similar the features are, the closer the classification results are. Therefore, the original audio to be compared can be found according to the classification model. It can be implemented by inputting the audio segment to be detected into a pre-built classification model, and after feature extraction, a plurality of predicted labels and their confidence levels of the audio segment to be detected can be output based on the extracted features. According to the confidence level, the original audio can be screened for the original audio whose correlation with the predicted label of the audio segment to be detected is greater than the preset label correlation level. For example, a confidence threshold can be set, and the original audio with a confidence level greater than the confidence threshold can be selected as the original audio for comparison. Alternatively, a certain number of original audios can be selected in descending order of confidence level as the audio for comparison.

[0105] For example, if the predicted label output after feature extraction of the audio clip to be detected is A, assuming that all original audio in the audio sample includes Audio 1, Audio 2, ..., Audio n (n is a positive integer) as shown in Table 2, and the predicted label corresponding to each audio is as shown in Table 1, and the preset algorithm rules determine that the correlation between Label 1, Label 2, ..., and Label i is greater than the preset label correlation, then Audio 1, Audio 2, ..., and Audio i are determined to be the filtered original audio. The specific rules of the preset algorithm are not limited here and can be adjusted according to actual application conditions.

[0106] Table 2

[0107] Audio Sample Predicted Label Audio 1 Tag 1 Audio 2 Tag 2 ... ... Audio Tag i ... ... Audio Tag

[0108] By screening the original singing audio and using the original singing audio whose correlation with the predicted label of the audio segment to be detected is greater than the preset label correlation as the original singing audio to be compared with the feature information of the audio segment to be detected, the amount of calculation can be reduced, thereby improving the detection efficiency.

[0109] After the comparison result is obtained based on step S202, in step S203, if the similarity between the feature information of the audio segment to be detected and the feature information of the original audio is greater than a preset similarity, it can be determined that the audio segment to be detected is a cover audio.

[0110] Correspondingly, if the similarity between the feature information of the audio segment to be detected and any original singing audio is less than or equal to the preset similarity, the audio segment to be detected can be determined to be the original singing audio. Afterwards, the audio segment to be detected can be stored in the original singing audio database.

[0111] By using a pre-built classification model to extract features from the audio clip to be detected, the feature information of the audio clip to be detected is compared with the feature information of the original audio to determine whether the audio clip to be detected is a cover audio. Compared with the existing technology of manually setting the feature information of the audio clip to be detected, the feature information of the audio clip to be detected is extracted using the classification model, which makes the feature information more robust and improves the accuracy of identifying whether the audio to be detected is the original audio or the cover audio.

[0112] In some embodiments, in order to reduce the false positive rate as much as possible, data mining technology can be used to reasonably determine the preset similarity. Specifically, Figure 4 As shown, the preset similarity can be determined in the following ways:

[0113] S401: Acquire multiple verification samples, where the verification samples include audio segments other than audio samples.

[0114] S402: Input multiple verification samples into a pre-trained classification model to obtain feature information of each verification sample.

[0115] S403, the first category of sample pairs is composed of the verification samples corresponding to the original audio and the verification samples corresponding to the cover audio of the original audio, and the second category of sample pairs is composed of the verification samples of different music, and the similarity between two samples in multiple first category sample pairs and the similarity between two samples in multiple second category sample pairs are obtained.

[0116] S404 , classifying the similarities between two samples in the plurality of first-category sample pairs and the similarities between two samples in the plurality of second-category sample pairs, and obtaining a cutoff point that can distinguish between the first-category sample pairs and the second-category sample pairs as a preset similarity.

[0117] Exemplarily, if all the original audios in the audio sample include audio 1, audio 2, ..., audio n (n is a positive integer) as shown in Table 1, then the multiple verification samples obtained are as shown in Table 3, including audio n+1, audio n+2, ..., audio 2n, and each audio corresponds to an original audio and three cover audios. The multiple verification samples are input into the pre-trained classification model to obtain the feature information of each verification sample. Here, the feature information of each verification sample is the bottleneck feature, as shown in Table 4.

[0118] Table 3

[0119]

[0120] Table 4

[0121] Audio Bottleneck characteristics Original audio n+1 A1 Cover audio (n+1) 1 A2 Cover Audio (n+1) 2 A3 Cover audio (n+1) 3 A4 Original audio n+2 B1 ... ... Original audio 2n N1

[0122] For example, the original audio n+1 and the cover audio (n+1)1 in Table 4 are determined as the first sample pair x1 in the first category of sample pairs, the original audio n+1 and the cover audio (n+1)2 are determined as the second sample pair x2 in the first category of sample pairs, the original audio n+1 and the cover audio (n+1)3 are determined as the third sample pair x3 in the first category of sample pairs, the original audio n+1 and the original audio n+2 are determined as the first sample pair y1 in the second category of sample pairs, the original audio n+1 and the original audio 2n are determined as the second sample pair y2 in the second category of sample pairs, and the original audio n+2 and the original audio 2n can also be determined as the third sample pair y3 in the second category of sample pairs.

[0123] According to the cosine similarity formula, the similarity of two samples in the plurality of first sample pairs and the similarity of two samples in the plurality of second sample pairs are determined, and the similarities are classified, for example, the similarities of x1, x2, and x3 are classified into one category, and the similarities of y1, y2, and y3 are classified into another category, and then the preset similarity is determined based on the demarcation point after the classification process. The specific method for determining the preset similarity based on the demarcation point is not limited here and can be adjusted according to the actual application situation. Among them, the cosine similarity formula is shown in the following formula (1):

[0124]

[0125] Where m 1k 、m 2k Determined by any two of the above x1, x2, x3, or y1, y2, y3.

[0126] By adding audio clips other than the audio sample to obtain more verification samples, and using the preset similarity calculated using the richer verification samples, the accuracy of determining whether the audio clip to be detected is a cover audio can be improved.

[0127] This embodiment uses a pre-built classification model to extract features from the audio segment to be detected. After obtaining the feature information of the audio segment to be detected, the feature information of the audio segment to be detected is compared with the feature information of the original audio. If the similarity between the feature information of the audio segment to be detected and the feature information of the original audio is greater than a preset similarity, it is determined that the audio segment to be detected is a cover audio, thereby accurately identifying the audio as the original audio or the cover audio.

[0128] like Figure 5As shown, based on the same inventive concept as the above-mentioned song recognition method, the embodiment of the present application further provides a song recognition device 50, comprising:

[0129] The feature extraction unit 501 is configured to perform feature extraction on the audio segment to be detected using a pre-built classification model to obtain feature information of the audio segment to be detected;

[0130] The comparison unit 502 is configured to compare the feature information of the audio segment to be detected with the feature information of the original singing audio;

[0131] The determining unit 503 is configured to determine that the audio segment to be detected is a cover audio segment if the similarity between the feature information of the audio segment to be detected and the feature information of the original audio segment is greater than a preset similarity;

[0132] The classification model is used to extract features of the audio sample and identify the original audio identifier corresponding to the audio sample based on the features of the audio sample.

[0133] An optional implementation manner is that the extracted feature information is the bottleneck feature of the classification model;

[0134] The classification model is constructed according to the following method:

[0135] Obtaining the audio sample and its corresponding annotation label, wherein the annotation label includes an identifier of the original audio corresponding to the audio sample;

[0136] Inputting the audio sample into the classification model so that the classification model outputs a predicted label of the audio sample;

[0137] The classification model is trained based on the loss between the predicted labels and the labeled labels.

[0138] In an optional implementation manner, before the feature extraction unit 501, the device further includes:

[0139] an acquiring unit, configured to acquire a plurality of verification samples, wherein the verification samples include audio segments other than the audio samples;

[0140] A first determining unit is configured to input the plurality of verification samples into the classification model to obtain feature information of each verification sample;

[0141] The second determining unit is configured to form a first type of sample pair using a verification sample corresponding to the original singer's audio and a verification sample corresponding to the cover version of the original singer's audio, and form a second type of sample pair using verification samples of different music, and obtain similarities between two samples in a plurality of first type sample pairs and similarities between two samples in a plurality of second type sample pairs;

[0142] The third determining unit is configured to perform classification processing on the similarities between two samples in multiple first-category sample pairs and the similarities between two samples in multiple second-category sample pairs, and obtain a dividing point that can distinguish between the first-category sample pairs and the second-category sample pairs as the preset similarity.

[0143] In an optional implementation manner, the feature extraction unit 501 is configured to perform:

[0144] Obtaining Mel-frequency cepstral coefficients of the audio segment to be detected;

[0145] The classification model is used to perform feature extraction on the Mel-frequency cepstral coefficients.

[0146] In an optional implementation manner, before the feature extraction unit 501 extracts features from the audio segment to be detected, the apparatus further includes:

[0147] The audio to be detected is divided into multiple audio segments of specified duration at equal intervals to obtain the audio segment to be detected, wherein the audio segment to be detected is any one of the multiple audio segments.

[0148] In an optional implementation manner, the feature extraction unit 501 is configured to perform:

[0149] Based on a preset window length, a preset moving distance per frame, and a preset dimension of the Mel-cepstral coefficient, the audio segment to be detected is framed and windowed to determine the Mel-cepstral coefficient of the audio segment to be detected.

[0150] In an optional implementation manner, the feature extraction unit 501 is configured to perform:

[0151] Inputting the audio segment to be detected into the classification model, performing feature extraction, and outputting a predicted label of the audio segment to be detected and a bottleneck feature of the classification model;

[0152] Before the comparing unit 502, the apparatus further includes:

[0153] The original singing audio acquisition unit is configured to execute the predicted label based on the audio sample, and screen the original singing audio whose correlation with the predicted label of the audio segment to be detected is greater than the preset label correlation as the original singing audio to be compared with the feature information of the audio segment to be detected.

[0154] After introducing the song recognition method and device according to an exemplary embodiment of the present application, next, an electronic device according to another exemplary embodiment of the present application is introduced.

[0155] Those skilled in the art will appreciate that various aspects of the present application can be implemented as systems, methods, or program products. Therefore, various aspects of the present application can be specifically implemented in the following forms: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or an implementation that combines hardware and software aspects, which may be collectively referred to herein as a "circuit," "module," or "system."

[0156] In some possible implementations, an electronic device according to the present application may include at least one processor and at least one memory. The memory stores program code that, when executed by the processor, causes the processor to perform the steps of the song identification method according to various exemplary embodiments of the present application described above. For example, the processor may perform the steps in the song identification method.

[0157] Refer to the following Figure 6 The electronic device 130 according to this embodiment of the present application is described. Figure 6 The electronic device 130 shown is merely an example and should not limit the functions and scope of use of the embodiments of the present application.

[0158] like Figure 6 As shown, the electronic device 130 is a general electronic device. Components of the electronic device 130 may include, but are not limited to, the at least one processor 131, the at least one memory 132, and a bus 133 connecting different system components (including the memory 132 and the processor 131).

[0159] Bus 133 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, and a processor or local bus using any of a variety of bus architectures.

[0160] The memory 132 may include a readable medium in the form of a volatile memory, such as a random access memory (RAM) 1321 and / or a cache memory 1322 , and may further include a read-only memory (ROM) 1323 .

[0161] The memory 132 may also include a program / utility 1325 having a set (at least one) of program modules 1324, such program modules 1324 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0162] The electronic device 130 may also communicate with one or more external devices 134 (e.g., a keyboard, pointing device, etc.), one or more devices that enable a user to interact with the electronic device 130, and / or any device that enables the electronic device 130 to communicate with one or more other electronic devices (e.g., a router, a modem, etc.). Such communication may occur via an input / output (I / O) interface 135. Furthermore, the electronic device 130 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network such as the Internet) via a network adapter 136. As shown, the network adapter 136 communicates with other modules of the electronic device 130 via a bus 133. It should be understood that, although not shown, other hardware and / or software modules may be used in conjunction with the electronic device 130, including but not limited to microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0163] In an exemplary embodiment, a computer-readable storage medium including instructions is also provided, such as a memory 132 including instructions, which can be executed by the processor 131 to perform the above method. Alternatively, the computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0164] In an exemplary embodiment, a computer program product is also provided, including a computer program / instruction, which, when executed by the processor 131, implements any of the song identification methods provided in the present application.

[0165] In an exemplary embodiment, various aspects of a song identification method provided by the present application can also be implemented in the form of a program product, which includes program code. When the program product is run on a computer device, the program code is used to enable the computer device to execute the steps of the song identification method according to various exemplary embodiments of the present application described above in this specification.

[0166] The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0167] The program product for image scaling of the embodiment of the present application may be a portable compact disc read-only memory (CD-ROM) and include program code, and can be run on an electronic device. However, the program product of the present application is not limited thereto. In this document, a readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0168] A readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries readable program code. Such propagated data signals may take a variety of forms, including, but not limited to, electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium that can transmit, propagate, or transfer a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0169] Program code embodied on a readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0170] The program code for performing the operations of the present application can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code can be executed entirely on the user electronic device, partially on the user device, as a separate software package, partially on the user electronic device and partially on a remote electronic device, or entirely on the remote electronic device or server. In cases involving remote electronic devices, the remote electronic device can be connected to the user electronic device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external electronic device (for example, using an Internet service provider to connect through the Internet).

[0171] It should be noted that although several units or subunits of the device are mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, depending on the embodiment of the application, the features and functions of two or more units described above can be embodied in a single unit. Conversely, the features and functions of a single unit described above can be further divided and embodied by multiple units.

[0172] Furthermore, although the operations of the method of the present application are described in a particular order in the accompanying drawings, this does not require or imply that the operations must be performed in this particular order, or that all illustrated operations must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps.

[0173] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0174] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable image scaling device to produce a machine, so that the instructions executed by the processor of the computer or other programmable image scaling device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0175] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable image scaling device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, the instruction device being implemented in the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0176] These computer program instructions may also be loaded onto a computer or other programmable image scaling device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, whereby the instructions executed on the computer or other programmable device provide for implementing the process described in the flow. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0177] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present application.

[0178] Obviously, those skilled in the art may make various changes and modifications to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalents, this application is intended to include these modifications and variations.

Claims

1. A song recognition method, characterized in that: include: Using a pre-built classification model to perform feature extraction on the audio segment to be detected, to obtain feature information of the audio segment to be detected; Comparing the feature information of the audio segment to be detected with the feature information of the original singing audio; If the similarity between the feature information of the audio segment to be detected and the feature information of the original audio is greater than a preset similarity, determining that the audio segment to be detected is a cover audio; The classification model is used to extract features of the audio sample and identify the original audio identifier of the audio sample based on the features of the audio sample; Before extracting features of the audio segment to be detected, the method further includes: Acquire a plurality of verification samples, wherein the verification samples include audio segments other than the audio samples; Inputting the plurality of verification samples into the classification model to obtain feature information of each verification sample; the feature information of each verification sample is a bottleneck feature; A first type of sample pair is formed by a verification sample corresponding to the original singer's audio and a verification sample corresponding to the cover version of the original singer's audio, and a second type of sample pair is formed by verification samples of different music, and the similarity between two samples in multiple first type sample pairs and the similarity between two samples in multiple second type sample pairs are obtained; The verification samples corresponding to the original audio and the verification samples corresponding to the cover audio of the original audio constitute the first type of sample pairs, and the verification samples of different music constitute the second type of sample pairs, including: If there are multiple verification samples corresponding to the cover audio of the original audio, each verification sample corresponding to the original audio and the verification sample corresponding to the cover audio of the original audio constitutes a first type of sample pair, thereby obtaining multiple first type of sample pairs; if there are multiple verification samples of different music, each two verification samples of the verification samples of different music constitute a second type of sample pair, thereby obtaining multiple second type of sample pairs; The similarities between two samples in the plurality of first-category sample pairs and the similarities between two samples in the plurality of second-category sample pairs are classified and processed to obtain a dividing point capable of distinguishing the first-category sample pairs from the second-category sample pairs as the preset similarity.

2. The method according to claim 1, characterized in that The extracted feature information is the bottleneck feature of the classification model; The classification model is constructed according to the following method: Obtaining the audio sample and its corresponding annotation label, wherein the annotation label includes an identifier of the original audio corresponding to the audio sample; Inputting the audio sample into the classification model so that the classification model outputs a predicted label of the audio sample; The classification model is trained based on the loss between the predicted labels and the labeled labels.

3. The method according to claim 2, characterized in that Before extracting features of the audio segment to be detected, the method further includes: Acquire a plurality of verification samples, wherein the verification samples include audio segments other than the audio samples; Inputting the plurality of verification samples into the classification model to obtain feature information of each verification sample; A first type of sample pair is formed by a verification sample corresponding to the original singer's audio and a verification sample corresponding to the cover version of the original singer's audio, and a second type of sample pair is formed by verification samples of different music, and the similarity between two samples in multiple first type sample pairs and the similarity between two samples in multiple second type sample pairs are obtained; The similarities between two samples in the plurality of first-category sample pairs and the similarities between two samples in the plurality of second-category sample pairs are classified and processed to obtain a dividing point capable of distinguishing the first-category sample pairs from the second-category sample pairs as the preset similarity.

4. The method according to claim 1, wherein The method of extracting features from the audio segment to be detected using a pre-built classification model includes: Obtaining Mel-frequency cepstral coefficients of the audio segment to be detected; The classification model is used to perform feature extraction on the Mel-frequency cepstral coefficients.

5. The method according to claim 1, wherein Before extracting features from the audio segment to be detected using the pre-built classification model, the method further includes: The audio to be detected is divided into multiple audio segments of specified duration at equal intervals to obtain the audio segment to be detected, wherein the audio segment to be detected is any one of the multiple audio segments.

6. The method according to claim 4, characterized in that The Mel-frequency cepstral coefficients of the audio segment to be detected are obtained by: Based on a preset window length, a preset moving distance per frame, and a preset dimension of the Mel-cepstral coefficient, the audio segment to be detected is framed and windowed to determine the Mel-cepstral coefficient of the audio segment to be detected.

7. The method according to claim 1, characterized in that The method of extracting features from the audio segment to be detected using a pre-built classification model includes: Inputting the audio segment to be detected into the classification model, performing feature extraction, and outputting a predicted label of the audio segment to be detected and a bottleneck feature of the classification model; Before comparing the feature information of the audio segment to be detected with the feature information of the original singing audio, the method further includes: Based on the predicted label of the audio sample, the original singing audio whose correlation with the predicted label of the audio segment to be detected is greater than the preset label correlation is selected from the original singing audio as the original singing audio to be compared with the feature information of the audio segment to be detected.

8. A song recognition device, characterized in that: include: a feature extraction unit configured to perform feature extraction on the audio segment to be detected using a pre-built classification model to obtain feature information of the audio segment to be detected; a comparing unit configured to compare the feature information of the audio segment to be detected with the feature information of the original singing audio; a determining unit configured to determine that the audio segment to be detected is a cover audio segment if the similarity between the feature information of the audio segment to be detected and the feature information of the original audio segment is greater than a preset similarity; The classification model is used to extract features of the audio sample and identify the original audio identifier of the audio sample based on the features of the audio sample; Before the feature extraction unit, the device further includes: an acquiring unit, configured to acquire a plurality of verification samples, wherein the verification samples include audio segments other than the audio samples; A first determining unit is configured to input the plurality of verification samples into the classification model to obtain feature information of each verification sample; the feature information of each verification sample is a bottleneck feature; The second determining unit is configured to form a first type of sample pair using a verification sample corresponding to the original singer's audio and a verification sample corresponding to the cover version of the original singer's audio, and form a second type of sample pair using verification samples of different music, and obtain similarities between two samples in a plurality of first type sample pairs and similarities between two samples in a plurality of second type sample pairs; The verification sample corresponding to the original singing audio and the verification sample corresponding to the cover version of the original singing audio constitute a first type of sample pair, and the verification samples of different music constitute a second type of sample pair, and the second determining unit is configured to: If there are multiple verification samples corresponding to the cover audio of the original audio, each verification sample corresponding to the original audio and the verification sample corresponding to the cover audio of the original audio constitutes a first type of sample pair, thereby obtaining multiple first type of sample pairs; if there are multiple verification samples of different music, each two verification samples of the verification samples of different music constitute a second type of sample pair, thereby obtaining multiple second type of sample pairs; The third determining unit is configured to perform classification processing on the similarities between two samples in multiple first-category sample pairs and the similarities between two samples in multiple second-category sample pairs, and obtain a dividing point that can distinguish between the first-category sample pairs and the second-category sample pairs as the preset similarity.

9. The device according to claim 8, characterized in that The extracted feature information is the bottleneck feature of the classification model; The classification model is constructed according to the following method: Obtaining the audio sample and its corresponding annotation label, wherein the annotation label includes an identifier of the original audio corresponding to the audio sample; Inputting the audio sample into the classification model so that the classification model outputs a predicted label of the audio sample; The classification model is trained based on the loss between the predicted labels and the labeled labels.

10. The device according to claim 9, characterized in that Before the feature extraction unit, the device further includes: an acquiring unit, configured to acquire a plurality of verification samples, wherein the verification samples include audio segments other than the audio samples; A first determining unit is configured to input the plurality of verification samples into the classification model to obtain feature information of each verification sample; The second determining unit is configured to form a first type of sample pair using a verification sample corresponding to the original singer's audio and a verification sample corresponding to the cover version of the original singer's audio, and form a second type of sample pair using verification samples of different music, and obtain similarities between two samples in a plurality of first type sample pairs and similarities between two samples in a plurality of second type sample pairs; The third determining unit is configured to perform classification processing on the similarities between two samples in multiple first-category sample pairs and the similarities between two samples in multiple second-category sample pairs, and obtain a dividing point that can distinguish between the first-category sample pairs and the second-category sample pairs as the preset similarity.

11. The device according to claim 8, characterized in that The feature extraction unit is configured to perform: Obtaining Mel-frequency cepstral coefficients of the audio segment to be detected; The classification model is used to perform feature extraction on the Mel-frequency cepstral coefficients.

12. The device according to claim 8, characterized in that Before the feature extraction unit extracts features from the audio segment to be detected, the device further includes: The audio to be detected is divided into multiple audio segments of specified duration at equal intervals to obtain the audio segment to be detected, wherein the audio segment to be detected is any one of the multiple audio segments.

13. The device according to claim 11, characterized in that The feature extraction unit is configured to perform: Based on a preset window length, a preset moving distance per frame, and a preset dimension of the Mel-cepstral coefficient, the audio segment to be detected is framed and windowed to determine the Mel-cepstral coefficient of the audio segment to be detected.

14. The device according to claim 9, characterized in that The feature extraction unit is configured to perform: Inputting the audio segment to be detected into the classification model, performing feature extraction, and outputting a predicted label of the audio segment to be detected and a bottleneck feature of the classification model; Before the comparison unit, the device further includes: The original singing audio acquisition unit is configured to execute the predicted label based on the audio sample, and screen the original singing audio whose correlation with the predicted label of the audio segment to be detected is greater than the preset label correlation as the original singing audio to be compared with the feature information of the audio segment to be detected.

15. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the song recognition method according to any one of claims 1 to 7.

16. A computer-readable storage medium, characterized in that When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform the song identification method according to any one of claims 1 to 7.

Images