Cover song recognition model training method and cover song recognition method, equipment, and medium

Abstract

The present application discloses a training method for a cover song recognition model and a cover song recognition method, device, and medium, which belong to the field of artificial intelligence. The method includes: obtaining original audio, cover audio, and other audio in a music library; extracting embedded features of the original audio, cover audio, and other audio through a first neural network; obtaining real song audio, which is song audio collected in a real environment scene; extracting the first audio feature of the real song audio, performing a reconstruction operation on the first audio feature through a second neural network to obtain a second audio feature; constructing a triplet loss based on the similarity between the embedded features of the original audio, cover audio, and other audio; and constructing a reconstruction loss based on the similarity between the first audio feature and the second audio feature; training a cover song recognition model based on the triplet loss and the reconstruction loss. The above training method solves the domain mismatch problem of the cover song recognition model.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and in particular to a training method for a cover song recognition model and a cover song recognition method, device, and medium. Background Art

[0002] The cover song recognition model is used to match the cover song audio collected in real-world scenarios with the original song audio in the music library.

[0003] In the related art, a cover song recognition model is trained based on the audio in a music library. For example, the cover song recognition model is trained based on the similarity between the original song audio and the cover song audio in the music library.

[0004] However, there's a domain mismatch between cover songs in a music library and those collected in real-world scenarios. Cover songs in a music library are typically recorded in a studio or with professional equipment, resulting in clear, noise-free sound. Real-world cover songs, on the other hand, may be recorded on a user's phone, which can be noisy and potentially recorded in the far field. Therefore, the accuracy of a cover song recognition model trained using cover songs from a music library still needs to be improved in real-world applications. Summary of the Invention

[0005] This application provides a training method for a cover song recognition model, as well as a cover song recognition method, device, and medium, which solves the problem of domain mismatch of the cover song recognition model. The technical solution includes the following contents.

[0006] According to one aspect of the present application, a method for training a cover song recognition model is provided, wherein the cover song recognition model includes a first neural network and a second neural network in parallel. The method includes the following steps.

[0007] Obtain original audio, cover audio, and other audio from the music library, where the cover audio is the audio of the original song corresponding to the cover audio, and the other audio is the audio of a song track different from the original song; extract embedding features of the original audio, cover audio, and other audio through a first neural network;

[0008] Acquire real song audio, where the real song audio is song audio collected in a real environment scene; extract a first audio feature of the real song audio, and perform a reconstruction operation on the first audio feature through a second neural network to obtain a second audio feature;

[0009] Based on the similarity between the embedding features of the original audio, the cover audio, and the other audio, a triplet loss is constructed; and based on the similarity between the first audio feature and the second audio feature, a reconstruction loss is constructed;

[0010] Based on triplet loss and reconstruction loss, the cover song recognition model is trained.

[0011] According to one aspect of the present application, a method for identifying cover songs is provided, comprising the following steps:

[0012] Obtain multiple original singer audios from a music library, and extract the embedding features of each original singer audio using a pre-trained first neural network to obtain the original singer embedding features; wherein the first neural network is trained by any of the above methods;

[0013] Obtaining a cover song audio to be retrieved, and extracting embedding features of the cover song audio to be retrieved through a first neural network to obtain embedding features to be retrieved;

[0014] Based on the feature distance between the original singer embedded feature of each original singer audio and the embedded feature to be retrieved, the cover song recognition result of the cover song audio to be retrieved is determined in multiple original singer audios.

[0015] According to one aspect of the present application, a computer device is provided, comprising: a processor and a memory, wherein the memory stores a computer program, and the computer program is loaded and executed by the processor to implement the above-mentioned training method for a cover song recognition model or the cover song recognition method.

[0016] According to another aspect of the present application, a computer-readable storage medium is provided, which stores a computer program. The computer program is loaded and executed by a processor to implement the above-mentioned training method for the cover song recognition model or the cover song recognition method.

[0017] According to another aspect of the present application, a computer program product or computer program is provided, the computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the above-described method for training a cover song recognition model or the cover song recognition method.

[0018] The beneficial effects of the technical solutions provided in the embodiments of the present application include at least:

[0019] In an embodiment of the present application, a triplet loss is constructed based on the similarity between the embedded features of the original audio, cover audio, and other audio in the music library. The triplet loss constrains the model to learn the ability to "match the cover audio to the original audio"; and the present application will also construct a reconstruction loss based on the similarity between the first audio feature of the actual song audio and the reconstructed second audio feature. The reconstruction loss constrains the model to learn the "environmental features when the song was actually collected." In an embodiment of the present application, the reconstruction loss is used to allow the cover recognition model to learn the complex environment when the song was actually collected, thereby solving the domain mismatch problem caused by training the cover recognition model with cover audio in the music library. The trained cover recognition model is more suitable for cover audio collected in real-world scenarios, and the accuracy of model recognition is improved.

[0020] Moreover, the embodiment of the present application does not directly use the cover audio in the real environment scenario (i.e., directly constructing the triplet loss based on the cover audio in the real environment scenario) to solve the domain mismatch problem. This is because obtaining the labels of the cover audio in the real environment scenario requires a lot of manpower (the label here refers to the correspondence between the cover audio and the original audio in the real environment scenario). The embodiment of the present application adds a second neural network to assist in the training of the first neural network, thereby eliminating the need to label the real song audio and saving labeling resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0022] Figure 1 Schematic diagram of a training and reasoning method for a cover song recognition model provided in one embodiment of the present application.

[0023] Figure 2 This is a flowchart of a method for training a cover song recognition model provided by one embodiment of the present application.

[0024] Figure 3 This is a schematic diagram of a method for constructing reconstruction loss provided in an embodiment of the present application.

[0025] Figure 4 This is a schematic diagram of a method for constructing triplet loss provided in an embodiment of the present application.

[0026] Figure 5 This is a flowchart of an inference method for a cover song recognition model provided by an embodiment of the present application.

[0027] Figure 6 This is a structural block diagram of a training device for a cover song recognition model provided by an embodiment of the present application.

[0028] Figure 7 This is a structural block diagram of a cover song recognition device provided by an embodiment of the present application.

[0029] Figure 8 This is a structural block diagram of a computer device provided in one embodiment of the present application.

[0030] Figure 9 This is a structural block diagram of a computer device provided in another embodiment of the present application. DETAILED DESCRIPTION

[0031] In order to make the objectives, technical solutions and advantages of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0032] First, a brief introduction is given to the terms involved in the embodiments of this application.

[0033] Cover song recognition model: Used to match cover songs collected in real-world scenarios to original songs in the music library. Users often use the song recognition feature in real-world scenarios. When users collect cover songs in real-world scenarios, the cover song recognition model will match the collected cover songs to the original songs in the music library. In some cases, the cover songs may be adapted from the original songs. In this case, the cover songs and the original songs may be similar in melody overall, but differ in some details.

[0034] When the cover recognition model is applied, the embedded features of the original audio in the music library will be extracted through the cover recognition model, as well as the embedded features of the cover audio to be retrieved collected in the real environment scene. The original audio whose embedded features are closest to the embedded features of the cover audio will be determined as the cover recognition result.

[0035] Mel-feature: also known as the Mel spectrum. The Mel spectrum is a frequency domain representation that better aligns with the human auditory characteristics. Two pairs of frequencies with equal scale differences on the Mel spectrum have the same perceived difference to the human ear. In other words, human perception is linearly related to the scale on the Mel spectrum.

[0036] CQT (Constant Q Transform) Features: Constant Q transforms exist in the audio field. Constant Q transforms use a set of constant-Q filters to filter sound signals in the time domain, thereby generating nonlinear frequency-domain signals. Constant-Q filters are constant-Q, meaning the ratio of the filter's center frequency to its bandwidth remains constant. Consequently, the filter bandwidth is narrow at low frequencies and wide at high frequencies.

[0037] Figure 1 A schematic diagram of the training process and inference process of a cover song recognition model provided by an exemplary embodiment of the present application is shown. In the present application, the cover song recognition model includes a first neural network 11 and a second neural network 12. During the training process, the first neural network 11 and the second neural network 12 will be trained, and during the inference process, the first neural network 11 will be used to perform the inference process. Figure 1 Also shown are a training device 10 and a using device 20 of the cover song recognition model. The training device 10 sends the trained first neural network 11 to the using device 20, and the using device 20 uses the first neural network 11 to perform an inference process.

[0038] like Figure 1 As shown, during the training process, original audio 101, cover audio 102, and other audio 103 are obtained from the music library. Cover audio 102 is the audio of the original song corresponding to the cover of original audio 101 in the music library, and other audio 103 is the audio of a song in the music library that is different from the original song. A first neural network 11 is used to extract embedding features of original audio 101, cover audio 102, and other audio 103, respectively, to obtain first embedding features 104, second embedding features 105, and third embedding features 106. Based on the similarity between the first embedding features 104, the second embedding features 105, and the third embedding features 106, a triplet loss 107 is constructed.

[0039] The triplet loss 107 is used to learn the melodic similarity between the original audio 101 and the cover audio 102, as well as the melodic dissimilarity between the original audio 101 and other audio 103, so as to facilitate the accurate matching of the collected cover audio to the original audio in the music library when identifying the cover.

[0040] Figure 1 It is also shown that during the training process, real song audio 108 will be obtained. Real song audio 108 includes song audio collected in a real environment scene, for example, song audio collected by a user through a mobile phone in real life. The audio features of the real song audio 108 are extracted to obtain a first audio feature 109; a reconstruction operation is performed on the first audio feature 109 through the second neural network 12 to obtain a second audio feature 110. Optionally, the second neural network 12 is an autoencoder network. Based on the similarity between the first audio feature 109 and the second audio feature 110, a reconstruction loss 111 is constructed. Based on the triple loss 107 and the reconstruction loss 111, a fusion loss 112 is obtained. Based on the fusion loss 112, a cover recognition model is trained.

[0041] The reconstruction loss 111 aims to improve the similarity between the first audio feature 109 before reconstruction and the second audio feature 110 after reconstruction, so that the cover recognition model can learn the complex environment when collecting songs in reality, and thus solve the domain mismatch problem caused by training the cover recognition model with cover audio in the music library. The trained cover recognition model is more suitable for songs collected in real environment scenarios, and the recognition accuracy is improved.

[0042] exist Figure 1 The training process shown not only learns the ability to match cover audio to original audio through triplet loss, but also learns the environmental characteristics of the actual song collection through reconstruction loss. This reconstruction loss addresses the domain mismatch issue and further improves the accuracy of cover recognition. Furthermore, by using fusion loss to simultaneously introduce cover audio from the music library and real-world song audio into the cover recognition model for training, the representational spatial distribution learned by the cover recognition model more closely matches that of the real-world song audio, further improving the model's performance in real-world applications.

[0043] exist Figure 1 In the reasoning process shown, multiple original singer audios 201 in the music library are obtained, and the embedding features of the multiple original singer audios 201 are extracted through the first neural network 11 to obtain multiple original singer embedding features ( Figure 1 It shows original singer embedding feature 1, original singer embedding feature 2, original singer embedding feature 3, and original singer embedding feature 4).

[0044] Obtain cover song audio 202 to be retrieved. Cover song audio 202 is a cover song audio collected by a user in a real-world scenario. Using a first neural network 11, embedding features of cover song audio 202 are extracted to obtain embedding features 203 to be retrieved. From multiple original singer embedding features, the original singer embedding feature with the closest feature distance to embedding feature 203 to be retrieved is determined to obtain target original singer embedding features 204. Target original singer audio 205 corresponding to target original singer embedding features 204 is determined as the cover song recognition result.

[0045] In one embodiment, the training device 10 of the cover song recognition model includes at least one of a terminal device and a server. The using device 20 of the cover song recognition model includes at least one of a terminal device and a server. Figure 1 In the figure, the training device 10 is used as a server and the user device 20 is used as a terminal device for illustration.

[0046] Terminal devices include at least one of the following: smartphones, smartwatches, in-vehicle devices, wearable devices, smart TVs, tablets, e-book readers, MP3 players, MP4 players, laptops, and desktop computers. Terminal devices include, but are not limited to, mobile phones, computers, intelligent voice interaction devices, smart home appliances, in-vehicle devices, and aircraft.

[0047] In some embodiments, the server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content distribution networks, and big data and artificial intelligence platforms.

[0048] It should be noted that the information involved in this application (including but not limited to user device information, user personal information, songs collected by users, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.) and signals are all authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0049] Moreover, when it comes to relevant information, the relevant information processors will follow the principles of legality, legitimacy and necessity, clarify the purpose, method and scope of relevant information processing, obtain the consent of the relevant information subjects, and take necessary technical and organizational measures to ensure the security of relevant information.

[0050] Figure 2 The present invention provides a method for training a cover song recognition model according to an exemplary embodiment of the present invention. The cover song recognition model includes a first neural network and a second neural network in parallel. Figure 1 For example, the method includes:

[0051] Step 210: Acquire the original singing audio, cover singing audio, and other audio in the music library, where the cover singing audio is the audio of the original singing song corresponding to the cover singing audio, and the other audio is the audio of the song track different from the original singing song;

[0052] In this application, triples are constructed by obtaining song audio from a music library. Optionally, the music library is the sound source library of a music player. The song audio in the music library is usually recorded in a recording studio or with professional equipment. The song audio in the music library is noise-free and has clear sound sources. In the constructed triples, the original audio serves as the target sample in the triple, the cover audio serves as the positive sample in the triple, and the other audio serves as the negative sample in the triple.

[0053] For example, original audio A, cover audio B of original audio A, and other audio C with different tracks from original audio A are obtained from the music library. Original audio A, cover audio B and other audio C form a triplet. Original audio A serves as the target sample (anchor) in the triplet (a, p, n), cover audio B serves as the positive sample (positive) in the triplet, and other audio C serves as the negative sample (negative) in the triplet. The melody of original audio A is more similar to that of cover audio B, while the melody of original audio A is not similar to that of other audio C.

[0054] Step 220: extracting embedding features of the original audio, the cover audio, and other audio using a first neural network;

[0055] Embedding features, also known as embedding features, are extracted by the first neural network from the original audio, cover audio, and other audio, yielding first, second, and third embedding features, respectively. Typically, embedding features have a higher feature dimension and can better characterize the song audio. Optionally, the first neural network can be a convolutional neural network (CNN), a residual neural network (ResNet), or the like.

[0056] In one embodiment, a first neural network is used to extract embedding features of the entire audio of the original singer, the entire audio of the cover version, and the entire audio of the other audio. The embedding features of the entire audio of the original singer are used as the embedding features of the original singer; the embedding features of the entire audio of the cover version are used as the embedding features of the cover version; and the embedding features of the entire audio of the other audio are used as the embedding features of the other audio.

[0057] In another embodiment, a slicing operation is performed on the original audio and the cover audio to obtain multiple original segments and multiple cover segments, respectively, wherein the original segments and the cover segments contain the same lyrics; and a slicing operation is performed on other audio to obtain multiple other audio segments; embedding features of the multiple original segments are extracted through a first neural network as embedding features of the original audio; embedding features of the multiple cover segments are extracted through the first neural network as embedding features of the cover audio; and embedding features of the multiple other audio segments are extracted through the first neural network as embedding features of the other audio.

[0058] Optionally, a slicing operation is performed based on the timestamps of the lyrics in the original and cover segments. For example, if the same line of lyrics occupies the 4th to 8th second in the original segment and the 5.5th to 9th second in the cover segment, the slicing operation will obtain the original and cover segments containing the same lyrics, and extract the embedding features of the original and cover segments through the first neural network. For other audio, other audio segments containing any line of lyrics from the other audio will be obtained, and the embedding features of the other audio segments will be extracted through the first neural network.

[0059] In the above embodiment, the original audio, cover audio and other audio are sliced, and then the embedded features of the three segments are extracted, which can perform cover recognition model training at a finer granularity, which is conducive to the cover recognition model learning the mapping relationship between finer-grained cover segments and original segments, thereby improving the accuracy of the cover recognition model in matching cover audio to original audio at a macro level.

[0060] Step 230: Acquire real song audio, where the real song audio is a song collected in a real environment scene;

[0061] In one embodiment, the real song audio is the song audio obtained through the online song recognition function. The user inputs the real song audio for song recognition through the song recognition function provided by the music player. In this application, the real song audio input by the user will be used to train the cover recognition model. The real song audio input by the user covers a variety of environmental conditions, such as various noisy background sounds, the distance between various microphones and the sound source, and so on. Training the cover recognition model with the real song audio obtained through the online song recognition function allows the cover recognition model to learn various environmental features.

[0062] Step 240: extracting a first audio feature of the actual song audio;

[0063] The first audio feature is an audio feature of a real song audio. Optionally, the first audio feature is any one of a Mel feature, a Mel cepstral coefficient feature (MFCC), a Bark spectrum feature, a CQT feature, an energy feature, a short-time zero-crossing rate feature, a fundamental frequency feature, and a harmonic feature. Optionally, the first audio feature is a Mel feature, which retains a large number of characteristics of the audio itself, and is conducive to the cover song recognition model learning the complex environmental characteristics when collecting song audio in a real scene environment.

[0064] Step 250: Reconstruct the first audio feature using a second neural network to obtain a second audio feature.

[0065] The second neural network is a neural network designed to reconstruct the first audio feature. Optionally, the second neural network is an autoencoding network, which includes an encoder and a decoder. The encoder continuously compresses the size of the first audio feature, and the decoder continuously amplifies the size of the first audio feature until the second audio feature is restored, and the size of the second audio feature is consistent with the size of the first audio feature. Optionally, the autoencoding network includes a three-layer encoder and a three-layer decoder.

[0066] Step 260: constructing a triplet loss based on the similarity between the embedding features of the original audio, the cover audio, and the other audio;

[0067] Based on the first embedding feature of the original audio, the second embedding feature of the cover audio, and the third embedding feature of the other audio, a triplet loss is constructed based on the similarity between the three embedding features.

[0068] Step 270 , constructing a reconstruction loss based on the similarity between the first audio feature and the second audio feature;

[0069] A reconstruction loss is constructed based on the similarity between the first audio feature and the second audio feature. The reconstruction loss is intended to increase the similarity between the first audio feature and the second audio feature and decrease the feature distance between the first audio feature and the second audio feature.

[0070] Step 280: Train a cover song recognition model based on triplet loss and reconstruction loss.

[0071] Based on the triplet loss and the reconstruction loss, the cover song recognition model is trained. Specifically, based on the triplet loss and the reconstruction loss, the network parameters of the first neural network and the second neural network are adjusted. In one embodiment, a weighted sum operation is performed based on the triplet loss and the reconstruction loss to obtain a fusion loss; based on the fusion loss, the cover song recognition model is trained. Optionally, the sum of the weight corresponding to the triplet loss and the weight corresponding to the reconstruction loss is one. Schematically, the fusion loss is represented by the following formula:

[0072] L total =a*L+(1-a)*L′;

[0073] Where L represents triplet loss, L′ represents reconstruction loss, and a is a weight coefficient used to control the ratio of triplet loss and reconstruction loss in the fusion loss. Optionally, a is greater than 0.5 to highlight the importance of matching the cover audio to the original audio based on melody in cover recognition, while considering the environmental features in the real environment scene to be secondary, L total In this application, the triplet loss and reconstruction loss are fused by weighted summation, which is conducive to adjusting the weights of the two, and thus the network parameters of the cover song recognition model can be successfully converged.

[0074] Figure 3 The figure shows a process of constructing the reconstruction loss provided by an exemplary embodiment of the present application.

[0075] Figure 3 In the example, real song audio 301 is obtained; first audio features 302 of real song audio 301 are extracted; a masking operation is performed on some features of first audio features 302 to obtain masked audio features 303; masked audio features 303 are input into a second neural network 31, and a reconstruction operation is performed on masked audio features 303 in a direction close to first audio features 302 by the second neural network 31 to obtain second audio features 304. A reconstruction loss 305 is constructed based on the similarity between first audio features 302 and second audio features 304.

[0076] Figure 3 During the construction of the reconstruction loss 305 shown, the first audio feature 302 will be reconstructed by masking, and the second neural network 31 will predict the masked value through the unmasked value. After continuously adjusting the parameters, the second audio feature 304 output by the second neural network 31 is more similar to the first audio feature 302, and the second neural network 31 can reconstruct the first audio feature 302 more accurately.

[0077] In one embodiment, the masking method includes at least one of the following:

[0078] First, randomly select a preset proportion of features from the first audio features and perform a masking operation to obtain masked audio features. Illustratively, the preset proportion is 50%, and 50% of the feature values ​​from the first audio features are randomly selected to perform the masking operation. Optionally, the masking operation includes setting the values ​​to zero or another smaller value.

[0079] Second, a masking operation is performed on the features of the first audio features that have maximum and / or minimum values ​​to obtain masked audio features. In the first audio features, the values ​​that are relatively abnormal (maximum and minimum) are more likely to correspond to environmental noise. By performing a masking operation on the values ​​corresponding to the environmental noise, the second neural network 31 can be constrained to reconstruct the characteristic values ​​of the environmental noise, and the cover song recognition model can better learn the environmental characteristics of the real environment scene.

[0080] In one embodiment, the real song audio includes multiple real song audios in the same training batch. For each of the multiple real song audios, a feature gap between a first audio feature and a second audio feature of the real song audio is calculated; and a reconstruction loss is obtained based on the feature gap between each of the multiple real song audios.

[0081] For the i-th real song audio among N real song audios, calculate the feature gap between the first audio feature and the second audio feature of the i-th real song audio to obtain the i-th feature gap, where N and i are both positive integers and N is greater than 1; based on the N feature gaps of the N real song audios, obtain the reconstruction loss.

[0082] Optionally, the square value of the i-th feature gap is calculated, and N square values ​​corresponding to N real song audios are accumulated to obtain the reconstruction loss.

[0083] Schematically, the reconstruction loss is expressed as follows:

[0084]

[0085] Among them, emb input_i Represents the first audio feature of the i-th real song audio, emb output_i Represents the second audio feature of the i-th real song audio, emb input_i -emb output_i represents the i-th feature gap, and L′ represents the reconstruction loss.

[0086] In the embodiment of the present application, by constructing a reconstruction loss based on each feature difference of multiple real-life song audios, it is possible to avoid the contingency of a real-life song audio, such as whether the song is easier or less difficult to reconstruct. By comprehensively constraining the reconstruction quality of multiple real-life song audios, the second neural network has a more universal ability to reconstruct real-life song audios, and the trained cover song recognition model can learn more universal environmental features. For any environmental features, the song representation constructed by the cover song recognition model can reflect them.

[0087] Figure 4 The diagram shows a construction process of triplet loss provided by an exemplary embodiment of the present application.

[0088] Figure 4 In the example, original audio 401, cover audio 402, and other audio 403 are obtained from the music library. Cover audio 402 is the audio of the original song corresponding to original audio 401, and other audio 403 is the audio of a song with a different track from the original. CQT features are extracted from original audio 401 to obtain a first CQT feature 404; CQT features are extracted from cover audio 402 to obtain a second CQT feature 405; and CQT features are extracted from other audio 403 to obtain a third CQT feature 406.

[0089] The first neural network 41 performs an embedding operation on the first CQT feature 404 to obtain a first embedded feature 407. The first neural network 41 also performs an embedding operation on the second CQT feature 405 to obtain a second embedded feature 408. The first neural network 41 also performs an embedding operation on the third CQT feature 406 to obtain a third embedded feature 409. Based on the similarity between the first embedded feature 407, the second embedded feature 408, and the third embedded feature 409, a triplet loss 410 is constructed.

[0090] In the embodiments of this application, CQT features are more suitable for melody-related tasks than other types of audio features. Extracting CQT features from the original audio, cover audio, and other audio can better learn the melodic similarities between the cover audio and the original audio, as well as the melodic dissimilarities between the original audio and other audio, thereby achieving more accurate matching between the cover audio and the original audio.

[0091] In one embodiment, the embedding feature of the original audio is the first embedding feature, the embedding feature of the cover audio is the second embedding feature, and the embedding feature of other audio is the third embedding feature.

[0092] Based on a first feature distance between the first embedded feature and the second embedded feature, a first sub-loss is obtained, where the first sub-loss is intended to reduce the first feature distance;

[0093] and, obtaining a second sub-loss based on a second feature distance between the first embedded feature and the third embedded feature; wherein the second sub-loss is intended to increase the second feature distance;

[0094] Based on the first sub-loss and the second sub-loss, the triplet loss is obtained.

[0095] Schematically, the triplet loss is formulated as follows:

[0096] L=max(d(a,p)-d(a,n)+margin,0);

[0097] Among them, a represents the first embedding feature, p represents the second embedding feature, n represents the third embedding feature, d(a,p) represents the first feature distance, that is, the first sub-loss, d(a,n) represents the second feature distance, that is, the second sub-loss, margin is an adjustable degree coefficient, and L represents the triplet loss.

[0098] In an embodiment of the present application, the triplet loss aims to shorten the first feature distance between the first embedded feature and the second embedded feature, and to increase the second feature distance between the first embedded feature and the third embedded feature. The feature distance can be understood as the distance between two embedded features in the feature space of the embedding feature (embedding). Optionally, the feature distance is a cosine distance.

[0099] In an embodiment of the present application, through triplet loss, the cover recognition model can more accurately distinguish between original audio, cover audio and other audio, and then when performing cover recognition, the cover recognition model can accurately match the cover audio to the original audio.

[0100] The above article has introduced the training process of the cover song recognition model. Next, we will introduce the inference process of the cover song recognition model.

[0101] Figure 5 A flowchart of a cover song recognition method provided by an exemplary embodiment of the present application is shown. Figure 1 For example, the method is performed using the device 20, and includes:

[0102] Step 520: Obtain multiple original singer audios from the music library, and extract the embedding features of each original singer audio using a pre-trained first neural network to obtain the original singer embedding features;

[0103] The first neural network is trained using the training method of the cover song recognition model introduced above.

[0104] In one embodiment, for multiple original singer audios in a music library, embedding features of the entire original singer audio are extracted using a first neural network to obtain multiple original singer embedding features. Optionally, CQT features of the entire original singer audio are extracted, and then the first neural network performs an embedding operation on the CQT features of the entire original singer audio to obtain the original singer embedding features.

[0105] In another embodiment, a music library contains multiple original singing audios, and a segmentation operation is performed on each original singing audio in the music library to obtain multiple original singing audio segments of each original singing audio; the embedding features of the multiple original singing audio segments are extracted through a pre-trained first neural network as the original singing embedding features of each original singing audio.

[0106] Illustratively, for the jth original singing audio among multiple original singing audios, a segmentation operation is performed on the jth original singing audio to obtain multiple jth original singing audio segments, where the jth original singing audio is any original singing audio among the multiple original singing audios, and j is a positive integer; embedding features of the multiple jth original singing audio segments are extracted through a first neural network to obtain multiple jth original singing segment embedding features, where the multiple jth original singing segment embedding features correspond one-to-one to the multiple jth original singing audio segments, and the multiple jth original singing segment embedding features serve as the jth original singing embedding features. Optionally, CQT features of the multiple jth original singing audio segments are extracted, and then an embedding operation is performed on the CQT features of the multiple jth original singing audio segments through the first neural network to obtain multiple jth original singing segment embedding features.

[0107] Step 540: Obtain the cover song audio to be retrieved, and extract the embedded features of the cover song audio to be retrieved through the first neural network to obtain the embedded features to be retrieved;

[0108] During inference, a cover song audio to be retrieved is obtained, where the cover song audio to be retrieved is a cover song audio collected in a real-world scenario. The cover song audio to be retrieved is input into the first neural network, and embedded features of the cover song audio to be retrieved are extracted. Optionally, the cover song audio to be retrieved is song audio collected by the user when using the song recognition function.

[0109] Optionally, CQT features of the cover audio to be retrieved are extracted, and an embedding operation is performed on the CQT features of the cover audio to be retrieved through a first neural network to obtain embedded features of the cover audio to be retrieved.

[0110] Step 560: determining a cover song recognition result of the cover song audio to be retrieved from the multiple original song audios based on the feature distance between the original song embedded feature of each original song audio and the embedded feature to be retrieved;

[0111] Multiple original singer audios have their own original singer embedding features. Based on the feature distance between the original singer embedding feature of each original singer audio and the embedding feature to be retrieved, the original singer embedding feature with the closest feature distance to the embedding feature to be retrieved is determined from the multiple original singer embedding features. The original singer embedding feature with the closest feature distance is then designated as the target original singer embedding feature. The original singer audio corresponding to the target original singer embedding feature is the cover recognition result for the cover audio to be retrieved.

[0112] In summary, the first neural network trained by the training method of the present application, combined with the reasoning process introduced in the above embodiment, can improve the accuracy of cover song recognition results.

[0113] Figure 6 A training device for a cover song recognition model provided by an exemplary embodiment of the present application is shown. The cover song recognition model includes a first neural network and a second neural network in parallel. The device includes:

[0114] An acquisition module 601 is configured to acquire original audio, cover audio, and other audio from a music library, wherein the cover audio is the audio of the original song corresponding to the cover audio, and the other audio is the audio of a song track different from the original song;

[0115] Processing module 602, configured to extract embedding features of the original audio, the cover audio, and other audio using a first neural network;

[0116] The acquisition module 601 is further used to acquire real song audio, where the real song audio is a song collected in a real environment scene;

[0117] The processing module 602 is further configured to extract a first audio feature of the actual song audio;

[0118] The processing module 602 is further configured to perform a reconstruction operation on the first audio feature through a second neural network to obtain a second audio feature;

[0119] A construction module 603 is used to construct a triplet loss based on the similarity between the embedding features of the original audio, the cover audio, and the other audio;

[0120] The construction module 603 is further configured to construct a reconstruction loss based on the similarity between the first audio feature and the second audio feature;

[0121] The training module 604 is used to train the cover song recognition model based on triplet loss and reconstruction loss.

[0122] In an optional embodiment, the processing module 602 is further configured to perform a masking operation on some features in the first audio feature to obtain a masked audio feature;

[0123] The masked audio features are input into a second neural network, and the second neural network performs a reconstruction operation on the masked audio features in a direction close to the first audio features to obtain a second audio feature.

[0124] In an optional embodiment, the processing module 602 is further configured to randomly select a preset proportion of features from the first audio features to perform a masking operation to obtain masked audio features; or

[0125] A masking operation is performed on features of the first audio features whose values ​​are maximum values ​​and / or minimum values ​​to obtain masked audio features.

[0126] In an optional embodiment, the real song audio includes a plurality of real song audios. The construction module 603 is further configured to calculate, for each of the plurality of real song audios, a feature gap between a first audio feature and a second audio feature of the real song audio;

[0127] A reconstruction loss is obtained based on the feature gap of each real song audio among the multiple real song audios.

[0128] In an optional embodiment, the processing module 602 is further used to perform an embedding operation on the constant Q transform (CQT) features of the original audio through the first neural network to obtain the embedded features of the original audio; perform an embedding operation on the CQT features of the cover audio through the first neural network to obtain the embedded features of the cover audio; and perform an embedding operation on the CQT features of other audio through the first neural network to obtain the embedded features of other audio.

[0129] In an optional embodiment, the processing module 602 is further configured to perform a slicing operation on the original audio and the cover audio to obtain a plurality of original recording segments and a plurality of cover recording segments, respectively, wherein the original recording segments and the cover recording segments contain the same lyrics; and perform a slicing operation on other audio to obtain a plurality of other audio segments;

[0130] The embedding features of multiple original singing segments are extracted as the embedding features of the original singing audio; the embedding features of multiple cover singing segments are extracted as the embedding features of the cover singing audio; and the embedding features of multiple other audio segments are extracted as the embedding features of other audio.

[0131] In an optional embodiment, the construction module 603 is further configured to perform a weighted sum operation based on the triplet loss and the reconstruction loss to obtain a fusion loss;

[0132] The training module 604 is further used to train a cover song recognition model based on the fusion loss.

[0133] In summary, in this application, a triplet loss will be constructed based on the similarity between the embedded features of the original audio, cover audio and other audio in the music library. The triplet loss constrains the model to learn the ability of "matching the cover audio to the original audio"; and, this application will also construct a reconstruction loss based on the similarity between the first audio feature of the real song audio and the reconstructed second audio feature. The reconstruction loss constrains the model to learn the "environmental features when the song is actually collected." In this application, the reconstruction loss is used to allow the cover recognition model to learn the complex environment when the song is actually collected, thereby solving the domain mismatch problem caused by training the cover recognition model with cover audio in the music library. The trained cover recognition model is more suitable for cover audio collected in real environment scenes, and the accuracy of model recognition is improved.

[0134] Moreover, the present application does not directly use the cover audio in the real environment scenario (i.e., directly construct the triplet loss based on the cover audio in the real environment scenario) to solve the domain mismatch problem. This is because obtaining the labels of the cover audio in the real environment scenario requires a lot of manpower for labeling (labeling the correspondence between the cover audio and the original audio in the real environment scenario). The present application assists in the training of the first neural network by adding a second neural network, thereby eliminating the need to label the real song audio, thereby saving labeling resources.

[0135] Figure 7 The following is a structural block diagram of a cover song recognition device provided by an exemplary embodiment of the present application, the device comprising:

[0136] The acquisition module 701 is used to acquire multiple original audio files in the music library;

[0137] Processing module 702, configured to extract the embedding features of each original singer audio through a pre-trained first neural network to obtain the original singer embedding features;

[0138] The acquisition module 701 is also used to obtain the cover audio to be retrieved;

[0139] Processing module 702, configured to extract embedded features of the cover audio to be retrieved through a first neural network to obtain embedded features to be retrieved;

[0140] The determination module 703 is used to determine the cover song recognition result of the cover song audio to be retrieved from multiple original song audios based on the feature distance between the original song embedded feature of each original song audio and the embedded feature to be retrieved.

[0141] In an optional embodiment, the processing module 702 is further configured to perform a segmentation operation on each original singing audio to obtain multiple original singing audio segments of each original singing audio;

[0142] The embedded features of multiple original singing audio clips are extracted through a pre-trained first neural network as the original singing embedded features of each original singing audio.

[0143] In summary, the first neural network trained by the training device of the present application, combined with the reasoning process introduced in the above embodiment, can improve the accuracy of cover song recognition results.

[0144] Figure 8 1 is a schematic diagram illustrating the structure of a computer device according to an exemplary embodiment. The computer device 800 includes a central processing unit (CPU) 801, a system memory 804 including a random access memory (RAM) 802 and a read-only memory (ROM) 803, and a system bus 805 connecting the system memory 804 and the CPU 801. The computer device 800 also includes a basic input / output (I / O) system 806 that facilitates information transmission between various components within the computer device, and a mass storage device 807 for storing an operating system 813, application programs 814, and other program modules 815.

[0145] The basic input / output system 806 includes a display 808 for displaying information and an input device 809 such as a mouse and keyboard for user input. The display 808 and the input device 809 are both connected to the central processing unit 801 via an input / output controller 810 connected to the system bus 805. The basic input / output system 806 may also include an input / output controller 810 for receiving and processing input from a variety of other devices such as a keyboard, mouse, or electronic stylus. Similarly, the input / output controller 810 also provides output to a display screen, printer, or other types of output devices.

[0146] The mass storage device 807 is connected to the central processing unit 801 via a mass storage controller (not shown) connected to the system bus 805. The mass storage device 807 and its associated computer-readable medium provide non-volatile storage for the computer device 800. In other words, the mass storage device 807 may include a computer-readable medium (not shown) such as a hard disk or a CD-ROM drive.

[0147] Without loss of generality, the computer device readable medium may include computer device storage media and communication media. Computer device storage media include volatile and non-volatile, removable and non-removable media implemented by any method or technology for storing information such as computer device readable instructions, data structures, program modules or other data. Computer device storage media include RAM, ROM, Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), CD-ROM, Digital Video Disc (DVD) or other optical storage, tape cassettes, magnetic tapes, disk storage or other magnetic storage devices. Of course, those skilled in the art will appreciate that the computer device storage media are not limited to the above-mentioned ones. The above-mentioned system memory 804 and mass storage device 807 can be collectively referred to as memory.

[0148] According to various embodiments of the present disclosure, the computer device 800 may also be connected to a remote computer device on a network such as the Internet for operation. That is, the computer device 800 may be connected to the network 811 via the network interface unit 812 connected to the system bus 805, or the network interface unit 812 may be used to connect to other types of networks or remote computer device systems (not shown).

[0149] The memory also includes one or more programs, which are stored in the memory. The central processing unit 801 implements all or part of the steps of the training method or cover song recognition method applied to the cover song recognition model by executing the one or more programs.

[0150] Figure 9 The following is a block diagram of a computer device 900 according to an exemplary embodiment of the present application. The computer device 900 may be a portable mobile terminal, such as a smartphone, a tablet computer, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 player (Moving Picture Experts Group Audio Layer IV), a laptop computer, or a desktop computer. The computer device 900 may also be referred to as a user device, a portable terminal, a laptop terminal, a desktop terminal, or other similar names.

[0151] Typically, the computer device 900 includes a processor 901 and a memory 902 .

[0152] The processor 901 may include one or more processing cores, such as a 4-core processor, a 9-core processor, etc. The processor 901 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), or PLA (Programmable Logic Array). The processor 901 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 901 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 901 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0153] The memory 902 may include one or more computer-readable storage media, which may be non-transitory. The memory 902 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices and flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 902 is used to store at least one instruction, which is used to be executed by the processor 901 to implement the training method of the cover song recognition model or the cover song recognition method provided in the method embodiment of the present application.

[0154] In some embodiments, computer device 900 may optionally include a peripheral device interface 903 and at least one peripheral device. Processor 901, memory 902, and peripheral device interface 903 may be connected via a bus or signal lines. Each peripheral device may be connected to peripheral device interface 903 via a bus, signal lines, or circuit boards. For example, the peripheral device may include at least one of a radio frequency circuit 904, a display screen 905, a camera assembly 906, an audio circuit 907, and a power supply 908.

[0155] The peripheral device interface 903 can be used to connect at least one I / O (Input / Output)-related peripheral device to the processor 901 and the memory 902. In some embodiments, the processor 901, the memory 902, and the peripheral device interface 903 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 901, the memory 902, and the peripheral device interface 903 can be implemented on separate chips or circuit boards, which is not limited in this embodiment.

[0156] The RF circuit 904 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The RF circuit 904 communicates with communication networks and other communication devices via electromagnetic signals. The RF circuit 904 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals into electrical signals. Optionally, the RF circuit 904 includes an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, and the like. The RF circuit 904 can communicate with other terminals via at least one wireless communication protocol. Such wireless communication protocols include, but are not limited to, the World Wide Web, metropolitan area networks, intranets, various generations of mobile communication networks (2G, 3G, 4G, and 5G), wireless local area networks, and / or WiFi (Wireless Fidelity) networks. In some embodiments, the RF circuit 904 may also include circuitry related to Near Field Communication (NFC), which is not limited in this application.

[0157] The display screen 905 is used to display a user interface (UI). This UI may include graphics, text, icons, videos, or any combination thereof. When the display screen 905 is a touch screen, it is also capable of collecting touch signals on or above the surface of the display screen 905. These touch signals can be input as control signals to the processor 901 for processing. In this case, the display screen 905 can also be used to provide virtual buttons and / or a virtual keyboard, also known as soft buttons and / or a soft keyboard. In some embodiments, there can be one display screen 905, located on the front panel of the computer device 900. In other embodiments, there can be at least two display screens 905, located on different surfaces of the computer device 900 or in a foldable design. In other embodiments, the display screen 905 can be a flexible display, located on a curved or foldable surface of the computer device 900. Furthermore, the display screen 905 can be configured as a non-rectangular irregular shape, i.e., a special-shaped screen. The display screen 905 can be made of materials such as LCD (Liquid Crystal Display) and OLED (Organic Light-Emitting Diode).

[0158] The camera assembly 906 is used to capture images or videos. Optionally, the camera assembly 906 includes a front camera and a rear camera. Typically, the front camera is arranged on the front panel of the terminal, and the rear camera is arranged on the back of the terminal. In some embodiments, there are at least two rear cameras, which are any one of a main camera, a depth of field camera, a wide-angle camera, and a telephoto camera, so as to realize the fusion of the main camera and the depth of field camera to realize the background blur function, the fusion of the main camera and the wide-angle camera to realize panoramic shooting and VR (Virtual Reality) shooting function or other fusion shooting functions. In some embodiments, the camera assembly 906 may also include a flash. The flash can be a monochrome temperature flash or a dual-color temperature flash. A dual-color temperature flash refers to a combination of a warm light flash and a cold light flash, which can be used for light compensation at different color temperatures.

[0159] The audio circuit 907 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, and convert the sound waves into electrical signals to be input into the processor 901 for processing, or input into the radio frequency circuit 904 to achieve voice communication. For the purpose of stereo sound collection or noise reduction, there can be multiple microphones, each located in different parts of the computer device 900. The microphone can also be an array microphone or an omnidirectional collection microphone. The speaker is used to convert the electrical signal from the processor 901 or the radio frequency circuit 904 into sound waves. The speaker can be a traditional thin film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can not only convert the electrical signal into sound waves audible to humans, but also convert the electrical signal into sound waves inaudible to humans for purposes such as distance measurement. In some embodiments, the audio circuit 907 may also include a headphone jack.

[0160] Power supply 908 is used to power various components in computer device 900. Power supply 908 can be AC ​​power, DC power, a disposable battery, or a rechargeable battery. When power supply 908 includes a rechargeable battery, the rechargeable battery can be a wired rechargeable battery or a wireless rechargeable battery. A wired rechargeable battery is a battery that is charged via a wired line, while a wireless rechargeable battery is a battery that is charged via a wireless coil. The rechargeable battery can also be used to support fast charging technology.

[0161] In some embodiments, the computer device 900 further includes one or more sensors 909 , including but not limited to: an acceleration sensor 910 , a gyroscope sensor 911 , a pressure sensor 912 , an optical sensor 913 , and a proximity sensor 914 .

[0162] The accelerometer 910 can detect the magnitude of acceleration along the three coordinate axes of the coordinate system established by the computer device 900. For example, the accelerometer 910 can be used to detect the components of gravity acceleration along the three coordinate axes. The processor 901 can control the display screen 905 to display the user interface in a landscape or portrait view based on the gravity acceleration signal collected by the accelerometer 910. The accelerometer 910 can also be used to collect game or user motion data.

[0163] The gyroscope sensor 911 can detect the orientation and rotation angle of the computer device 900. It can also work with the accelerometer 910 to collect 3D motions of the user on the computer device 900. Based on the data collected by the gyroscope sensor 911, the processor 901 can implement the following functions: motion sensing (e.g., changing the UI based on the user's tilt), image stabilization during shooting, game control, and inertial navigation.

[0164] The pressure sensor 912 can be installed on the side frame of the computer device 900 and / or below the display screen 905. When the pressure sensor 912 is installed on the side frame of the computer device 900, it can detect the user's grip signal of the computer device 900. The processor 901 can perform left and right hand recognition or shortcut operations based on the grip signal collected by the pressure sensor 912. When the pressure sensor 912 is installed below the display screen 905, the processor 901 controls the operational controls on the UI interface based on the user's pressure operation on the display screen 905. The operational controls include at least one of a button control, a scroll bar control, an icon control, and a menu control.

[0165] The optical sensor 913 is used to detect ambient light intensity. In one embodiment, the processor 901 can control the display brightness of the display screen 905 based on the ambient light intensity detected by the optical sensor 913. For example, when the ambient light intensity is high, the display brightness of the display screen 905 is increased; when the ambient light intensity is low, the display brightness of the display screen 905 is decreased. In another embodiment, the processor 901 can also dynamically adjust the shooting parameters of the camera assembly 906 based on the ambient light intensity detected by the optical sensor 913.

[0166] Proximity sensor 914, also known as a distance sensor, is typically located on the front panel of computer device 900. Proximity sensor 914 is used to detect the distance between the user and the front of computer device 900. In one embodiment, when proximity sensor 914 detects that the distance between the user and the front of computer device 900 is gradually decreasing, processor 901 controls display screen 905 to switch from a screen-on state to a screen-off state. When proximity sensor 914 detects that the distance between the user and the front of computer device 900 is gradually increasing, processor 901 controls display screen 905 to switch from a screen-off state to a screen-on state.

[0167] Those skilled in the art will understand that Figure 9 The structure shown in the figure does not constitute a limitation on the computer device 900, and the computer device 900 may include more or fewer components than shown in the figure, or combine some components, or adopt a different component arrangement.

[0168] The present application also provides a computer-readable storage medium, which stores at least one instruction, at least one program, code set or instruction set. The at least one instruction, at least one program, code set or instruction set is loaded and executed by a processor to implement the training method of the cover song recognition model or the cover song recognition method provided by the above method embodiment.

[0169] The present application provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the method for training a cover song recognition model or the method for cover song recognition provided in the above-mentioned method embodiment.

[0170] The serial numbers of the above embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0171] Those skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware, or by a program to instruct the relevant hardware, and the program may be stored in a computer-readable storage medium, which may be a read-only memory, a disk, or an optical disk, etc.

[0172] The above description is merely an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A method for training a cover song recognition model, characterized in that: The cover song recognition model includes a first neural network and a second neural network in parallel, and the method includes: Obtaining original audio, cover audio, and other audio from a music library, wherein the cover audio is an audio of a cover of the original song corresponding to the original audio, and the other audio is an audio of a song track different from the original song; extracting embedding features of the original audio, the cover audio, and the other audio through the first neural network; Acquire real song audio, where the real song audio is song audio collected in a real environment scene; extract a first audio feature of the real song audio, and perform a reconstruction operation on the first audio feature using the second neural network to obtain a second audio feature; Constructing a triplet loss based on the similarity between the embedding features of the original audio, the cover audio, and the other audio; and constructing a reconstruction loss based on the similarity between the first audio feature and the second audio feature; The cover song recognition model is trained based on the triplet loss and the reconstruction loss.

2. The method according to claim 1, characterized in that The performing a reconstruction operation on the first audio feature by the second neural network to obtain the second audio feature includes: performing a masking operation on some features of the first audio features to obtain masked audio features; The masked audio feature is input into the second neural network, and the second neural network performs a reconstruction operation on the masked audio feature in a direction close to the first audio feature to obtain the second audio feature.

3. The method according to claim 2, characterized in that The performing a masking operation on some features of the first audio features to obtain masked audio features includes: Randomly selecting a preset proportion of features from the first audio features to perform the masking operation to obtain the masked audio features; or The masking operation is performed on the features of the first audio features whose values ​​are maximum values ​​and / or minimum values ​​to obtain the masked audio features.

4. The method according to any one of claims 1 to 3, characterized in that: The real song audio includes a plurality of real song audios, and constructing the reconstruction loss based on the similarity between the first audio feature and the second audio feature includes: For each real song audio among the plurality of real song audios, calculating a feature gap between the first audio feature and the second audio feature of the real song audio; The reconstruction loss is obtained based on the feature gap of each real song audio in the multiple real song audios.

5. The method according to any one of claims 1 to 3, characterized in that: The extracting, by the first neural network, the embedding features of the original audio, the cover audio, and the other audio includes: Through the first neural network, an embedding operation is performed on the constant Q transform CQT feature of the original audio to obtain the embedding feature of the original audio; through the first neural network, an embedding operation is performed on the CQT feature of the cover audio to obtain the embedding feature of the cover audio; through the first neural network, an embedding operation is performed on the CQT feature of the other audio to obtain the embedding feature of the other audio.

6. The method according to any one of claims 1 to 3, characterized in that: The extracting of embedded features of the original audio, the cover audio, and the other audio includes: performing a slicing operation on the original audio and the cover audio to obtain a plurality of original recording segments and a plurality of cover recording segments, respectively, wherein the original recording segments and the cover recording segments contain the same lyrics; and performing a slicing operation on the other audio to obtain a plurality of other audio segments; The embedded features of the multiple original singing segments are extracted as the embedded features of the original singing audio; the embedded features of the multiple cover singing segments are extracted as the embedded features of the cover singing audio; and the embedded features of the multiple other audio segments are extracted as the embedded features of the other audio.

7. The method according to any one of claims 1 to 3, characterized in that: The step of training the cover song recognition model based on the triplet loss and the reconstruction loss includes: Performing a weighted sum operation based on the triplet loss and the reconstruction loss to obtain a fusion loss; The cover song recognition model is trained based on the fusion loss.

8. A method for identifying cover songs, characterized in that: The method comprises: Obtain multiple original singer audios from a music library, and extract the embedding features of each of the original singer audios using a pre-trained first neural network to obtain the original singer embedding features; wherein the first neural network is trained using the method described in any one of claims 1 to 7; Obtaining a cover song audio to be retrieved, and extracting embedded features of the cover song audio to be retrieved through the first neural network to obtain embedded features to be retrieved; Based on the feature distance between the original singing embedded feature of each of the original singing audios and the embedded feature to be retrieved, the cover singing recognition result of the cover singing audio to be retrieved is determined in the multiple original singing audios.

9. The method according to claim 8, characterized in that The extracting the embedding features of each original singer audio by the pre-trained first neural network to obtain the original singer embedding features includes: Perform a segmentation operation on each of the original singing audios to obtain multiple original singing audio segments of each of the original singing audios; The embedded features of multiple original singing audio segments of each original singing audio are extracted through the pre-trained first neural network, and are respectively used as the original singing embedded features of each original singing audio.

10. A computer device, characterized in that: The computer device includes: a processor and a memory, the memory storing a computer program, and the computer program is loaded and executed by the processor to implement the training method of the cover song recognition model according to any one of claims 1 to 7, or the cover song recognition method according to claim 8 or 9.

11. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which is loaded and executed by a processor to implement the training method for a cover song recognition model according to any one of claims 1 to 7, or the cover song recognition method according to claim 8 or 9.

12. A computer program product, characterized in that The computer program product stores a computer program, which is loaded and executed by a processor to implement the method for training a cover song recognition model according to any one of claims 1 to 7, or the method for recognizing a cover song according to claim 8 or 9.

Images