Training method of singing recognition model, singing recognition method, equipment and medium

Abstract

The invention discloses a training method of a turning singing recognition model, a turning singing recognition method, equipment and a medium, and belongs to the field of artificial intelligence. The method comprises the following steps: acquiring an original singing audio, a turning singing audio and other audios in a music library; extracting embedding features of the original audio, the turning audio and other audios through a first neural network; obtaining a real song audio, wherein the real song audio is a song audio collected in a real environment scene; extracting a first audio feature of the real song audio, and performing a reconstruction operation on the first audio feature through a second neural network to obtain a second audio feature; on the basis of the similarity degree among the embedding features of the original audio, the turning audio and other audios, triple loss is constructed; and constructing a reconstruction loss based on a degree of similarity between the first audio feature and the second audio feature; and training a turning singing recognition model based on the triple loss and the reconstruction loss. According to the training method, the problem of field incompatibility of the turning and singing recognition model is solved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and in particular to a training method for a cover song recognition model and a cover song recognition method, device, and medium. Background Art

[0002] The cover song recognition model is used to match the cover song audio collected in real-world scenarios with the original song audio in the music library.

[0003] In the related art, the cover recognition model is trained based on the audio in the music library. For example, the cover recognition model is trained based on the similarity between the original audio and the cover audio in the music library.

[0004] However, there is a domain mismatch between the cover audio in the music library and the cover audio collected in real-world scenarios. The cover audio in the music library is generally recorded in a recording studio or with professional equipment, and has no noise and clear sound sources; while the cover audio collected in real-world scenarios may be recorded by the user's mobile phone, and the cover audio in real-world scenarios has noise and may be recorded in the far field. Therefore, the cover recognition model trained with the cover audio in the music library still has room for improvement in recognition accuracy when it is actually applied. Summary of the invention

[0005] The present application provides a method for training a cover song recognition model and a method, device, and medium for cover song recognition, which solves the problem of domain mismatch of the cover song recognition model. The technical solution includes the following contents.

[0006] According to one aspect of the present application, a method for training a cover song recognition model is provided, wherein the cover song recognition model includes a first neural network and a second neural network in parallel. The method includes the following steps.

[0007] Obtain the original audio, cover audio and other audio in the music library, where the cover audio is the audio of the original song corresponding to the cover original audio, and the other audio is the audio of the song track different from the original song; extract the embedding features of the original audio, cover audio and other audio through the first neural network;

[0008] Acquire real song audio, where the real song audio is song audio collected in a real environment scene; extract a first audio feature of the real song audio, and perform a reconstruction operation on the first audio feature through a second neural network to obtain a second audio feature;

[0009] Based on the similarity between the embedding features of the original audio, the cover audio and the other audio, a triplet loss is constructed; and based on the similarity between the first audio feature and the second audio feature, a reconstruction loss is constructed;

[0010] Based on triplet loss and reconstruction loss, the cover song recognition model is trained.

[0011] According to one aspect of the present application, a method for identifying a cover song is provided, comprising the following steps:

[0012] Acquire multiple original singer audios in the music library, and extract the embedding features of each original singer audio through a pre-trained first neural network to obtain the original singer embedding features; wherein the first neural network is trained by any of the above methods;

[0013] Obtaining a cover song audio to be retrieved, and extracting embedded features of the cover song audio to be retrieved through a first neural network to obtain embedded features to be retrieved;

[0014] Based on the feature distance between the original singing embedded feature of each original singing audio and the embedded feature to be retrieved, the cover singing recognition result of the cover singing audio to be retrieved is determined in multiple original singing audios.

[0015] According to one aspect of the present application, a computer device is provided, comprising: a processor and a memory, the memory storing a computer program, the computer program being loaded and executed by the processor to implement the above-mentioned training method for a cover song recognition model or the cover song recognition method.

[0016] According to another aspect of the present application, a computer-readable storage medium is provided, the storage medium storing a computer program, and the computer program is loaded and executed by a processor to implement the above-mentioned training method of a cover song recognition model or the cover song recognition method.

[0017] According to another aspect of the present application, a computer program product or a computer program is provided, the computer program product or the computer program comprising computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the above-mentioned training method for a cover song recognition model or the cover song recognition method.

[0018] The beneficial effects brought by the technical solution provided by the embodiment of the present application include at least:

[0019] In an embodiment of the present application, a triplet loss is constructed based on the similarity between the embedded features of the original audio, cover audio and other audio in the music library, and the triplet loss constrains the model to learn the ability of "matching the cover audio to the original audio"; and, the present application will also construct a reconstruction loss based on the similarity between the first audio feature of the actual song audio and the reconstructed second audio feature, and the reconstruction loss constrains the model to learn the "environmental features when the song was actually collected". In an embodiment of the present application, the reconstruction loss is used to allow the cover recognition model to learn the complex environment when the song was actually collected, thereby solving the problem of domain mismatch caused by training the cover recognition model with the cover audio in the music library. The trained cover recognition model is more suitable for the cover audio collected in the actual environment scene, and the accuracy of model recognition is improved.

[0020] Furthermore, the embodiment of the present application does not directly use the cover audio in the real environment scenario (i.e., directly constructing the triplet loss based on the cover audio in the real environment scenario) to solve the domain mismatch problem. This is because obtaining the labels of the cover audio in the real environment scenario requires a lot of manpower (the labels here refer to the correspondence between the cover audio and the original audio in the real environment scenario). The embodiment of the present application adds a second neural network to assist in the training of the first neural network, thereby eliminating the need to label the real song audio, saving labeling resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0022] Figure 1 It is a schematic diagram of a training and reasoning method of a cover song recognition model provided by an embodiment of the present application.

[0023] Figure 2 It is a flowchart of a method for training a cover song recognition model provided by an embodiment of the present application.

[0024] Figure 3 It is a schematic diagram of a method for constructing reconstruction loss provided in an embodiment of the present application.

[0025] Figure 4 It is a schematic diagram of a method for constructing triplet loss provided in an embodiment of the present application.

[0026] Figure 5 It is a flowchart of an inference method of a cover song recognition model provided by an embodiment of the present application.

[0027] Figure 6 It is a structural block diagram of a training device for a cover song recognition model provided by an embodiment of the present application.

[0028] Figure 7 It is a structural block diagram of a cover song recognition device provided by an embodiment of the present application.

[0029] Figure 8 It is a structural block diagram of a computer device provided by an embodiment of the present application.

[0030] Fig. 9 It is a structural block diagram of a computer device provided in another embodiment of the present application. DETAILED DESCRIPTION

[0031] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0032] First, the nouns involved in the embodiments of the present application are briefly introduced.

[0033] Cover recognition model: used to match the cover audio collected in real-world scenarios to the original audio in the music library. Users often use the song recognition function in real-world scenarios. When users collect cover audio in real-world scenarios, the cover recognition model will be used to match the collected cover audio to the original audio in the music library. In some cases, the cover audio may be adapted from the original audio. In this case, the cover audio and the original audio are similar in melody, but there are differences in some details.

[0034] When the cover recognition model is applied, the embedded features of the original audio in the music library will be extracted through the cover recognition model, as well as the embedded features of the cover audio to be retrieved collected in the real environment scene. The original audio whose embedded features are closest to the embedded features of the cover audio will be determined as the cover recognition result.

[0035] Mel feature: Mel spectrum. Mel spectrum is a frequency domain representation method that is more in line with the auditory characteristics of the human ear. For two pairs of frequencies with equal scale differences on the Mel spectrum, the perceived difference of the human ear is also the same, that is, the human ear perception and the scale on the Mel spectrum are linearly related.

[0036] CQT (Constant Q Transform) features: In the audio field, there is a constant Q transform, which uses a set of constant Q filters to filter the sound signal in the time domain to obtain a nonlinear frequency domain signal. Among them, the constant Q filter is constant Q, that is, the center frequency and bandwidth ratio of the filter are the same, then the bandwidth of the filter is narrow at low frequencies, and the bandwidth of the filter is large at high frequencies.

[0037] Figure 1 A schematic diagram of the training process and the reasoning process of a cover recognition model provided by an exemplary embodiment of the present application is shown. In the present application, the cover recognition model includes a first neural network 11 and a second neural network 12. During the training process, the first neural network 11 and the second neural network 12 will be trained, and during the reasoning process, the first neural network 11 will be used to perform the reasoning process. Figure 1 Also shown are a training device 10 and a using device 20 of the cover song recognition model. The training device 10 sends the trained first neural network 11 to the using device 20, and the using device 20 uses the first neural network 11 to perform an inference process.

[0038] like Figure 1 As shown, during the training process, the original audio 101, the cover audio 102 and other audio 103 are obtained from the music library. The cover audio 102 is the audio of the original song corresponding to the cover of the original audio 101 in the music library, and the other audio 103 is the audio of the song track in the music library that is different from the original song. Through the first neural network 11, the embedding features of the original audio 101, the cover audio 102 and the other audio 103 are extracted to obtain the first embedding feature 104, the second embedding feature 105 and the third embedding feature 106 respectively. Based on the similarity of the first embedding feature 104, the second embedding feature 105 and the third embedding feature 106, the triple loss 107 is constructed.

[0039] The triplet loss 107 is used to learn the melodic similarity between the original audio 101 and the cover audio 102, and to learn the melodic dissimilarity between the original audio 101 and other audio 103, so as to facilitate the accurate matching of the cover audio with the original audio in the music library based on the collected cover audio during cover recognition.

[0040] Figure 1 It is also shown that real song audio 108 will be obtained during the training process. The real song audio 108 includes song audio collected in a real environment scene, for example, song audio collected by a user through a mobile phone in real life. The audio features of the real song audio 108 are extracted to obtain a first audio feature 109; the first audio feature 109 is reconstructed through the second neural network 12 to obtain a second audio feature 110. Optionally, the second neural network 12 is an autoencoder network. Based on the similarity between the first audio feature 109 and the second audio feature 110, a reconstruction loss 111 is constructed. Based on the triple loss 107 and the reconstruction loss 111, a fusion loss 112 is obtained. Based on the fusion loss 112, a cover recognition model is trained.

[0041] The reconstruction loss 111 aims to improve the similarity between the first audio feature 109 before reconstruction and the second audio feature 110 after reconstruction, so that the cover recognition model can learn the complex environment when collecting songs in reality, and then solve the domain mismatch problem caused by training the cover recognition model with cover audio in the music library. The trained cover recognition model is more suitable for songs collected in real environment scenes, and the recognition accuracy is improved.

[0042] exist Figure 1 In the training process shown, not only can the ability to "match the cover audio to the original audio" be learned through the triplet loss, but also the "environmental features when the song was actually collected" can be learned through the reconstruction loss. The reconstruction loss solves the problem of domain mismatch and further improves the accuracy of cover recognition. In addition, the cover audio and real song audio in the music library are introduced into the cover recognition model for training through the fusion loss, so that the representation space distribution learned by the cover recognition model is closer to the representation space distribution of the real song audio, which can improve the performance of the cover recognition model in actual use.

[0043] exist Figure 1 In the reasoning process shown, multiple original singer audios 201 in the music library are obtained, and the embedding features of the multiple original singer audios 201 are extracted through the first neural network 11 to obtain multiple original singer embedding features ( Figure 1 It shows original singer embedding feature 1, original singer embedding feature 2, original singer embedding feature 3, and original singer embedding feature 4).

[0044] Obtain the cover audio 202 to be retrieved, which is the cover audio collected by the user in the real environment scene, and extract the embedded features of the cover audio 202 to be retrieved through the first neural network 11 to obtain the embedded features 203 to be retrieved. From multiple original singer embedded features, determine the original singer embedded feature whose feature distance is closest to the embedded feature 203 to be retrieved, and obtain the target original singer embedded feature 204; determine the target original singer audio 205 corresponding to the target original singer embedded feature 204 as the cover recognition result.

[0045] In one embodiment, the training device 10 of the cover song recognition model includes at least one of a terminal device and a server. The using device 20 of the cover song recognition model includes at least one of a terminal device and a server. Figure 1 In the figure, the training device 10 is used as a server and the use device 20 is used as a terminal device for illustration.

[0046] The types of terminal devices include: at least one of smart phones, smart watches, vehicle terminals, wearable devices, smart TVs, tablet computers, e-book readers, MP3 players, MP4 players, laptop computers and desktop computers. Terminal devices include but are not limited to mobile phones, computers, intelligent voice interaction devices, smart home appliances, vehicle terminals, aircraft, etc.

[0047] In some embodiments, the server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content distribution networks, as well as big data and artificial intelligence platforms.

[0048] It should be noted that the information (including but not limited to user device information, user personal information, songs collected by users, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.) and signals involved in this application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0049] Moreover, when it comes to relevant information, the relevant information processors will follow the principles of legality, legitimacy and necessity, clarify the purpose, method and scope of relevant information processing, obtain the consent of the relevant information subjects, and take necessary technical and organizational measures to ensure the security of relevant information.

[0050] Figure 2 A training method for a cover song recognition model provided by an exemplary embodiment of the present application is shown. The cover song recognition model includes a first neural network and a second neural network in parallel. Figure 1 The training device 10 in the embodiment of the present invention is used as an example to illustrate that the method includes:

[0051] Step 210, obtaining the original audio, cover audio and other audio in the music library, where the cover audio is the audio of the original song corresponding to the original audio, and the other audio is the audio of the song track different from the original song;

[0052] In this application, the song audio in the music library is obtained to construct a triple. Optionally, the music library is the sound source library of a music player. The song audio in the music library is usually recorded in a recording studio or with professional equipment. The song audio in the music library is noise-free and has a clear sound source. In the constructed triple, the original audio is used as the target sample in the triple, the cover audio is used as the positive sample in the triple, and the other audio is used as the negative sample in the triple.

[0053] Exemplarily, original audio A, cover audio B of original audio A, and other audio C whose tracks are different from original audio A are obtained from the music library. Original audio A, cover audio B and other audio C form a triplet. Original audio A serves as the target sample (anchor) in the triplet (a, p, n), cover audio B serves as the positive sample (positive) in the triplet, and other audio C serves as the negative sample (negative) in the triplet. The melody of original audio A is more similar to that of cover audio B, while the melody of original audio A is not similar to that of other audio C.

[0054] Step 220, extracting embedded features of the original audio, the cover audio, and other audio through a first neural network;

[0055] Embedding features, i.e., Embedding features, the first neural network will extract the embedding features of the original audio, cover audio, and other audio, and obtain the first embedding features, the second embedding features, and the third embedding features, respectively. Generally, the embedding features have a higher feature dimension, and the embedding features can better characterize the song audio. Optionally, the first neural network is a convolutional neural network (CNN), a residual neural network (Resnet), etc.

[0056] In one embodiment, the embedding features of the entire audio of the original audio, the embedding features of the entire audio of the cover audio, and the embedding features of the entire audio of the other audio are extracted through the first neural network. The embedding features of the entire audio of the original audio are used as the embedding features of the original audio; the embedding features of the entire audio of the cover audio are used as the embedding features of the cover audio; and the embedding features of the entire audio of the other audio are used as the embedding features of the other audio.

[0057] In another embodiment, a slicing operation is performed on the original audio and the cover audio to obtain multiple original segments and multiple cover segments respectively, wherein the original segments and the cover segments contain the same lyrics; and a slicing operation is performed on other audio to obtain multiple other audio segments; embedding features of the multiple original segments are extracted through a first neural network as embedding features of the original audio; embedding features of the multiple cover segments are extracted through the first neural network as embedding features of the cover audio; and embedding features of the multiple other audio segments are extracted through the first neural network as embedding features of other audio.

[0058] Optionally, a slicing operation is performed based on the timestamps of the lyrics in the original recording segment and the cover recording segment. For example, for the same line of lyrics, the lyrics in the original recording segment occupy the time from 4s to 8s, and the lyrics in the cover recording segment occupy the time from 5.5s to 9s. When performing the slicing operation, the original recording segment and the cover recording segment containing the same lyrics will be obtained, and the embedding features of the original recording segment and the embedding features of the cover recording segment will be extracted through the first neural network. For other audio, other audio segments containing any line of lyrics in other audio will be obtained, and the embedding features of other audio segments will be extracted through the first neural network.

[0059] In the above embodiment, the original audio, cover audio and other audio are sliced, and then the embedded features of the three segments are extracted, so that the cover recognition model can be trained at a finer granularity, which is conducive to the cover recognition model learning the mapping relationship between the cover segment and the original segment at a finer granularity, thereby improving the accuracy of the cover recognition model in matching the cover audio to the original audio at a macro level.

[0060] Step 230, obtaining real song audio, where the real song audio is a song collected in a real environment scene;

[0061] In one embodiment, the real song audio is the song audio obtained through the online song recognition function. The user inputs the real song audio for song recognition through the song recognition function provided by the music player. In this application, the real song audio input by the user will be used to train the cover recognition model. The real song audio input by the user covers a variety of environmental conditions, such as various noisy background sounds, the distance between various microphones and the sound source, and so on. Training the cover recognition model with the real song audio obtained through the online song recognition function allows the cover recognition model to learn various environmental characteristics.

[0062] Step 240, extracting a first audio feature of the actual song audio;

[0063] The first audio feature is an audio feature of the actual song audio. Optionally, the first audio feature is any one of a Mel feature, a Mel cepstral coefficient feature (MFCC), a Bark spectrum feature, a CQT feature, an energy feature, a short-time zero-crossing rate feature, a fundamental frequency feature, and a harmonic feature. Optionally, the first audio feature is a Mel feature, which retains a large number of characteristics of the audio itself, and is conducive to the cover recognition model learning the complex environmental characteristics when collecting song audio in a real scene environment.

[0064] Step 250, performing a reconstruction operation on the first audio feature through a second neural network to obtain a second audio feature;

[0065] The second neural network is a neural network designed to reconstruct the first audio feature. Optionally, the second neural network is an autoencoding network, which includes an encoder and a decoder. The encoder will continuously compress the size of the first audio feature, and the decoder will continuously amplify the size of the first audio feature until the second audio feature is restored, and the size of the second audio feature is consistent with the size of the first audio feature. Optionally, the autoencoding network includes a three-layer encoder and a three-layer decoder.

[0066] Step 260, constructing a triplet loss based on the similarity between the embedded features of the original audio, the cover audio, and the other audio;

[0067] Based on the first embedding feature of the original audio, the second embedding feature of the cover audio, and the third embedding feature of the other audio, a triplet loss is constructed based on the similarity between the three embedding features.

[0068] Step 270, constructing a reconstruction loss based on the similarity between the first audio feature and the second audio feature;

[0069] Based on the similarity between the first audio feature and the second audio feature, a reconstruction loss is constructed, wherein the reconstruction loss is intended to increase the similarity between the first audio feature and the second audio feature and reduce the feature distance between the first audio feature and the second audio feature.

[0070] Step 280: training a cover song recognition model based on triplet loss and reconstruction loss.

[0071] Based on the triplet loss and the reconstruction loss, the cover song recognition model is trained. Specifically, based on the triplet loss and the reconstruction loss, the network parameters of the first neural network and the second neural network are adjusted. In one embodiment, a weighted sum operation is performed based on the triplet loss and the reconstruction loss to obtain a fusion loss; based on the fusion loss, the cover song recognition model is trained. Optionally, the sum of the weight corresponding to the triplet loss and the weight corresponding to the reconstruction loss is one. Schematically, the fusion loss is represented by the following formula:

[0072] L total =a*L+(1-a)*L′;

[0073] Wherein, L represents triplet loss, L′ represents reconstruction loss, and a is a weight coefficient, which is used to control the proportion of triplet loss and reconstruction loss in fusion loss. Optionally, a is greater than 0.5 to highlight the importance of matching the cover audio to the original audio based on the melody in cover recognition, while considering the environmental features in the real environment scene to be secondary. L total In this application, the triplet loss and the reconstruction loss are fused by weighted summation, which is conducive to adjusting the weights of the two, so that the network parameters of the cover song recognition model can be successfully converged.

[0074] Figure 3 The figure shows a process of constructing a reconstruction loss provided by an exemplary embodiment of the present application.

[0075] Figure 3 In the method, a real song audio 301 is obtained; a first audio feature 302 of the real song audio 301 is extracted; a masking operation is performed on some features of the first audio feature 302 to obtain a masked audio feature 303; the masked audio feature 303 is input into a second neural network 31, and a reconstruction operation is performed on the masked audio feature 303 in a direction close to the first audio feature 302 by the second neural network 31 to obtain a second audio feature 304. Based on the similarity between the first audio feature 302 and the second audio feature 304, a reconstruction loss 305 is constructed.

[0076] Figure 3 During the construction of the reconstruction loss 305 shown, the first audio feature 302 will be reconstructed by masking, and the second neural network 31 will predict the masked value through the unmasked value. After continuously adjusting the parameters, the second audio feature 304 output by the second neural network 31 is more similar to the first audio feature 302, and the second neural network 31 can reconstruct the first audio feature 302 more accurately.

[0077] In one embodiment, the masking method includes at least one of the following:

[0078] First, randomly select a preset ratio of features in the first audio feature to perform a masking operation to obtain a masked audio feature. Schematically, the preset ratio is 50%, and 50% of the feature values ​​in the first audio feature are randomly selected to perform a masking operation. Optionally, the masking operation includes setting the value to zero or other smaller values.

[0079] Second, perform a masking operation on the features of the first audio features that have maximum and / or minimum values ​​to obtain masked audio features. In the first audio features, the values ​​that are relatively abnormal (maximum and minimum values) are more likely to correspond to noise in the environment. By performing a masking operation on the values ​​corresponding to the noise in the environment, the second neural network 31 can be constrained to reconstruct the characteristic values ​​of the environmental noise, and the cover song recognition model can better learn the environmental characteristics of the real environment scene.

[0080] In one embodiment, the real song audio includes multiple real song audios in the same training batch. For each of the multiple real song audios, a feature gap between a first audio feature and a second audio feature of the real song audio is calculated; and a reconstruction loss is obtained based on the feature gap between each of the multiple real song audios.

[0081] For the ith real song audio among N real song audios, calculate the feature gap between the first audio feature and the second audio feature of the ith real song audio to obtain the ith feature gap, where N and i are both positive integers and N is greater than 1; based on the N feature gaps of the N real song audios, obtain the reconstruction loss.

[0082] Optionally, the square value of the i-th feature gap is calculated, and N square values ​​corresponding to N real song audios are accumulated to obtain the reconstruction loss.

[0083] Schematically, the reconstruction loss is expressed as follows:

[0084]

[0085] Among them, emb input_i represents the first audio feature of the i-th real song audio, emb output_i represents the second audio feature of the i-th real song audio, emb input_i -emb output_i represents the i-th feature gap, and L′ represents the reconstruction loss.

[0086] In the embodiment of the present application, by constructing the reconstruction loss through each feature gap of multiple real song audios, the contingency of a real song audio can be avoided, for example, the song is easier to reconstruct or less easy to reconstruct. By constraining the reconstruction quality of multiple real song audios as a whole, the second neural network has a more general ability to reconstruct real song audios, and then the trained cover recognition model can learn more general environmental features. For any environmental features, the song representation constructed by the cover recognition model can reflect them.

[0087] Figure 4 The figure shows a construction process of triplet loss provided by an exemplary embodiment of the present application.

[0088] Figure 4 In the example, the original audio 401, the cover audio 402, and the other audio 403 in the music library are obtained. The cover audio 402 is the audio of the original song corresponding to the cover audio 401, and the other audio 403 is the audio of the song track different from the original song. The CQT feature of the original audio 401 is extracted to obtain the first CQT feature 404; the CQT feature of the cover audio 402 is extracted to obtain the second CQT feature 405; the CQT feature of the other audio 403 is extracted to obtain the third CQT feature 406.

[0089] Through the first neural network 41, an embedding operation is performed on the first CQT feature 404 to obtain a first embedded feature 407; through the first neural network 41, an embedding operation is performed on the second CQT feature 405 to obtain a second embedded feature 408; through the first neural network 41, an embedding operation is performed on the third CQT feature 406 to obtain a third embedded feature 409. Based on the similarity between the first embedded feature 407, the second embedded feature 408 and the third embedded feature 409, a triple loss 410 is constructed.

[0090] In the embodiment of the present application, compared with other types of audio features, CQT features are more suitable for melody-related tasks. Extracting CQT features of original audio, cover audio and other audio can better learn the similarity of the melody between the cover audio and the original audio, as well as the dissimilarity of the melody between the original audio and other audio, thereby better achieving accurate matching between the cover audio and the original audio.

[0091] In one embodiment, the embedding feature of the original audio is the first embedding feature, the embedding feature of the cover audio is the second embedding feature, and the embedding feature of other audio is the third embedding feature.

[0092] Based on a first feature distance between the first embedded feature and the second embedded feature, a first sub-loss is obtained, where the first sub-loss is intended to reduce the first feature distance;

[0093] and, based on a second feature distance between the first embedded feature and the third embedded feature, obtaining a second sub-loss; the second sub-loss is intended to increase the second feature distance;

[0094] Based on the first sub-loss and the second sub-loss, a triplet loss is obtained.

[0095] Schematically, the triplet loss is formulated as follows:

[0096] L=max(d(a,p)-d(a,n)+margin,0);

[0097] Among them, a represents the first embedded feature, p represents the second embedded feature, n represents the third embedded feature, d(a,p) represents the first feature distance, that is, the first sub-loss, d(a,n) represents the second feature distance, that is, the second sub-loss, margin is an adjustable degree coefficient, and L represents the triplet loss.

[0098] In an embodiment of the present application, the triplet loss aims to shorten the first feature distance between the first embedded feature and the second embedded feature, and to increase the second feature distance between the first embedded feature and the third embedded feature. The feature distance can be understood as the distance between two embedded features in the feature space of the embedded feature (embedding). Optionally, the feature distance is a cosine distance.

[0099] In an embodiment of the present application, through triplet loss, the cover recognition model can more accurately distinguish between original audio, cover audio and other audio, and then when performing cover recognition, the cover recognition model can accurately match the cover audio to the original audio.

[0100] The above has introduced the training process of the cover song recognition model. Next, we will introduce the reasoning process of the cover song recognition model.

[0101] Figure 5 A flowchart of a method for identifying a cover song provided by an exemplary embodiment of the present application is shown. Figure 1 The method is illustrated by using the device 20 in the example, and includes:

[0102] Step 520, obtaining multiple original singer audios in the music library, and extracting the embedding features of each original singer audio through a pre-trained first neural network to obtain the original singer embedding features;

[0103] The first neural network is trained using the training method of the cover song recognition model introduced above.

[0104] In one embodiment, for multiple original singer audios in the music library, the embedding features of the entire original singer audio are extracted through the first neural network to obtain multiple original singer embedding features. Optionally, the CQT features of the entire original singer audio are extracted, and then the CQT features of the entire original singer audio are embedded through the first neural network to obtain the original singer embedding features.

[0105] In another embodiment, the music library contains multiple original singing audios, and a segmentation operation is performed on each original singing audio in the music library to obtain multiple original singing audio segments of each original singing audio; the embedding features of the multiple original singing audio segments are extracted through a pre-trained first neural network as the original singing embedding features of each original singing audio.

[0106] Schematically, for the jth original audio among multiple original audios, a segmentation operation is performed on the jth original audio to obtain multiple jth original audio segments, where the jth original audio is any original audio among the multiple original audios, and j is a positive integer; the embedding features of the multiple jth original audio segments are extracted through the first neural network to obtain multiple jth original recording segment embedding features, where the multiple jth original recording segment embedding features correspond one-to-one to the multiple jth original audio segments, and the multiple jth original recording segment embedding features are used as the jth original recording embedding features. Optionally, CQT features of multiple jth original audio segments are extracted, and then the CQT features of multiple jth original audio segments are embedded through the first neural network to obtain multiple jth original recording segment embedding features.

[0107] Step 540, obtaining the cover song audio to be retrieved, and extracting the embedded features of the cover song audio to be retrieved through the first neural network to obtain the embedded features to be retrieved;

[0108] During inference, the cover song audio to be retrieved is obtained, and the cover song audio to be retrieved is the cover song audio collected in the real environment scene, and the cover song audio to be retrieved is input into the first neural network to extract the embedded features of the cover song audio to be retrieved. Optionally, the cover song audio to be retrieved is the song audio collected by the user when using the song recognition function.

[0109] Optionally, CQT features of the cover audio to be retrieved are extracted, and an embedding operation is performed on the CQT features of the cover audio to be retrieved through a first neural network to obtain embedded features of the cover audio to be retrieved.

[0110] Step 560, determining a cover song recognition result of the cover song audio to be retrieved from the multiple original song audios based on the feature distance between the original song embedded feature of each original song audio and the embedded feature to be retrieved;

[0111] Multiple original singer audios have their own original singer embedding features. Based on the feature distance between the original singer embedding feature of each original singer audio and the embedding feature to be retrieved, the original singer embedding feature with the closest feature distance to the embedding feature to be retrieved is determined from multiple original singer embedding features. At this time, the original singer embedding feature with the closest feature distance is the target original singer embedding feature, and the original singer audio corresponding to the target original singer embedding feature is the cover recognition result of the cover audio to be retrieved.

[0112] In summary, the first neural network trained by the training method of the present application, combined with the reasoning process introduced in the above embodiment, can improve the accuracy of the cover song recognition results.

[0113] Figure 6 A training device for a cover song recognition model provided by an exemplary embodiment of the present application is shown. The cover song recognition model includes a first neural network and a second neural network in parallel. The device includes:

[0114] The acquisition module 601 is used to acquire the original audio, cover audio and other audio in the music library, where the cover audio is the audio of the original song corresponding to the cover original audio, and the other audio is the audio of the song track different from the original song;

[0115] Processing module 602, for extracting embedded features of original audio, cover audio and other audio through a first neural network;

[0116] The acquisition module 601 is also used to acquire real song audio, where the real song audio is a song collected in a real environment scene;

[0117] The processing module 602 is further used to extract the first audio feature of the actual song audio;

[0118] The processing module 602 is further configured to perform a reconstruction operation on the first audio feature through a second neural network to obtain a second audio feature;

[0119] A construction module 603 is used to construct a triplet loss based on the similarity between the embedding features of the original audio, the cover audio and other audio;

[0120] The construction module 603 is further used to construct a reconstruction loss based on the similarity between the first audio feature and the second audio feature;

[0121] The training module 604 is used to train the cover song recognition model based on the triplet loss and the reconstruction loss.

[0122] In an optional embodiment, the processing module 602 is further configured to perform a masking operation on some features in the first audio feature to obtain a masked audio feature;

[0123] The masked audio feature is input into a second neural network, and the second neural network performs a reconstruction operation on the masked audio feature in a direction close to the first audio feature to obtain a second audio feature.

[0124] In an optional embodiment, the processing module 602 is further configured to randomly select features of a preset proportion in the first audio features to perform a masking operation to obtain masked audio features; or

[0125] A masking operation is performed on the features whose values ​​are maximum values ​​and / or minimum values ​​in the first audio features to obtain masked audio features.

[0126] In an optional embodiment, the real song audio includes a plurality of real song audios. The construction module 603 is further configured to calculate, for each real song audio in the plurality of real song audios, a feature gap between a first audio feature and a second audio feature of the real song audio;

[0127] A reconstruction loss is obtained based on the feature gap of each real song audio among the multiple real song audios.

[0128] In an optional embodiment, the processing module 602 is further used to perform an embedding operation on the constant Q transform CQT feature of the original audio through the first neural network to obtain the embedded feature of the original audio; perform an embedding operation on the CQT feature of the cover audio through the first neural network to obtain the embedded feature of the cover audio; perform an embedding operation on the CQT feature of other audio through the first neural network to obtain the embedded feature of other audio.

[0129] In an optional embodiment, the processing module 602 is further configured to perform a slicing operation on the original audio and the cover audio to obtain a plurality of original audio segments and a plurality of cover audio segments, respectively, wherein the original audio segment and the cover audio segment contain the same lyrics; and perform a slicing operation on other audio to obtain a plurality of other audio segments;

[0130] The embedding features of multiple original singing segments are extracted as the embedding features of the original singing audio; the embedding features of multiple cover singing segments are extracted as the embedding features of the cover singing audio; and the embedding features of multiple other audio segments are extracted as the embedding features of other audio.

[0131] In an optional embodiment, the construction module 603 is further used to perform a weighted sum operation based on the triplet loss and the reconstruction loss to obtain a fusion loss;

[0132] The training module 604 is also used to train a cover song recognition model based on the fusion loss.

[0133] In summary, in this application, a triplet loss will be constructed based on the similarity between the embedded features of the original audio, cover audio and other audio in the music library, and the triplet loss constraint model learns the ability of "matching the cover audio to the original audio"; and, this application will also construct a reconstruction loss based on the similarity between the first audio feature of the actual song audio and the reconstructed second audio feature, and the reconstruction loss constraint model learns the "environmental features when the song was actually collected." In this application, the reconstruction loss is used to allow the cover recognition model to learn the complex environment when the song was actually collected, thereby solving the problem of domain mismatch caused by training the cover recognition model with the cover audio in the music library. The trained cover recognition model is more suitable for the cover audio collected in the real environment scene, and the accuracy of model recognition is improved.

[0134] Furthermore, the present application does not directly use cover audio in real-world scenarios (i.e., directly construct triplet loss based on cover audio in real-world scenarios) to solve the domain mismatch problem. This is because obtaining labels for cover audio in real-world scenarios requires a large amount of manpower for labeling (labeling the correspondence between cover audio and original audio in real-world scenarios). The present application adds a second neural network to assist in the training of the first neural network, thereby eliminating the need to label real song audio and saving labeling resources.

[0135] Figure 7 The structure block diagram of a cover song recognition device provided by an exemplary embodiment of the present application is shown, and the device includes:

[0136] The acquisition module 701 is used to acquire multiple original singing audios in the music library;

[0137] The processing module 702 is used to extract the embedding features of each original singer audio through a pre-trained first neural network to obtain the original singer embedding features;

[0138] The acquisition module 701 is also used to acquire the cover audio to be retrieved;

[0139] The processing module 702 is used to extract the embedded features of the cover audio to be retrieved through the first neural network to obtain the embedded features to be retrieved;

[0140] The determination module 703 is used to determine the cover recognition result of the cover audio to be retrieved in multiple original audios based on the feature distance between the original embedded feature of each original audio and the embedded feature to be retrieved.

[0141] In an optional embodiment, the processing module 702 is further configured to perform a segmentation operation on each original singing audio to obtain multiple original singing audio segments of each original singing audio;

[0142] The embedded features of multiple original singing audio clips are extracted through a pre-trained first neural network as the original singing embedded features of each original singing audio.

[0143] In summary, the first neural network trained by the training device of the present application, combined with the reasoning process introduced in the above embodiment, can improve the accuracy of cover song recognition results.

[0144] Figure 8 800 is a schematic diagram of a computer device according to an exemplary embodiment. The computer device 800 includes a central processing unit (CPU) 801, a system memory 804 including a random access memory (RAM) 802 and a read-only memory (ROM) 803, and a system bus 805 connecting the system memory 804 and the central processing unit 801. The computer device 800 also includes a basic input / output system (I / O system) 806 that helps transmit information between various devices in the computer device, and a large-capacity storage device 807 for storing an operating system 813, an application program 814 and other program modules 815.

[0145] The basic input / output system 806 includes a display 808 for displaying information and an input device 809 such as a mouse and a keyboard for user inputting information. The display 808 and the input device 809 are connected to the central processing unit 801 through an input / output controller 810 connected to the system bus 805. The basic input / output system 806 may also include an input / output controller 810 for receiving and processing inputs from a plurality of other devices such as a keyboard, a mouse, or an electronic stylus. Similarly, the input / output controller 810 also provides output to a display screen, a printer, or other types of output devices.

[0146] The mass storage device 807 is connected to the central processing unit 801 via a mass storage controller (not shown) connected to the system bus 805. The mass storage device 807 and its associated computer device readable medium provide non-volatile storage for the computer device 800. That is, the mass storage device 807 may include a computer device readable medium (not shown) such as a hard disk or a compact disc read-only memory (CD-ROM) drive.

[0147] Without loss of generality, the computer device readable medium may include computer device storage media and communication media. Computer device storage media include volatile and non-volatile, removable and non-removable media implemented by any method or technology for storing information such as computer device readable instructions, data structures, program modules or other data. Computer device storage media include RAM, ROM, Erasable Programmable Read Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), CD-ROM, Digital Video Disc (DVD) or other optical storage, cassette, tape, disk storage or other magnetic storage devices. Of course, those skilled in the art will know that the computer device storage medium is not limited to the above. The above-mentioned system memory 804 and mass storage device 807 can be collectively referred to as memory.

[0148] According to various embodiments of the present disclosure, the computer device 800 can also be connected to a remote computer device on a network through a network such as the Internet. That is, the computer device 800 can be connected to the network 811 through the network interface unit 812 connected to the system bus 805, or the network interface unit 812 can be used to connect to other types of networks or remote computer device systems (not shown).

[0149] The memory also includes one or more programs, which are stored in the memory. The central processing unit 801 implements all or part of the steps of the training method or cover recognition method applied to the cover recognition model by executing the one or more programs.

[0150] Fig. 9 The block diagram of the structure of a computer device 900 provided by an exemplary embodiment of the present application is shown. The computer device 900 may be a portable mobile terminal, such as a smart phone, a tablet computer, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 player (Moving Picture Experts Group Audio Layer IV), a laptop computer or a desktop computer. The computer device 900 may also be called a user device, a portable terminal, a laptop terminal, a desktop terminal or other names.

[0151] Typically, the computer device 900 includes a processor 901 and a memory 902 .

[0152] The processor 901 may include one or more processing cores, such as a 4-core processor, a 9-core processor, etc. The processor 901 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 901 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 901 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 901 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0153] The memory 902 may include one or more computer-readable storage media, which may be non-transitory. The memory 902 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 902 is used to store at least one instruction, which is used to be executed by the processor 901 to implement the training method of the cover recognition model or the cover recognition method provided in the method embodiment of the present application.

[0154] In some embodiments, the computer device 900 may further optionally include: a peripheral device interface 903 and at least one peripheral device. The processor 901, the memory 902 and the peripheral device interface 903 may be connected via a bus or a signal line. Each peripheral device may be connected to the peripheral device interface 903 via a bus, a signal line or a circuit board. For example, the peripheral device may include: at least one of a radio frequency circuit 904, a display screen 905, a camera assembly 906, an audio circuit 907 and a power supply 908.

[0155] The peripheral device interface 903 may be used to connect at least one peripheral device related to I / O (Input / Output) to the processor 901 and the memory 902. In some embodiments, the processor 901, the memory 902, and the peripheral device interface 903 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 901, the memory 902, and the peripheral device interface 903 may be implemented on a separate chip or circuit board, which is not limited in this embodiment.

[0156] The radio frequency circuit 904 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The radio frequency circuit 904 communicates with communication networks and other communication devices through electromagnetic signals. The radio frequency circuit 904 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals into electrical signals. Optionally, the radio frequency circuit 904 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, and the like. The radio frequency circuit 904 can communicate with other terminals through at least one wireless communication protocol. The wireless communication protocol includes, but is not limited to: the World Wide Web, a metropolitan area network, an intranet, various generations of mobile communication networks (2G, 3G, 4G and 5G), a wireless local area network and / or a WiFi (Wireless Fidelity) network. In some embodiments, the radio frequency circuit 904 may also include circuits related to NFC (Near Field Communication), which is not limited in this application.

[0157] The display screen 905 is used to display the UI (User Interface). The UI may include graphics, text, icons, videos, and any combination thereof. When the display screen 905 is a touch display screen, the display screen 905 also has the ability to collect touch signals on the surface or above the surface of the display screen 905. The touch signal can be input to the processor 901 as a control signal for processing. At this time, the display screen 905 can also be used to provide virtual buttons and / or virtual keyboards, also known as soft buttons and / or soft keyboards. In some embodiments, the display screen 905 can be one, set on the front panel of the computer device 900; in other embodiments, the display screen 905 can be at least two, respectively set on different surfaces of the computer device 900 or in a folding design; in other embodiments, the display screen 905 can be a flexible display screen, set on the curved surface or folding surface of the computer device 900. Even, the display screen 905 can also be set to a non-rectangular irregular shape, that is, a special-shaped screen. The display screen 905 can be made of materials such as LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode, organic light-emitting diode).

[0158] The camera assembly 906 is used to capture images or videos. Optionally, the camera assembly 906 includes a front camera and a rear camera. Typically, the front camera is arranged on the front panel of the terminal, and the rear camera is arranged on the back of the terminal. In some embodiments, there are at least two rear cameras, which are any one of a main camera, a depth of field camera, a wide-angle camera, and a telephoto camera, so as to realize the fusion of the main camera and the depth of field camera to realize the background blur function, the fusion of the main camera and the wide-angle camera to realize panoramic shooting and VR (Virtual Reality) shooting function or other fusion shooting functions. In some embodiments, the camera assembly 906 may also include a flash. The flash can be a monochrome temperature flash or a dual-color temperature flash. A dual-color temperature flash refers to a combination of a warm light flash and a cold light flash, which can be used for light compensation at different color temperatures.

[0159] The audio circuit 907 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, and convert the sound waves into electrical signals and input them into the processor 901 for processing, or input them into the radio frequency circuit 904 to achieve voice communication. For the purpose of stereo acquisition or noise reduction, there may be multiple microphones, which are respectively arranged at different parts of the computer device 900. The microphone may also be an array microphone or an omnidirectional acquisition microphone. The speaker is used to convert the electrical signal from the processor 901 or the radio frequency circuit 904 into sound waves. The speaker may be a traditional film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can not only convert the electrical signal into sound waves audible to humans, but also convert the electrical signal into sound waves inaudible to humans for purposes such as ranging. In some embodiments, the audio circuit 907 may also include a headphone jack.

[0160] The power supply 908 is used to power various components in the computer device 900. The power supply 908 can be an alternating current, a direct current, a disposable battery, or a rechargeable battery. When the power supply 908 includes a rechargeable battery, the rechargeable battery can be a wired rechargeable battery or a wireless rechargeable battery. A wired rechargeable battery is a battery that is charged through a wired line, and a wireless rechargeable battery is a battery that is charged through a wireless coil. The rechargeable battery can also be used to support fast charging technology.

[0161] In some embodiments, the computer device 900 further includes one or more sensors 909 , including but not limited to: an acceleration sensor 910 , a gyroscope sensor 911 , a pressure sensor 912 , an optical sensor 913 , and a proximity sensor 914 .

[0162] The acceleration sensor 910 can detect the magnitude of acceleration on the three coordinate axes of the coordinate system established by the computer device 900. For example, the acceleration sensor 910 can be used to detect the components of gravity acceleration on the three coordinate axes. The processor 901 can control the display screen 905 to display the user interface in a horizontal view or a vertical view according to the gravity acceleration signal collected by the acceleration sensor 910. The acceleration sensor 910 can also be used to collect game or user motion data.

[0163] The gyro sensor 911 can detect the body direction and rotation angle of the computer device 900, and the gyro sensor 911 can cooperate with the acceleration sensor 910 to collect the user's 3D actions on the computer device 900. The processor 901 can implement the following functions based on the data collected by the gyro sensor 911: motion sensing (such as changing the UI according to the user's tilt operation), image stabilization during shooting, game control, and inertial navigation.

[0164] The pressure sensor 912 can be set on the side frame of the computer device 900 and / or the lower layer of the display screen 905. When the pressure sensor 912 is set on the side frame of the computer device 900, it can detect the user's grip signal of the computer device 900, and the processor 901 performs left and right hand recognition or shortcut operations according to the grip signal collected by the pressure sensor 912. When the pressure sensor 912 is set on the lower layer of the display screen 905, the processor 901 controls the operability controls on the UI interface according to the user's pressure operation on the display screen 905. The operability controls include at least one of a button control, a scroll bar control, an icon control, and a menu control.

[0165] The optical sensor 913 is used to collect the ambient light intensity. In one embodiment, the processor 901 can control the display brightness of the display screen 905 according to the ambient light intensity collected by the optical sensor 913. For example, when the ambient light intensity is high, the display brightness of the display screen 905 is increased; when the ambient light intensity is low, the display brightness of the display screen 905 is reduced. In another embodiment, the processor 901 can also dynamically adjust the shooting parameters of the camera assembly 906 according to the ambient light intensity collected by the optical sensor 913.

[0166] The proximity sensor 914, also called a distance sensor, is usually disposed on the front panel of the computer device 900. The proximity sensor 914 is used to collect the distance between the user and the front of the computer device 900. In one embodiment, when the proximity sensor 914 detects that the distance between the user and the front of the computer device 900 is gradually decreasing, the processor 901 controls the display screen 905 to switch from the screen-on state to the screen-off state; when the proximity sensor 914 detects that the distance between the user and the front of the computer device 900 is gradually increasing, the processor 901 controls the display screen 905 to switch from the screen-off state to the screen-on state.

[0167] Those skilled in the art will understand that Fig. 9 The structure shown in the figure does not constitute a limitation on the computer device 900, and the computer device 900 may include more or less components than those shown in the figure, or combine some components, or adopt a different arrangement of components.

[0168] The present application also provides a computer-readable storage medium, in which at least one instruction, at least one program, a code set or an instruction set is stored. The at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by a processor to implement the training method of the cover song recognition model or the cover song recognition method provided by the above method embodiment.

[0169] The present application provides a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the training method of the cover song recognition model or the cover song recognition method provided by the above method embodiment.

[0170] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0171] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0172] The above description is only an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method for training a cover song recognition model, characterized in that: The cover song recognition model includes a first neural network and a second neural network in parallel, and the method includes: Obtaining original audio, cover audio and other audio from the music library, wherein the cover audio is an audio of an original song corresponding to the original audio, and the other audio is an audio of a song track different from the original song; extracting embedded features of the original audio, the cover audio and the other audio through the first neural network; Acquire real song audio, where the real song audio is song audio collected in a real environment scene; extract a first audio feature of the real song audio, and perform a reconstruction operation on the first audio feature through the second neural network to obtain a second audio feature; Based on the similarity between the embedding features of the original audio, the cover audio and the other audio, a triplet loss is constructed; and based on the similarity between the first audio feature and the second audio feature, a reconstruction loss is constructed; The cover song recognition model is trained based on the triplet loss and the reconstruction loss.

2. The method according to claim 1, characterized in that The step of performing a reconstruction operation on the first audio feature by the second neural network to obtain a second audio feature includes: Performing a masking operation on some features of the first audio features to obtain masked audio features; The masked audio feature is input into the second neural network, and the second neural network performs a reconstruction operation on the masked audio feature in a direction close to the first audio feature to obtain the second audio feature.

3. The method according to claim 2, characterized in that The performing a masking operation on some features of the first audio features to obtain masked audio features includes: Randomly select a preset proportion of features in the first audio features to perform the masking operation to obtain the masked audio features; or, The masking operation is performed on the features of the first audio features whose values ​​are maximum values ​​and / or minimum values ​​to obtain the masked audio features.

4. The method according to any one of claims 1 to 3, characterized in that: The real song audio includes a plurality of real song audios, and the reconstruction loss is constructed based on the similarity between the first audio feature and the second audio feature, including: For each real song audio among the plurality of real song audios, calculating a feature gap between the first audio feature and the second audio feature of the real song audio; The reconstruction loss is obtained based on the feature gap of each real song audio in the multiple real song audios.

5. The method according to any one of claims 1 to 3, characterized in that: The extracting, by the first neural network, the embedding features of the original audio, the cover audio, and the other audio includes: Through the first neural network, an embedding operation is performed on the constant Q transform CQT feature of the original audio to obtain the embedded feature of the original audio; through the first neural network, an embedding operation is performed on the CQT feature of the cover audio to obtain the embedded feature of the cover audio; through the first neural network, an embedding operation is performed on the CQT feature of the other audio to obtain the embedded feature of the other audio.

6. The method according to any one of claims 1 to 3, characterized in that: The extracting the embedding features of the original audio, the cover audio, and the other audio includes: Slicing the original audio and the cover audio to obtain a plurality of original audio segments and a plurality of cover audio segments, respectively, wherein the original audio segments and the cover audio segments contain the same lyrics; and slicing the other audio to obtain a plurality of other audio segments; The embedding features of the multiple original singing segments are extracted as the embedding features of the original singing audio; the embedding features of the multiple cover singing segments are extracted as the embedding features of the cover singing audio; and the embedding features of the multiple other audio segments are extracted as the embedding features of the other audio.

7. The method according to any one of claims 1 to 3, characterized in that: The step of training the cover song recognition model based on the triplet loss and the reconstruction loss comprises: Performing a weighted sum operation based on the triplet loss and the reconstruction loss to obtain a fusion loss; Based on the fusion loss, the cover song recognition model is trained.

8. A method for identifying a cover song, characterized in that: The method comprises: Acquire multiple original singer audios in a music library, and extract the embedding features of each of the original singer audios through a pre-trained first neural network to obtain the original singer embedding features; wherein the first neural network is trained by the method described in any one of claims 1 to 7; Acquire the cover audio to be retrieved, and extract the embedded features of the cover audio to be retrieved through the first neural network to obtain the embedded features to be retrieved; Based on the feature distance between the original singing embedded feature of each of the original singing audios and the embedded feature to be retrieved, the cover singing recognition result of the cover singing audio to be retrieved is determined in the multiple original singing audios.

9. The method according to claim 8, characterized in that The extracting the embedding features of each original singer audio by the pre-trained first neural network to obtain the original singer embedding features includes: Perform a segmentation operation on each of the original singing audios to obtain a plurality of original singing audio segments of each of the original singing audios; The embedded features of multiple original audio segments of each original audio are extracted through the pre-trained first neural network, and are used as the original embedded features of each original audio.

10. A computer device, characterized in that: The computer device comprises: a processor and a memory, the memory stores a computer program, and the computer program is loaded and executed by the processor to implement the training method of the cover song recognition model according to any one of claims 1 to 7, or the cover song recognition method according to claim 8 or 9.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is loaded and executed by a processor to implement the training method for a cover song recognition model according to any one of claims 1 to 7, or the cover song recognition method according to claim 8 or 9.

12. A computer program product, characterized in that The computer program product stores a computer program, and the computer program is loaded and executed by a processor to implement the training method for a cover song recognition model according to any one of claims 1 to 7, or the cover song recognition method according to claim 8 or 9.

Images