Cover song recognition model training methods, cover song recognition methods, equipment and storage media

Abstract

This application relates to a method for training a cover song recognition model, a cover song recognition method, a computer device, and a storage medium. The method for training the cover song recognition model includes: acquiring the original song and the cover song corresponding to the sample music, and extracting the original song features corresponding to the original song and the cover song features corresponding to the cover song; mixing the cover song features and the original song features to obtain mixed song features; inputting the mixed song features into the encoder of the cover song recognition model to be trained, obtaining the melody features corresponding to the mixed song features through the encoder, and inputting the melody features into the decoder of the cover song recognition model, obtaining the reconstructed song features through the decoder; training the cover song recognition model based on the differences between the reconstructed song features and the cover song features to obtain the trained cover song recognition model. This method can improve the cover song recognition accuracy of the trained cover song recognition model.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a method for training a cover song recognition model, a cover song recognition method, a computer device, and a storage medium. Background Technology

[0002] With the development of artificial intelligence technology, a technology has emerged that uses artificial intelligence networks to identify cover songs. This technology pre-trains a cover song recognition model, which extracts melody information from the song audio and identifies the cover song based on the similarity of the melody.

[0003] In traditional technologies, cover song recognition models are usually trained through supervised learning. By inputting spectral features, the cover song recognition model learns the overall similarity of songs with the same melody and the differences between individual songs with different melody, thus completing the training of the cover song recognition model.

[0004] However, current supervised learning methods for training cover song recognition models are susceptible to the influence of different songs because they need to learn the differences between different melodies. The trained model is also somewhat dependent on the number of music categories in the training set, resulting in a low accuracy rate for cover song recognition. Summary of the Invention

[0005] Therefore, it is necessary to provide a method for training a cover song recognition model, a cover song recognition method, a device, and a storage medium to address the aforementioned technical problems.

[0006] Firstly, this application provides a method for training a cover song recognition model, the method comprising:

[0007] Obtain the original song and cover song corresponding to the sample music, and extract the original song features corresponding to the original song and the cover song features corresponding to the cover song.

[0008] The features of the cover song are mixed with the features of the original song to obtain mixed song features;

[0009] The mixed song features are input into the encoder of the cover song recognition model to be trained, and the encoder obtains the melody features corresponding to the mixed song features. The melody features are then input into the decoder of the cover song recognition model, and the decoder obtains the reconstructed song features.

[0010] Based on the differences between the reconstructed song features and the cover song features, the cover song recognition model is trained to obtain the trained cover song recognition model.

[0011] In one embodiment, extracting the original song features corresponding to the original song and the cover song features corresponding to the cover song includes: performing a Fourier transform on a first target song to obtain a spectrum map corresponding to the first target song, and obtaining frequency peak points corresponding to the first target song from the spectrum map; the first target song includes the original song and the cover song; obtaining a spectrum peak sequence corresponding to the frequency peak points based on the frequency peak points, and using the spectrum peak sequence as the song features corresponding to the first target song; if the first target song is the original song, then the song features corresponding to the first target song are used as the original song features corresponding to the original song; if the first target song is the cover song, then the song features corresponding to the first target song are used as the cover song features corresponding to the cover song.

[0012] In one embodiment, obtaining the spectral peak sequence corresponding to the frequency peak points includes: acquiring the frequency value corresponding to each frequency peak point; obtaining the frequency mean and frequency variance corresponding to the spectrum based on the frequency value; normalizing the frequency value corresponding to each frequency peak point using the frequency mean and frequency variance to obtain normalized frequency peak points; and constructing the spectral peak sequence using the normalized frequency peak points.

[0013] In one embodiment, the step of mixing the cover song features with the original song features to obtain mixed song features includes: obtaining a pre-set perturbation parameter; obtaining a first feature weight of the cover song features and a second feature weight of the original song features based on the perturbation parameter; and using the first feature weight and the second feature weight to perform weighted processing on the cover song features and the original song features to obtain the mixed song features.

[0014] In one embodiment, extracting the original song features corresponding to the original song and the cover song features corresponding to the cover song includes: performing a constant Q-transform on the second target song to obtain the constant Q-transform features corresponding to the second target song; the second target song includes the original song and the cover song; if the second target song is the original song, then the constant Q-transform features corresponding to the second target song are used as the original song features corresponding to the original song; if the second target song is the cover song, then the constant Q-transform features corresponding to the second target song are used as the cover song features corresponding to the cover song.

[0015] In one embodiment, after obtaining the trained cover song recognition model, the method further includes: obtaining candidate song features corresponding to candidate songs; inputting the candidate song features into the encoder of the trained cover song recognition model, and obtaining candidate melody features corresponding to the candidate song features through the encoder; constructing a correspondence between the music corresponding to the candidate song and the candidate melody features, and using the correspondence to construct a melody feature library.

[0016] Secondly, this application also provides a cover song recognition method, the method comprising:

[0017] Obtain the cover song to be identified, and the song features corresponding to the cover song to be identified;

[0018] The song features corresponding to the cover song to be identified are input into the encoder of the trained cover song recognition model, and the melody features of the cover song to be identified are obtained through the encoder; the cover song recognition model is trained by the cover song recognition model training method as described in any embodiment of the first aspect.

[0019] Based on the melodic features, the music corresponding to the cover song to be identified is determined.

[0020] In one embodiment, identifying the song category corresponding to the cover song to be identified based on the melody features includes: inputting the melody features into a pre-constructed melody feature library; obtaining the target melody feature with the highest similarity to the melody features from the candidate melody features stored in the melody feature library; the melody feature library stores the correspondence between each candidate melody feature and music; and using the music corresponding to the target melody feature as the music corresponding to the cover song to be identified.

[0021] Thirdly, this application also provides a cover song recognition model training device, the device comprising:

[0022] The song feature acquisition module is used to acquire the original song and cover song corresponding to the sample music, and extract the original song features corresponding to the original song and the cover song features corresponding to the cover song.

[0023] The song feature mixing module is used to mix the features of the cover song with the features of the original song to obtain mixed song features;

[0024] The song feature reconstruction module is used to input the mixed song features into the encoder of the cover song recognition model to be trained, obtain the melody features corresponding to the mixed song features through the encoder, and input the melody features into the decoder of the cover song recognition model, and obtain the reconstructed song features through the decoder.

[0025] The recognition model training module is used to train the cover song recognition model based on the differences between the reconstructed song features and the cover song features, so as to obtain the trained cover song recognition model.

[0026] Fourthly, this application also provides a cover song recognition device, the device comprising:

[0027] The module for acquiring songs to be identified is used to acquire the cover songs to be identified, as well as the song features corresponding to the cover songs to be identified.

[0028] The melody feature extraction module is used to input the song features corresponding to the cover song to be identified into the encoder of the trained cover song recognition model, and obtain the melody features of the cover song to be identified through the encoder; the cover song recognition model is trained by the cover song recognition model training method as described in any embodiment of the first aspect;

[0029] The cover song recognition module is used to identify the music corresponding to the cover song to be identified based on the melodic features.

[0030] Fifthly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to perform the following steps:

[0031] Obtain the original song and cover song corresponding to the sample music, and extract the original song features corresponding to the original song and the cover song features corresponding to the cover song.

[0032] The features of the cover song are mixed with the features of the original song to obtain mixed song features;

[0033] The mixed song features are input into the encoder of the cover song recognition model to be trained, and the encoder obtains the melody features corresponding to the mixed song features. The melody features are then input into the decoder of the cover song recognition model, and the decoder obtains the reconstructed song features.

[0034] Based on the differences between the reconstructed song features and the cover song features, the cover song recognition model is trained to obtain the trained cover song recognition model.

[0035] When the processor executes the computer program, it also performs the following steps:

[0036] Obtain the cover song to be identified, and the song features corresponding to the cover song to be identified;

[0037] The song features are input into the encoder of the trained cover song recognition model, and the melody features of the cover song to be recognized are obtained through the encoder; the cover song recognition model is trained by the cover song recognition model training method as described in any embodiment of the first aspect;

[0038] Based on the melodic features, the music corresponding to the cover song to be identified is determined.

[0039] Sixthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, performs the following steps:

[0040] Obtain the original song and cover song corresponding to the sample music, and extract the original song features corresponding to the original song and the cover song features corresponding to the cover song.

[0041] The features of the cover song are mixed with the features of the original song to obtain mixed song features;

[0042] The mixed song features are input into the encoder of the cover song recognition model to be trained, and the encoder obtains the melody features corresponding to the mixed song features. The melody features are then input into the decoder of the cover song recognition model, and the decoder obtains the reconstructed song features.

[0043] Based on the differences between the reconstructed song features and the cover song features, the cover song recognition model is trained to obtain the trained cover song recognition model.

[0044] When the computer program is executed by the processor, it also performs the following steps:

[0045] Obtain the cover song to be identified, and the song features corresponding to the cover song to be identified;

[0046] The song features are input into the encoder of the trained cover song recognition model, and the melody features of the cover song to be recognized are obtained through the encoder; the cover song recognition model is trained by the cover song recognition model training method as described in any embodiment of the first aspect;

[0047] Based on the melodic features, the music corresponding to the cover song to be identified is determined.

[0048] Seventhly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, performs the following steps:

[0049] Obtain the original song and cover song corresponding to the sample music, and extract the original song features corresponding to the original song and the cover song features corresponding to the cover song.

[0050] The features of the cover song are mixed with the features of the original song to obtain mixed song features;

[0051] The mixed song features are input into the encoder of the cover song recognition model to be trained, and the encoder obtains the melody features corresponding to the mixed song features. The melody features are then input into the decoder of the cover song recognition model, and the decoder obtains the reconstructed song features.

[0052] Based on the differences between the reconstructed song features and the cover song features, the cover song recognition model is trained to obtain the trained cover song recognition model.

[0053] When the computer program is executed by the processor, it also performs the following steps:

[0054] Obtain the cover song to be identified, and the song features corresponding to the cover song to be identified;

[0055] The song features are input into the encoder of the trained cover song recognition model, and the melody features of the cover song to be recognized are obtained through the encoder; the cover song recognition model is trained by the cover song recognition model training method as described in any embodiment of the first aspect;

[0056] Based on the melodic features, the music corresponding to the cover song to be identified is determined.

[0057] The aforementioned method, apparatus, computer equipment, storage medium, and computer program product for training a cover song recognition model involve acquiring the original song and cover song corresponding to sample music, extracting the original song features corresponding to the original song, and the cover song features corresponding to the cover song; mixing the cover song features with the original song features to obtain mixed song features; inputting the mixed song features into the encoder of the cover song recognition model to be trained, obtaining the melody features corresponding to the mixed song features through the encoder, and inputting the melody features into the decoder of the cover song recognition model, obtaining the reconstructed song features through the decoder; and training the cover song recognition model based on the differences between the reconstructed song features and the cover song features to obtain the trained cover song recognition model. This application obtains the original song features corresponding to the original version of the sample music, as well as the cover song features corresponding to the cover version of the sample music. The cover song features and original song features are then mixed. The encoder of the cover song recognition model obtains the melody features corresponding to the mixed features. The decoder of the cover song recognition model then reconstructs the melody features. After obtaining the reconstructed song features, the cover song recognition model is trained based on the differences between the reconstructed song features and the cover song features. The cover song recognition model trained in this way can learn the commonalities of the melody between the original and cover songs. Therefore, it does not need to learn the differences between different melodies and is not affected by different songs. The trained model does not depend on the music category in the training set, thereby improving the cover song recognition accuracy of the trained cover song recognition model. Attached Figure Description

[0058] Figure 1 This is a flowchart illustrating the training method for a cover song recognition model in one embodiment;

[0059] Figure 2 This is a flowchart illustrating the process of extracting features from the original song and the cover song in one embodiment;

[0060] Figure 3 This is a flowchart illustrating the process of obtaining the spectral peak sequence corresponding to the frequency peak point in one embodiment.

[0061] Figure 4 This is a flowchart illustrating the process of constructing a melody feature library in one embodiment;

[0062] Figure 5 This is a flowchart illustrating a cover song recognition method in one embodiment;

[0063] Figure 6 This is a flowchart illustrating a cover song recognition method for extracting melody features in one embodiment;

[0064] Figure 7 This is a spectrum diagram of a song obtained by performing a Fourier transform on the song's audio signal in one embodiment;

[0065] Figure 8 This is a schematic diagram of peak points extracted from a spectrum in one embodiment;

[0066] Figure 9 This is a schematic diagram of the structure of a convolutional autoencoder network in one embodiment;

[0067] Figure 10 This is a structural block diagram of a cover song recognition model training device in one embodiment;

[0068] Figure 11 This is a structural block diagram of a cover song recognition device in one embodiment;

[0069] Figure 12 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0070] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0071] In one embodiment, such as Figure 1 As shown, a method for training a cover song recognition model is provided. This embodiment illustrates the application of this method to a terminal. It is understood that this method can also be applied to a server, and further to a system including both a terminal and a server, and implemented through interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0072] Step S101: Obtain the original song and cover song corresponding to the sample music, and extract the original song features corresponding to the original song and the cover song features corresponding to the cover song.

[0073] Here, "sample music" refers to the music samples used to train the cover song recognition model; "original song" refers to the original version of the sample music; and "cover song" refers to the cover version of the sample music. For a given sample music, there is one original version and multiple cover versions. "Original song features" refers to the song features corresponding to the original song, which can be the spectral characteristics of the original song. Similarly, "cover song features" refers to the song features corresponding to the cover song, which can also be the spectral characteristics of the cover song.

[0074] Specifically, after identifying a sample music, the terminal can collect the original song corresponding to that sample music, as well as multiple cover versions of that sample music. Then, it can extract song features from the original song and the cover songs to obtain the features of the original song and the cover songs.

[0075] Step S102: Mix the features of the cover song with the features of the original song to obtain mixed song features.

[0076] Hybrid song features refer to the song features obtained by mixing the features of the original song and the features of the cover song. After the terminal obtains the original song features of the original song and the cover song features corresponding to each cover song, it can mix the original song features and each cover song feature to obtain the hybrid song features corresponding to each cover song feature.

[0077] Step S103: Input the mixed song features into the encoder of the cover song recognition model to be trained, obtain the melody features corresponding to the mixed song features through the encoder, and input the melody features into the decoder of the cover song recognition model, obtain the reconstructed song features through the decoder.

[0078] The cover song recognition model to be trained refers to the model that needs to be trained. This model can include an encoder and a decoder. The encoder extracts the melody features of the music, which represent the common features of the sample music. By inputting the mixed song features into the encoder, the encoder can output the corresponding melody features. The decoder is used to reconstruct the song features from the melody features. By inputting the melody features output by the encoder into the decoder, the decoder reconstructs the song features from the input melody features, thus obtaining the reconstructed song features. Specifically, after obtaining the mixed song features corresponding to each cover song, the terminal can input each mixed song feature into the encoder of the cover song recognition model. The encoder outputs the melody features corresponding to each mixed song feature. Then, each melody feature can be input into the encoder, which reconstructs each melody feature, thus obtaining the reconstructed song features corresponding to each melody feature.

[0079] Step S104: Based on the differences between the reconstructed song features and the cover song features, the cover song recognition model is trained to obtain the trained cover song recognition model.

[0080] Finally, after obtaining the features of each reconstructed song, the loss value of the cover song recognition model can be calculated based on the difference between each reconstructed song feature and the corresponding cover song feature. This allows for the training of the cover song recognition model and the updating of its model parameters until the training termination condition is met. For example, the model parameters of the cover song recognition model can be fixed when the loss value converges to a certain loss threshold, thus obtaining the trained cover song recognition model.

[0081] The above-mentioned method for training a cover song recognition model involves acquiring the original song and the cover song corresponding to the sample music, extracting the original song features corresponding to the original song, and the cover song features corresponding to the cover song; mixing the cover song features with the original song features to obtain mixed song features; inputting the mixed song features into the encoder of the cover song recognition model to be trained, obtaining the melody features corresponding to the mixed song features through the encoder, and inputting the melody features into the decoder of the cover song recognition model, obtaining the reconstructed song features through the decoder; and training the cover song recognition model based on the differences between the reconstructed song features and the cover song features to obtain the trained cover song recognition model. This application obtains the original song features corresponding to the original version of the sample music, as well as the cover song features corresponding to the cover version of the sample music. The cover song features and original song features are then mixed. The encoder of the cover song recognition model obtains the melody features corresponding to the mixed features. The decoder of the cover song recognition model then reconstructs the melody features. After obtaining the reconstructed song features, the cover song recognition model is trained based on the differences between the reconstructed song features and the cover song features. The cover song recognition model trained in this way can learn the commonalities of the melody between the original and cover songs. Therefore, it does not need to learn the differences between different melodies and is not affected by different songs. The trained model does not depend on the music category in the training set, thereby improving the cover song recognition accuracy of the trained cover song recognition model.

[0082] In one embodiment, such as Figure 2 As shown, step S101 may further include:

[0083] Step S201: Perform Fourier transform processing on the first target song to obtain the spectrum diagram corresponding to the first target song, and obtain the frequency peak point corresponding to the first target song from the spectrum diagram; the first target song includes the original song and the cover song.

[0084] The first target song can refer to the original song corresponding to the sample music, or it can refer to the cover song corresponding to the sample music. In this embodiment, the terminal can take the original song corresponding to the sample music or any cover song of the sample music as the first target song, and perform Fourier transform processing on the first target song to obtain the spectrum of the first target song. Then, the frequency peak points corresponding to the first target song can be identified from the spectrum. The number of peak points can be multiple, corresponding to different playback times of the first target song.

[0085] Step S202: Based on the frequency peak points, obtain the spectrum peak sequence corresponding to the frequency peak points, and use the spectrum peak sequence as the song feature corresponding to the first target song.

[0086] A spectral peak sequence is a sequence formed by multiple spectral peak points. This sequence can be represented as a vector and used as the song feature vector corresponding to the first target song. Specifically, after obtaining multiple frequency peak points corresponding to the first target song, the terminal can use these frequency peak points to form a spectral peak sequence and use the spectral peak sequence as the song feature corresponding to the first target song.

[0087] Step S203: If the first target song is the original song, then the song features corresponding to the first target song are used as the original song features corresponding to the original song.

[0088] Step S204: If the first target song is a cover song, then the song features corresponding to the first target song are used as the cover song features corresponding to the cover song.

[0089] If the first target song is the original song, then its corresponding song features, namely the spectrum peak sequence of the original song, can be used as the original song features corresponding to the original song. If the first target song is any cover song, then its corresponding song features, namely the spectrum peak sequence of any cover song, can be used as the cover song features corresponding to that cover song.

[0090] In this embodiment, the terminal can obtain the spectral peak sequence corresponding to the original song or the cover song, and use the spectral peak sequence as the corresponding song feature of the original song or the cover song. By using the spectral peak sequence as the song feature, it is easier to completely separate information that is not related to the melody, thereby improving the accuracy of the melody feature obtained by the encoder.

[0091] Furthermore, such as Figure 3 As shown, step S202 may further include:

[0092] Step S301: Obtain the frequency value corresponding to each frequency peak point, and obtain the frequency mean and frequency variance corresponding to the spectrum graph based on the frequency value.

[0093] The frequency mean refers to the average of multiple frequency values ​​corresponding to each frequency peak point in the spectrum, while the frequency variance is the variance of the frequency values ​​corresponding to each frequency peak point. In this embodiment, after obtaining multiple frequency peak points of the spectrum corresponding to the first target song, the terminal can also obtain the frequency value corresponding to each frequency peak point, thereby calculating the frequency mean and frequency variance of the spectrum.

[0094] Step S302: Using the frequency mean and frequency variance, normalize the frequency values ​​corresponding to each frequency peak point to obtain the normalized frequency peak points.

[0095] Step S303: Construct a spectral peak sequence using normalized frequency peak points.

[0096] After obtaining the frequency mean and frequency variance in step S301, the frequency value corresponding to each frequency peak point can be normalized using the frequency mean and frequency variance. After obtaining the normalized frequency peak points, the frequency peak points can be used to construct the spectrum peak sequence.

[0097] For example, the normalized frequency peak point can be obtained using the following formula:

[0098]

[0099] Where x represents the frequency value corresponding to the frequency peak point, x represents the frequency mean, σ represents the frequency mean, and x′ represents the frequency value corresponding to the normalized frequency peak point.

[0100] In this embodiment, the process of obtaining the spectral peak sequence can be achieved by first normalizing each frequency peak point, and then using the normalized frequency peak points to construct the spectral peak sequence. The spectral peak sequence constructed in this way is more conducive to the training of the cover song recognition model, thereby further improving the recognition accuracy of the trained cover song recognition model.

[0101] In one embodiment, step S102 may further include: obtaining a pre-set perturbation parameter; obtaining a first feature weight of the cover song feature and a second feature weight of the original song feature based on the perturbation parameter; and using the first feature weight and the second feature weight to perform weighted processing on the cover song feature and the original song feature to obtain a mixed song feature.

[0102] The perturbation parameter refers to the pre-set fusion parameter used to fuse the features of the cover song and the original song. This fusion process can characterize the perturbation of song information to improve the model's noise resistance. This fusion method can be obtained by weighting the features of the cover song and the original song. The first feature weight refers to the weighted weight of the cover song features, while the second feature weight refers to the weighted weight of the original song features. These first and second feature weights can be determined based on the perturbation parameter. For example, if the perturbation parameter can be 'a', and 'a < 0.1, then the first feature weight can be set to 'a', and the second feature weight can be set to '1 - a'. Therefore, the mixed song features can be calculated using the following formula:

[0103] x mix =a*x cover +(1-a)*x raw

[0104] Where, x mix Indicates mixed song features, x cover This represents any characteristic of a cover song, while x raw This indicates the characteristics of the original song.

[0105] In this embodiment, by setting perturbation parameters, feature weights of cover song features and original song features are obtained based on the perturbation parameters, so as to realize the proportional mixing of cover song features and original song features, thereby simulating the perturbation of song information and further improving the noise resistance performance of the trained cover song recognition model.

[0106] In one embodiment, step S101 may further include: performing a constant Q-transform on the second target song to obtain the constant Q-transform feature corresponding to the second target song; the second target song includes the original song and the cover song; if the second target song is the original song, then the constant Q-transform feature corresponding to the second target song is used as the feature of the original song corresponding to the original song; if the second target song is the cover song, then the constant Q-transform feature corresponding to the second target song is used as the feature of the cover song corresponding to the cover song.

[0107] In this embodiment, the song features input to the encoder can also be constant Q transform features, i.e., CQT features. Similar to the first target song, the second target song can also refer to the original song corresponding to the sample music, or any cover song corresponding to the sample music.

[0108] Specifically, after obtaining the original and cover versions of the sample music, the terminal can use either the original or any cover version as the second target song. Then, it can perform a constant Q-transform on the second target song to obtain its corresponding constant Q-transform features. If the second target song is the original, the obtained constant Q-transform features can be used as the original song's features; if the second target song is any cover version, the obtained constant Q-transform features can be used as the cover song's features.

[0109] In this embodiment, the terminal can also use the constant Q-transform feature of the original song as the feature of the original song, and the constant Q-transform feature of the cover song as the feature of the cover song, thereby improving the feature diversity of the original song features and the cover song features, and further improving the diversity of the trained cover song recognition model.

[0110] In one embodiment, such as Figure 4 As shown, after step S104, the following may also be included:

[0111] Step S401: Obtain the candidate song features corresponding to the candidate songs.

[0112] Here, a candidate song refers to any musical song used to construct the melody feature library, while the candidate song feature refers to the song characteristics corresponding to that candidate song, which can refer to the spectral peak sequence corresponding to the candidate song. Specifically, in the process of constructing the melody feature library, candidate songs used to construct the melody feature library can be obtained first, and the song characteristics corresponding to those candidate songs can be extracted.

[0113] Step S402: Input the candidate song features into the encoder of the trained cover song recognition model, and obtain the candidate melody features corresponding to the candidate song features through the encoder.

[0114] The candidate melody features are obtained through the encoder of the cover song recognition model, specifically the melody features corresponding to the candidate song features. After training the cover song recognition model, the terminal can input the extracted candidate song features into the encoder of the cover song recognition model, and the encoder outputs the melody features corresponding to the candidate song features, thus obtaining the candidate melody features.

[0115] Step S403: Construct the correspondence between the music and melody features of the candidate songs, and use the correspondence to construct a melody feature library.

[0116] A melody feature library can be used to identify the music corresponding to a song. For example, if a user covers a song, resulting in a cover version, the melody feature library can be used to identify the music corresponding to that cover. Finally, the terminal can construct the correspondence between the music corresponding to candidate songs and the candidate melody features, and then use this correspondence to construct the melody feature library. For example, if candidate song A is a cover version of music A, after obtaining the melody features of candidate song A through the encoder of the trained cover recognition model, the correspondence between these melody features and music A can be constructed, and thus the melody feature library can be built using this correspondence.

[0117] In this embodiment, after training the cover song recognition model, the encoder of the cover song recognition model can be used to extract the melody features of the candidate songs, thereby constructing the correspondence between the melody features and the music corresponding to the candidate songs, and then constructing a melody feature library for recognizing the music corresponding to the cover songs, which further improves the accuracy of cover song recognition.

[0118] In one embodiment, such as Figure 5 As shown, a cover song recognition training method is also provided. This embodiment illustrates the application of this method to a terminal. It is understood that this method can also be applied to a server, and can also be applied to a system including a terminal and a server, and implemented through the interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0119] Step S501: Obtain the cover song to be identified and the song features corresponding to the cover song to be identified.

[0120] The cover song to be identified refers to the cover song whose corresponding music needs to be identified. In this embodiment, if it is necessary to identify the music corresponding to a certain cover song, the cover song can be used as the music to be identified, and the song features of the cover song to be identified can be further extracted. For example, the spectral peak sequence corresponding to the cover song to be identified can be used as the song features corresponding to the cover song to be identified.

[0121] Step S502: Input the song features corresponding to the cover song to be identified into the encoder of the trained cover song recognition model, and obtain the melody features of the cover song to be identified through the encoder; the cover song recognition model is trained by the cover song recognition model training method of any of the above embodiments.

[0122] Step S503: Identify the music corresponding to the cover song to be identified based on the melody characteristics.

[0123] After obtaining the song features corresponding to the cover song to be identified, the terminal can input these features into the encoder of the trained cover song recognition model. The encoder then outputs the melody features of the cover song to be identified. This cover song recognition model can be trained using the original song features of the sample music and the cover song features corresponding to the sample music. By mixing the original and cover song features, a mixed song feature is obtained and input into the encoder of the cover song recognition model. The encoder obtains the melody features, which are then reconstructed by the decoder of the cover song recognition model. The model is trained using the loss between the reconstructed melody features and the cover song features. Because this method of training does not require learning the differences between different melodies and is not affected by different songs, the trained model is independent of the music categories in the training set. Therefore, the encoder of the trained cover song recognition model can obtain more accurate melody features. Furthermore, since these melody features can be used to identify the music corresponding to the cover song to be identified, the accuracy of identifying the corresponding music can be improved.

[0124] The aforementioned cover song recognition method involves acquiring the cover song to be recognized and its corresponding song features; inputting these features into the encoder of a trained cover song recognition model to obtain the melody features; training the cover song recognition model using the method described in any of the above embodiments; and identifying the music corresponding to the cover song based on the melody features. This application utilizes the encoder of the trained cover song recognition model to extract the melody features of the cover song to be recognized, thereby using these melody features to identify the music corresponding to the cover song. Since the trained cover song recognition model can learn the commonalities in melody between the original and cover songs, it does not need to learn the differences between songs with different melodies, nor is it affected by different songs. The trained model does not depend on the music category in the training set. Therefore, the encoder of the trained cover song recognition model can obtain more accurate melody features, and since these melody features can be used to identify the music corresponding to the cover song, the accuracy of identifying the music corresponding to the cover song can be improved.

[0125] In one embodiment, step S503 may further include: inputting the melody feature into a pre-built melody feature library, obtaining the target melody feature with the highest similarity to the melody feature from the candidate melody features stored in the melody feature library; the melody feature library stores the correspondence between each candidate melody feature and music; and using the music corresponding to the target melody feature as the music corresponding to the cover song to be identified.

[0126] The melody feature library is pre-built and stores various melody features, i.e., a feature library that corresponds to candidate melody features and music. The target melody feature refers to the candidate melody feature among the aforementioned candidate melody features that has the highest similarity to the melody feature corresponding to the cover song to be identified. Specifically, after the terminal obtains the melody feature corresponding to the cover song to be identified through the encoder of the cover song recognition model, it can input the melody feature into the pre-built melody feature library. From the candidate melody features stored in the melody feature library, the candidate melody feature with the highest similarity is selected as the target melody feature. Then, based on the correspondence between the candidate melody feature and music, the music corresponding to the target melody feature can be used as the music corresponding to the cover song to be identified.

[0127] For example, the melody feature database stores the correspondence between melody feature A and music A, melody feature B and music B, and melody feature C and music C. After the terminal obtains the melody features of the cover song to be identified, it can obtain the melody feature with the highest similarity from melody feature A, melody feature B and melody feature C. If melody feature B is the melody feature with the highest similarity, then melody feature B can be used as the target melody feature, that is, music B can be used as the music corresponding to the cover song to be identified.

[0128] In this embodiment, by pre-constructing a melody feature library that stores the correspondence between each candidate melody feature and the music, the target melody feature with the greatest similarity to the melody feature of the cover song to be identified can be determined through the melody feature library. Then, the music corresponding to the target melody feature is used as the music corresponding to the cover song to be identified. This method can improve the efficiency of identifying the cover song to be identified.

[0129] In one embodiment, a method for cover song recognition based on melody feature extraction using an autoencoder network is also provided. First, a training set is constructed, which can consist of one original song and multiple cover versions of that original song. Then, spectral peak sequences are extracted from these songs, and the spectral peak sequence features of the original and cover versions are proportionally mixed as augmented data. This augmented data is used for network training to improve the network's noise resistance. The original data and augmented data are then input into the encoder of the convolutional autoencoder network. The encoder's learning task is to compress song features, and the decoder's learning task is to reconstruct cover song features from the compressed features. The network learns to compress the original song features and reconstruct the cover song features. After training the network until the loss function converges, the network parameters are saved. The song features are then input into the encoder, and the melody features are obtained at the encoder output for database construction, retrieval, and recognition. Figure 6 As shown, this embodiment may specifically include the following steps:

[0130] Step 1: Obtain the song training dataset, which includes one original song and multiple cover versions of the original song.

[0131] Step 2: Extract the song features. These features can be spectral peak sequences or CQT features (constant Q transform features). This embodiment uses spectral peak sequences as an example and may include the following steps:

[0132] (1) Perform a Fourier transform on the song's audio signal to obtain a spectrogram, which can be shown as follows: Figure 7 As shown.

[0133] (2) Extract the peak points from the spectrum. The peak points can be obtained as follows: Figure 8 As shown, the resulting peak point sequence is x = (x1, x2, ..., xn).

[0134] (3) Since the melody features are contained in the differences between different peak points, and to make the network easier to train, the peak sequence features are normalized. The normalization formula is:

[0135]

[0136] Where x represents the frequency value corresponding to the frequency peak point, x represents the frequency mean, σ represents the frequency mean, and x′ represents the frequency value corresponding to the normalized frequency peak point.

[0137] Step 3: Mix the peak sequences. Proportionally mix the peak sequences of the original song and the cover song. This is equivalent to appropriately perturbing the song information, which can improve the model's noise resistance. Alternatively, this process can be achieved by adding random noise to enhance the sample perturbation. Taking the mixing of peak sequences as an example, the following formula can be used to achieve the mixing of peak sequences:

[0138] x mix =a*x cover +(1-a)*x raw

[0139] Where a is the disturbance coefficient (a < 0.1), x mix Indicates mixed song features, x cover This represents any characteristic of a cover song, while x raw This indicates the characteristics of the original song.

[0140] Step 4: Train the neural network model. This neural network model can be a convolutional autoencoder network or an autoencoder network with other structures, such as a fully connected autoencoder network. This embodiment uses a convolutional autoencoder network as an example. A schematic diagram of this network is shown below. Figure 9 As shown.

[0141] The network inputs a mixed spectral peak sequence into an encoder, which consists of convolutional layers to compress the mixed peak sequence features. The decoder, composed of deconvolutional layers, is responsible for reconstructing the compressed features back into the spectral peak sequence of the cover song. The network uses a mean squared error loss function as shown below:

[0142]

[0143] Where n represents the number of cover songs, Y i This represents the characteristics of the i-th cover song. This represents the i-th reconstructed song feature, and the training objective is to reconstruct the feature. and input features Y i As close as possible. This training method is less restricted by songs compared to supervised learning; it focuses on the process of reconstructing cover songs from all original songs through melodic similarity.

[0144] Step 5: After training until the loss function converges to a pre-set threshold, save the network parameters.

[0145] Step 6: Input the song's spectral peak sequence into the trained encoder. Melodic features can be obtained at the encoder output, which can be used for database construction and retrieval recognition.

[0146] This embodiment proposes a cover song recognition method based on autoencoder networks to extract melody features. Compared with supervised cover song recognition methods, this embodiment learns the commonalities in melody between the original and cover songs, unaffected by the differences between different songs. The trained model has stronger feature extraction capabilities and noise resistance. Mainstream supervised learning models often incorporate cross-entropy loss functions during training, thus the trained model is somewhat dependent on the number of song categories in the training set. However, the cover song recognition model provided in this embodiment does not have this category dependency problem. Furthermore, this embodiment trains by mixing peak sequence features, allowing the model to avoid interference from other irrelevant information during the input stage and focus more on melody characteristics.

[0147] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0148] Based on the same inventive concept, this application also provides a cover song recognition model training device for implementing the cover song recognition model training method described above. The solution provided by this device is similar to the implementation described in the above method; therefore, the specific limitations in one or more embodiments of the cover song recognition model training device provided below can be found in the limitations of the cover song recognition model training method described above, and will not be repeated here.

[0149] In one embodiment, such as Figure 10 As shown, a cover song recognition model training device is provided, including: a song feature acquisition module 1001, a song feature mixing module 1002, a song feature reconstruction module 1003, and a recognition model training module 1004, wherein:

[0150] The song feature acquisition module 1001 is used to acquire the original song and cover song corresponding to the sample music, and extract the original song features corresponding to the original song and the cover song features corresponding to the cover song.

[0151] The song feature mixing module 1002 is used to mix the features of the cover song with the features of the original song to obtain mixed song features;

[0152] The song feature reconstruction module 1003 is used to input the mixed song features into the encoder of the cover song recognition model to be trained, obtain the melody features corresponding to the mixed song features through the encoder, and input the melody features into the decoder of the cover song recognition model, and obtain the reconstructed song features through the decoder.

[0153] The recognition model training module 1004 is used to train the cover song recognition model based on the differences between the features of the reconstructed song and the features of the cover song, so as to obtain the trained cover song recognition model.

[0154] In addition, this application also provides a cover song recognition device for implementing the cover song recognition method described above. The solution provided by this device is similar to the solution described in the above method. Therefore, the specific limitations of one or more cover song recognition device embodiments provided below can be found in the limitations of the cover song recognition method above, and will not be repeated here.

[0155] In one embodiment, such as Figure 11 As shown, a cover song recognition device is provided, including: a song acquisition module 1101, a melody feature extraction module 1102, and a cover song recognition module 1103, wherein:

[0156] The song acquisition module 1101 is used to acquire the cover song to be identified and the song features corresponding to the cover song to be identified.

[0157] The melody feature extraction module 1102 is used to input the song features corresponding to the cover song to be identified into the encoder of the trained cover song recognition model, and obtain the melody features of the cover song to be identified through the encoder; the cover song recognition model is trained by the cover song recognition model training method of any of the above embodiments.

[0158] The cover song recognition module 1103 is used to identify the music corresponding to the cover song to be identified based on the melody characteristics.

[0159] The aforementioned cover song recognition model training device, or each module in the cover song recognition device, can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0160] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 12As shown, the computer device includes a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a cover song recognition model training method or a cover song recognition method. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0161] Those skilled in the art will understand that Figure 12 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0162] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0163] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0164] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0165] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0166] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0167] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0168] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for training a cover song recognition model, characterized in that, The method includes: Obtain the original song and cover song corresponding to the sample music, and extract the original song features corresponding to the original song and the cover song features corresponding to the cover song. The process of mixing the features of the cover song with the features of the original song to obtain mixed song features includes: obtaining a pre-set perturbation parameter; obtaining a first feature weight of the cover song features and a second feature weight of the original song features based on the perturbation parameter; and using the first feature weight and the second feature weight to perform weighted processing on the cover song features and the original song features to obtain the mixed song features. The mixed song features are input into the encoder of the cover song recognition model to be trained, and the encoder obtains the melody features corresponding to the mixed song features. The melody features are then input into the decoder of the cover song recognition model, and the decoder obtains the reconstructed song features. Based on the differences between the reconstructed song features and the cover song features, the cover song recognition model is trained to obtain the trained cover song recognition model.

2. The method according to claim 1, characterized in that, The extraction of the original song features corresponding to the original song and the cover song features corresponding to the cover song includes: A Fourier transform is performed on the first target song to obtain the spectrum diagram corresponding to the first target song, and the frequency peak points corresponding to the first target song are obtained from the spectrum diagram; the first target song includes the original song and the cover song; Based on the frequency peak points, the spectrum peak sequence corresponding to the frequency peak points is obtained, and the spectrum peak sequence is used as the song feature corresponding to the first target song; If the first target song is the original song, then the song features corresponding to the first target song are used as the original song features corresponding to the original song. If the first target song is the cover song, then the song features corresponding to the first target song are used as the cover song features corresponding to the cover song.

3. The method according to claim 2, characterized in that, The step of obtaining the spectral peak sequence corresponding to the frequency peak point based on the frequency peak point includes: Obtain the frequency value corresponding to each frequency peak point, and obtain the frequency mean and frequency variance corresponding to the spectrum based on the frequency value; Using the frequency mean and frequency variance, the frequency values ​​corresponding to each frequency peak point are normalized to obtain normalized frequency peak points. The spectral peak sequence is constructed using the normalized frequency peak points.

4. The method according to claim 1, characterized in that, The extraction of the original song features corresponding to the original song and the cover song features corresponding to the cover song includes: The second target song is subjected to constant Q-transform processing to obtain the constant Q-transform feature corresponding to the second target song; the second target song includes the original song and the cover song. If the second target song is the original song, then the constant Q-transform feature corresponding to the second target song is used as the feature of the original song corresponding to the original song. If the second target song is the cover song, then the constant Q-transform feature corresponding to the second target song is used as the cover song feature corresponding to the cover song.

5. The method according to any one of claims 1 to 4, characterized in that, After obtaining the trained cover song recognition model, the process also includes: Obtain the candidate song features corresponding to the candidate songs; The candidate song features are input into the encoder of the trained cover song recognition model, and the candidate melody features corresponding to the candidate song features are obtained through the encoder. Construct a correspondence between the music corresponding to the candidate song and the candidate melody features, and use the correspondence to construct a melody feature library.

6. A method for identifying cover songs, characterized in that, The method includes: Obtain the cover song to be identified, and the song features corresponding to the cover song to be identified; The song features corresponding to the cover song to be identified are input into the encoder of the trained cover song recognition model, and the melody features of the cover song to be identified are obtained through the encoder; the cover song recognition model is trained by the cover song recognition model training method as described in any one of claims 1 to 5. Based on the melodic features, the music corresponding to the cover song to be identified is determined.

7. The method according to claim 6, characterized in that, The step of identifying the music corresponding to the cover song to be identified based on the melodic features includes: The melody features are input into a pre-constructed melody feature library, and the target melody feature with the highest similarity to the melody features is obtained from the candidate melody features stored in the melody feature library; the melody feature library stores the correspondence between each candidate melody feature and music. The music corresponding to the target melody features is used as the music corresponding to the cover song to be identified.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images