Cover song identification method, computer device and storage medium

Abstract

The application relates to a cover song segment identification method, computer equipment, a storage medium and a computer program product. The method comprises the following steps: performing audio supplementing processing on a to-be-identified cover song segment according to a preset audio duration, so as to obtain a supplemented audio segment of the to-be-identified cover song segment; the duration of the supplemented audio segment is equal to the preset audio duration; inputting the supplemented audio segment into a pre-trained audio feature extraction model, so as to obtain cover audio features corresponding to the supplemented audio segment; the pre-trained audio feature extraction model is obtained through training of an original song segment, a positive cover song segment corresponding to the original song segment and a negative cover song segment corresponding to the original song segment; and according to the feature similarity between the cover audio features and the audio features of each song in a song library, filtering out songs satisfying a preset similarity condition from the song library as a cover song segment identification result corresponding to the to-be-identified cover song segment. The method can improve the cover song segment identification efficiency.

Description

Technical Field

[0001] This application relates to the field of audio technology, and in particular to a method for recognizing recorded audio segments, a computer device, a storage medium, and a computer program product. Background Technology

[0002] Song recognition is an important technology in the music field. Cover songs and adaptations differ from the originals in rhythm, timbre, and pitch, making accurate identification of cover songs quite difficult.

[0003] Currently, cover song audio is often processed using dense slicing, and then multiple audio segments obtained based on this method are identified. Although existing slicing methods can reduce the loss of information in audio segments, they still require multiple feature extraction and recognition operations on a large number of audio segments, which greatly increases the computational load and time consumption, resulting in low recognition efficiency of existing methods. Summary of the Invention

[0004] Therefore, it is necessary to provide a method, computer device, computer-readable storage medium, and computer program product that can improve the efficiency of copying segment recognition in response to the above-mentioned technical problems.

[0005] Firstly, this application provides a method for identifying copied music segments. The method includes:

[0006] Based on the preset audio duration, the audio segment to be identified is subjected to audio completion processing to obtain the completed audio segment of the segment to be identified; the duration of the completed audio segment is equal to the preset audio duration.

[0007] The completed audio segment is input into a pre-trained audio feature extraction model to obtain the cover audio features corresponding to the completed audio segment; the pre-trained audio feature extraction model is trained by the original song segment, the positive cover segment corresponding to the original song segment, and the negative cover segment corresponding to the original song segment;

[0008] Based on the feature similarity between the cover song audio features and the audio features of each song in the music library, songs that meet the preset similarity conditions are selected from the music library and used as the cover song recognition result corresponding to the cover song to be identified.

[0009] In one embodiment, the pre-trained audio feature extraction model is trained as follows:

[0010] Obtain the original song fragment, the positive cover segment corresponding to the original song fragment, and the negative cover segment corresponding to the original song fragment;

[0011] The original song segment, the positive cover song segment, and the negative cover song segment are input into the audio feature extraction model to be trained to obtain the audio features of the original song segment, the audio features of the positive cover song segment, and the audio features of the negative cover song segment.

[0012] The metric learning loss of the audio feature extraction model to be trained is obtained based on the distance between the audio features of the original song segment and the audio features of the positive cover song segment, and the distance between the audio features of the original song segment and the audio features of the negative cover song segment.

[0013] The audio feature extraction model to be trained is iteratively trained based on the metric learning loss to obtain the trained audio feature extraction model.

[0014] In one embodiment, obtaining the original song fragment includes:

[0015] Get the original song;

[0016] Based on the preset audio duration, the original song is divided into equal-length segments to obtain the original song segments corresponding to the original song.

[0017] In one embodiment, obtaining the positive cover segment corresponding to the original song fragment and the negative cover segment corresponding to the original song fragment includes:

[0018] Get the positive and negative cover songs corresponding to the original songs;

[0019] The positive cover songs and the negative cover songs are randomly divided into segments of varying durations to obtain the initial positive cover segments corresponding to the positive cover songs and the initial negative cover segments corresponding to the negative cover songs.

[0020] The initial positive cover song segment and the initial negative cover song segment are respectively processed to complete the segments, resulting in the positive cover song segment and the negative cover song segment; wherein the duration of the positive cover song segment and the duration of the negative cover song segment are both equal to the preset audio duration.

[0021] In one embodiment, the initial positive flip segment and the initial negative flip segment are respectively subjected to segment completion processing to obtain the positive flip segment and the negative flip segment, including:

[0022] Based on Gaussian white noise within a preset decibel range, noise is added to the initial positive flip record segment and the initial negative flip record segment respectively to obtain the processed positive flip record segment and the processed negative flip record segment.

[0023] Based on the scene noise of the target scene, the processed positive flip record segment and the processed negative flip record segment are spliced ​​together to obtain the positive flip record segment corresponding to the processed positive flip record segment and the negative flip record segment corresponding to the processed negative flip record segment.

[0024] In one embodiment, based on the scene noise of the target scene, the processed positive flip record segment and the processed negative flip record segment are respectively spliced ​​together to obtain the positive flip record segment corresponding to the processed positive flip record segment and the negative flip record segment corresponding to the processed negative flip record segment, including:

[0025] Determine the first duration difference between the processed positive cover song segment and the preset audio duration, and the second duration difference between the processed negative cover song segment and the preset audio duration;

[0026] The scene noise of the first target duration is spliced ​​to the first and last segments of the processed front-facing flip segment to obtain the front-facing flip segment corresponding to the processed front-facing flip segment; the first target duration is calculated based on the first duration difference of the processed front-facing flip segment;

[0027] The scene noise of the second target duration is spliced ​​to the first and last segments of the processed negative cover segment to obtain the negative cover segment corresponding to the processed negative cover segment; the second target duration is calculated based on the second duration difference of the processed negative cover segment.

[0028] In one embodiment, the metric learning loss of the audio feature extraction model to be trained is obtained based on the distance between the audio features of the original song segment and the audio features of the positive cover song segment, and the distance between the audio features of the original song segment and the audio features of the negative cover song segment, including:

[0029] Determine the frontal cosine distance between the audio features of the original song segment and the audio features of the front-recorded song segment;

[0030] Determine the negative cosine distance between the audio features of the original song segment and the audio features of the negative cover segment;

[0031] The metric learning loss of the audio feature extraction model to be trained is obtained based on the difference between the positive cosine distance and the negative cosine distance.

[0032] In one embodiment, based on a preset audio duration, audio completion processing is performed on the record-covering segment to be identified, resulting in a completed audio segment of the record-covering segment to be identified, including:

[0033] The duration of the record segment to be identified is obtained by performing duration detection processing.

[0034] If the duration of the record segment to be identified does not meet the preset audio duration, zero-padding is performed on the first and last segments of the record segment to be identified to obtain the padded audio segment of the record segment to be identified.

[0035] In one embodiment, before selecting songs that meet preset similarity conditions from the music library based on the feature similarity between the cover song audio features and the audio features of each song in the music library, and using these songs as the cover song recognition result corresponding to the cover song to be identified, the method further includes:

[0036] Retrieve each song from the music library;

[0037] Based on the preset audio duration, each song in the music library is divided into equal duration segments to obtain music library song segments for each song in the music library.

[0038] The song fragments from the music library of each song are sequentially input into the pre-trained audio feature extraction model to obtain the audio features of each song.

[0039] Secondly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to perform the following steps:

[0040] Based on the preset audio duration, the audio segment to be identified is subjected to audio completion processing to obtain the completed audio segment of the segment to be identified; the duration of the completed audio segment is equal to the preset audio duration.

[0041] The completed audio segment is input into a pre-trained audio feature extraction model to obtain the cover audio features corresponding to the completed audio segment; the pre-trained audio feature extraction model is trained by the original song segment, the positive cover segment corresponding to the original song segment, and the negative cover segment corresponding to the original song segment;

[0042] Based on the feature similarity between the cover song audio features and the audio features of each song in the music library, songs that meet the preset similarity conditions are selected from the music library and used as the cover song recognition result corresponding to the cover song to be identified.

[0043] Thirdly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, performs the following steps:

[0044] Based on the preset audio duration, the audio segment to be identified is subjected to audio completion processing to obtain the completed audio segment of the segment to be identified; the duration of the completed audio segment is equal to the preset audio duration.

[0045] The completed audio segment is input into a pre-trained audio feature extraction model to obtain the cover audio features corresponding to the completed audio segment; the pre-trained audio feature extraction model is trained by the original song segment, the positive cover segment corresponding to the original song segment, and the negative cover segment corresponding to the original song segment;

[0046] Based on the feature similarity between the cover song audio features and the audio features of each song in the music library, songs that meet the preset similarity conditions are selected from the music library and used as the cover song recognition result corresponding to the cover song to be identified.

[0047] Fourthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, performs the following steps:

[0048] Based on the preset audio duration, the audio segment to be identified is subjected to audio completion processing to obtain the completed audio segment of the segment to be identified; the duration of the completed audio segment is equal to the preset audio duration.

[0049] The completed audio segment is input into a pre-trained audio feature extraction model to obtain the cover audio features corresponding to the completed audio segment; the pre-trained audio feature extraction model is trained by the original song segment, the positive cover segment corresponding to the original song segment, and the negative cover segment corresponding to the original song segment;

[0050] Based on the feature similarity between the cover song audio features and the audio features of each song in the music library, songs that meet the preset similarity conditions are selected from the music library and used as the cover song recognition result corresponding to the cover song to be identified.

[0051] The aforementioned method, computer equipment, storage medium, and computer program product for identifying cover songs perform audio padding on the cover song to be identified, based on a preset audio duration, to obtain a padded audio segment. The duration of the padded audio segment is equal to the preset audio duration. The padded audio segment is then input into a pre-trained audio feature extraction model to obtain the cover song audio features corresponding to the padded audio segment. The pre-trained audio feature extraction model is trained using original song segments, positive cover songs corresponding to the original song segments, and negative cover songs corresponding to the original song segments. Based on the feature similarity between the cover song audio features and the audio features of various songs in the music library, songs that meet the preset similarity conditions are selected from the music library as the cover song identification results corresponding to the cover song to be identified. Using this method, there is no need to slice the cover song to be identified. It can extract complete melody information from the padded audio segment of the cover song to be identified, greatly reducing the number of slicing, feature extraction, and music library searches in the cover song identification process, thereby improving the identification efficiency of the cover song to be identified. Attached Figure Description

[0052] Figure 1 This is a flowchart illustrating a method for identifying flipped record segments in one embodiment;

[0053] Figure 2 This is a flowchart illustrating the training steps of a pre-trained audio feature extraction model in one embodiment.

[0054] Figure 3 This is a schematic diagram illustrating the principle of measuring learning loss in one embodiment;

[0055] Figure 4 This is a flowchart illustrating the steps for obtaining a fragment of the original song in one embodiment;

[0056] Figure 5 This is a flowchart illustrating the steps of obtaining the positive cover segment and the negative cover segment corresponding to the original song fragment in one embodiment.

[0057] Figure 6 This is a schematic diagram of a record segment recognition method in another embodiment;

[0058] Figure 7 This is a flowchart illustrating the training process of a pre-trained audio feature extraction model in another embodiment.

[0059] Figure 8 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0060] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0061] In one embodiment, such as Figure 1 As shown, a method for identifying flipped music segments is provided. This embodiment illustrates the application of this method to a terminal. It is understood that this method can also be applied to a server, and further to a system including both a terminal and a server, and is implemented through interaction between the terminal and the server. The terminal can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. The server can be a standalone server or a server cluster consisting of multiple servers. In this embodiment, the method includes the following steps:

[0062] Step S101: Based on the preset audio duration, perform audio completion processing on the record segment to be identified to obtain the completed audio segment of the record segment to be identified; the duration of the completed audio segment is equal to the preset audio duration.

[0063] Among them, the cover song segment to be identified refers to the song segment of the cover song that needs to be identified to correspond to the original song.

[0064] Specifically, the terminal obtains a preset audio duration. After obtaining the record segment to be identified, the terminal does not need to slice the record segment to be identified. Instead, it checks whether the record segment to be identified meets the preset audio duration. When it is detected that the duration of the audio segment does not meet the preset audio duration, the terminal can perform audio padding on the audio segment so that the duration of the padding audio segment is equal to the preset audio duration.

[0065] It should be noted that the preset audio duration is set based on the maximum audio capture duration. For example, if a user uses music software on a smart terminal (such as a mobile phone) to perform a song recognition operation, and the music software captures audio for 0 to 15 seconds during this operation, then the preset audio duration can be set to 15 seconds.

[0066] Step S102: Input the completed audio segment into the pre-trained audio feature extraction model to obtain the cover audio features corresponding to the completed audio segment; the pre-trained audio feature extraction model is trained by the original song segment, the positive cover segment corresponding to the original song segment, and the negative cover segment corresponding to the original song segment.

[0067] Among them, the audio feature extraction model refers to the model used to extract audio features from audio data (such as completing audio segments).

[0068] Specifically, the terminal can pre-train an audio feature extraction model using original song segments, corresponding positive cover segments, and corresponding negative cover segments to obtain a pre-trained audio feature extraction model. Then, after acquiring the completed audio segment, the terminal can input the completed audio segment into the pre-trained audio feature extraction model for audio feature extraction processing. The pre-trained audio feature extraction model outputs the cover audio features corresponding to the completed audio segment.

[0069] Step S103: Based on the feature similarity between the cover song audio features and the audio features of each song in the music library, select songs that meet the preset similarity conditions from the music library as the cover song recognition results corresponding to the cover song to be identified.

[0070] Feature similarity is an indicator that measures the similarity between the features of a cover song's audio and the audio features of each song in the music library. Preset similarity conditions refer to the judgment conditions set for the feature similarity between the cover song's audio features and the audio features of each song in the music library. The cover song recognition result refers to the original song corresponding to the identified cover song.

[0071] Specifically, the terminal calculates the feature similarity between the cover song audio features and the audio features of each song in the music library. Then, based on the feature similarity, it selects the target feature similarity that meets the preset similarity conditions. The terminal then uses the song corresponding to the target feature similarity as the cover song recognition result corresponding to the cover song to be identified.

[0072] In the aforementioned method for identifying cover songs, the cover song to be identified is padded with audio according to a preset audio duration to obtain a padded audio segment. The duration of the padded audio segment is equal to the preset audio duration. The padded audio segment is then input into a pre-trained audio feature extraction model to obtain the cover song audio features corresponding to the padded audio segment. The pre-trained audio feature extraction model is trained using original song segments, positive cover songs corresponding to the original song segments, and negative cover songs corresponding to the original song segments. Based on the feature similarity between the cover song audio features and the audio features of various songs in the music library, songs that meet the preset similarity conditions are selected from the music library as the cover song identification results corresponding to the cover song to be identified. Using this method, there is no need to slice the cover song to be identified. It can extract complete melody information from the padded audio segment of the cover song to be identified, greatly reducing the number of slicing, feature extraction, and music library searches in the cover song identification process, thereby improving the efficiency of cover song identification.

[0073] In one embodiment, such as Figure 2As shown, the pre-trained audio feature extraction model in step S102 above is trained in the following way:

[0074] Step S201: Obtain the original song fragment, the corresponding positive cover segment of the original song fragment, and the corresponding negative cover segment of the original song fragment.

[0075] In this context, a positive cover song refers to a song segment that is directly covered or adapted based on the accompaniment, vocal style, and rhythm of the original song. A negative cover song, on the other hand, refers to a song segment that is directly covered or adapted based on a segment other than the original song. For example, suppose a user covers segment 'a' from segment A of song 1, and another user covers segment 'b' from segment B of song 2. Then, the positive cover song segment of segment A is 'a', and the negative cover song segment of segment A is 'b'. Similarly, the positive cover song segment of segment B is 'b', and the negative cover song segment of segment B is 'a'.

[0076] Specifically, the terminal acquires the original song, as well as the positive and negative cover songs corresponding to the original song; then, based on the preset audio duration, the terminal slices the original song, the positive cover songs, and the negative cover songs to obtain original song segments, as well as the positive and negative cover segments corresponding to the original song segments.

[0077] Step S202: Input the original song segment, the positive cover segment, and the negative cover segment into the audio feature extraction model to be trained to obtain the audio features of the original song segment, the audio features of the positive cover segment, and the audio features of the negative cover segment.

[0078] Specifically, the terminal can combine original song clips, positive cover clips, and negative cover clips to obtain corresponding triplet samples. The terminal then performs frequency domain transformation on these triplet samples to obtain their corresponding spectral features, providing richer feature information. These spectral features describe the frequency domain characteristics of the triplet samples; for example, they could be Mel-scale Frequency Cepstral Coefficients (MFCCs). The terminal inputs these spectral features into the audio feature extraction model to be trained, allowing the model to perform feature extraction. This results in the terminal obtaining the audio features of the original song clip, the positive cover clip, and the negative cover clip. These audio features can be embedding features.

[0079] Step S203: Based on the distance between the audio features of the original song segment and the audio features of the positive cover song segment, and the distance between the audio features of the original song segment and the audio features of the negative cover song segment, the metric learning loss of the audio feature extraction model to be trained is obtained.

[0080] Step S204: Based on the metric learning loss, iteratively train the audio feature extraction model to be trained to obtain the trained audio feature extraction model.

[0081] Specifically, the terminal calculates the metric learning loss for the audio feature extraction model to be trained based on the distances between the audio features of the original song segment and the audio features of the positive cover segment, as well as the distances between the audio features of the original song segment and the negative cover segment. The terminal iteratively trains the audio feature extraction model using this metric learning loss, making the distances between the audio features of the original song segment and the positive cover segment increasingly closer, while simultaneously increasing the distances between the audio features of the original song segment and the negative cover segment. When the metric learning loss converges, the terminal obtains the pre-trained audio feature extraction model. In practical applications, to ensure that the audio feature extraction model does not encounter gradient problems during training, a residual neural network can be used to construct the audio feature extraction model.

[0082] It should be noted that the implementation of metric learning loss differs from that of conventional loss. Figure 3 A schematic diagram illustrating the principle of measuring learning loss, as shown below. Figure 3 As shown, metric learning loss can learn a distance function to gradually cluster similar audio features together and separate dissimilar audio features from each other. Since the difference between negative cover segments and original song segments is obviously much greater than the difference between positive cover segments and original song segments, metric learning loss will make the audio features of positive cover segments output by the audio feature extraction model continuously approach the audio features of original song segments, thereby continuously improving the feature extraction ability of the audio feature extraction model for cover segments.

[0083] In this embodiment, the positive cover segments corresponding to the original song fragments and the negative cover segments corresponding to the original song fragments are used as the training dataset for the audio feature extraction model to be trained. The audio feature extraction model to be trained is trained by using the distance between the audio features of the original song fragments and the audio features of the positive cover segments, and the distance between the audio features of the original song fragments and the audio features of the negative cover segments. During the iterative training process, the audio feature extraction model outputs audio features of the original song fragments that are closer to those of the positive cover segments, and audio features of the original song fragments that are further apart from those of the negative cover segments. Thus, the pre-trained audio feature extraction model can accurately identify cover songs of the original songs. Even for cover song fragments adapted from the original song fragments, the trained audio feature extraction model has a high accuracy rate.

[0084] In one embodiment, such as Figure 4 As shown, step S201 above, obtaining the original song fragment, specifically includes the following:

[0085] Step S401: Obtain the original song.

[0086] Step S402: According to the preset audio duration, the original song is divided into segments of equal duration to obtain the original song segments corresponding to the original song.

[0087] Specifically, the terminal can obtain the original song from the server or local database. After obtaining the original song, the terminal divides it into equal-length segments according to a preset audio duration, thus obtaining the corresponding original song fragments.

[0088] In this embodiment, the original song is divided into segments of equal duration according to the preset audio duration to obtain the original song segments corresponding to the original song. By using the equal duration slicing method, the audio feature extraction model can identify the preset audio duration and song segments of any duration within the preset audio duration.

[0089] In one embodiment, such as Figure 5 As shown, step S201 above, which obtains the positive cover segment corresponding to the original song fragment and the negative cover segment corresponding to the original song fragment, specifically includes the following:

[0090] Step S501: Obtain the positive and negative cover songs corresponding to the original song.

[0091] Step S502: Randomly divide the positive cover song and the negative cover song into segments of varying durations to obtain the initial positive cover segment corresponding to the positive cover song and the initial negative cover segment corresponding to the negative cover song.

[0092] Step S503: Perform segment completion processing on the initial positive cover segment and the initial negative cover segment respectively to obtain the positive cover segment and the negative cover segment; wherein the duration of the positive cover segment and the duration of the negative cover segment are both equal to the preset audio duration.

[0093] Specifically, the terminal obtains the positive cover song and the negative cover song corresponding to the original song in step S401 above. For the positive and negative cover songs, the terminal can randomly divide them into sub-segments of unequal length, thus obtaining the initial positive cover segment for the positive cover song and the initial negative cover segment for the negative cover song. It is understood that to ensure each initial cover segment (including the initial positive and initial negative cover segments) contains melodic information, constraints can be set on the random division, i.e., the duration of each initial cover segment obtained by random division must not be less than 1 second. Furthermore, the terminal can also use noise to perform segment completion processing on the initial positive and initial negative cover segments that do not meet the preset audio duration, obtaining the positive cover segment and the negative cover segment respectively.

[0094] In this embodiment, by randomly segmenting positive and negative cover songs into segments of varying durations, and using noise to complete the initial positive and negative cover segments respectively, the audio feature extraction model can learn the audio features of cover segments of different durations through random duration segmentation. Therefore, the pre-trained audio feature extraction model can accurately identify cover segments of any duration within a preset audio duration. Combined with the method of directly identifying cover segments of any duration within a preset audio duration without slicing, this not only improves the efficiency of identifying cover segments but also enhances the accuracy of identifying cover segments of any duration within a preset audio duration in real-world song recognition scenarios.

[0095] In one embodiment, step S503 above, which involves performing segment completion processing on the initial positive flip record segment and the initial negative flip record segment to obtain a positive flip record segment and a negative flip record segment, specifically includes the following: adding noise to the initial positive flip record segment and the initial negative flip record segment according to Gaussian white noise within a preset decibel range to obtain a processed positive flip record segment and a processed negative flip record segment; and performing segment splicing processing on the processed positive flip record segment and the processed negative flip record segment according to the scene noise of the target scene to obtain a positive flip record segment corresponding to the processed positive flip record segment and a negative flip record segment corresponding to the processed negative flip record segment.

[0096] The target scene refers to the real-world scenarios that the music recognition technology may involve, such as parks, residences, supermarkets, schools, and subway stations. Scene noise refers to the noise data collected from the selected target scene.

[0097] Specifically, the terminal adds noise to the initial positive and negative cover segments respectively. This can be done by randomly selecting one or more Gaussian white noises from a preset decibel range and adding them to the initial positive and negative cover segments, thus obtaining processed positive and negative cover segments. The preset decibel range can be set to [-10dB, 10dB]. The terminal acquires multiple scene noises from the target scene, then randomly selects scene noises from these and splices them into the processed positive and negative cover segments, thus obtaining a positive cover segment corresponding to the processed positive cover segment and a negative cover segment corresponding to the processed negative cover segment. The duration of both the spliced ​​positive and negative cover segments is equal to the preset audio duration.

[0098] Furthermore, in addition to using scene noise to concatenate the processed positive and negative cover song segments, the terminal can also perform zero-padding on the processed positive and negative cover song segments to obtain positive and negative cover song segments with a duration equal to the preset audio duration. At this point, the terminal can use the positive and negative cover song segments obtained through scene noise concatenation, or the positive and negative cover song segments obtained through zero-padding, or it can use both positive and negative cover song segments obtained through scene noise and zero-padding together to train the audio feature extraction model. This not only improves the richness of the cover song samples used for training but also enhances the robustness of the audio feature extraction model, thereby improving the recognition performance of the pre-trained audio feature extraction model for different types of cover song segments.

[0099] In this embodiment, noise is added to the initial positive and negative cover song segments according to Gaussian white noise within a preset decibel range. This results in the processed positive and negative cover song segments carrying noise interference of different intensities, thereby improving the recognition performance of the audio feature extraction model in the presence of noise interference. Furthermore, based on the scene noise of the target scene, the processed positive and negative cover song segments are spliced ​​together to obtain the corresponding positive and negative cover song segments. This allows the audio feature extraction model to learn noise information in the real scene during training, thereby improving the robustness of the audio feature extraction model and ultimately enhancing the recognition performance of the cover song segments.

[0100] In one embodiment, based on the scene noise of the target scene, the processed positive cover song segment and the processed negative cover song segment are spliced ​​together to obtain the positive cover song segment corresponding to the processed positive cover song segment and the negative cover song segment corresponding to the processed negative cover song segment. Specifically, this includes: determining a first duration difference between the processed positive cover song segment and a preset audio duration, and a second duration difference between the processed negative cover song segment and the preset audio duration; splicing scene noise of the first target duration to the first and last segments of the processed positive cover song segment to obtain the positive cover song segment corresponding to the processed positive cover song segment; the first target duration is calculated based on the first duration difference of the processed positive cover song segment; splicing scene noise of the second target duration to the first and last segments of the processed negative cover song segment to obtain the negative cover song segment corresponding to the processed negative cover song segment; the second target duration is calculated based on the second duration difference of the processed negative cover song segment.

[0101] Specifically, the terminal determines the duration difference between the processed positive cover song segment and the preset audio duration, marking it as the first duration difference, and the duration difference between the processed negative cover song segment and the preset audio duration, marking it as the second duration difference. Based on the first duration difference, the terminal determines the first target duration, which can be obtained by dividing the first duration difference by a preset value, such as 2. Then, the terminal randomly selects two scene noises with a duration equal to the first target duration from the scene noise of the target scene, and splices them to the beginning and end segments of the processed positive cover song segment, respectively. That is, one scene noise of the first target duration is spliced ​​to the beginning segment of the processed positive cover song segment, and the other scene noise of the first target duration is spliced ​​to the end segment of the processed positive cover song segment. The terminal then obtains the positive cover song segment corresponding to the processed positive cover song segment. At this point, the duration of the positive cover song segment is equal to the preset audio duration, and also equal to the duration of the original song segment. Similarly, the terminal determines the second target duration based on the second duration difference. This can be achieved by dividing the second duration difference by a preset value, such as 2. Then, the terminal randomly selects two scene noises with the second target duration from the scene noise of the target scene and splices them into the beginning and end segments of the processed negative cover song segment, respectively. That is, one scene noise with the second target duration is spliced ​​into the beginning segment of the processed negative cover song segment, and the other scene noise with the second target duration is spliced ​​into the end segment of the processed negative cover song segment. Thus, the terminal obtains the negative cover song segment corresponding to the processed negative cover song segment. At this time, the duration of the negative cover song segment is equal to the preset audio duration, and also equal to the duration of the original song segment and the duration of the positive cover song segment.

[0102] In this embodiment, based on the first duration difference between the processed positive cover song segment and the preset audio duration, scene noise of the first target duration is spliced ​​to the first and last segments of the processed positive cover song segment to obtain the positive cover song segment corresponding to the processed positive cover song segment. At the same time, based on the second duration difference between the processed negative cover song segment and the preset audio duration, scene noise of the second target duration is spliced ​​to the first and last segments of the processed negative cover song segment to obtain the negative cover song segment corresponding to the processed negative cover song segment. This not only enables the positive and negative cover song segments to carry scene noise from the real song recognition scenario, but also makes the audio feature extraction model trained by the positive and negative cover song segments more robust, ensuring that it still has a high recognition ability even in the presence of noise interference in the real song recognition scenario, thereby improving the recognition effect of cover song segments.

[0103] In one embodiment, step S203 above, which obtains the metric learning loss of the audio feature extraction model to be trained based on the distance between the audio features of the original song segment and the audio features of the positive cover segment, and the distance between the audio features of the original song segment and the audio features of the negative cover segment, specifically includes the following: determining the positive cosine distance between the audio features of the original song segment and the audio features of the positive cover segment; determining the negative cosine distance between the audio features of the original song segment and the audio features of the negative cover segment; and obtaining the metric learning loss of the audio feature extraction model to be trained based on the difference between the positive cosine distance and the negative cosine distance.

[0104] In this context, metric learning loss can be a ternary loss, for example, a ternary loss constructed using the audio features of the original song clip, the audio features of the positive cover clip, and the audio features of the negative cover clip. Metric learning loss can also be contrastive loss, quaternary loss, etc., found in metric learning.

[0105] Taking the ternary loss as an example, specifically, the terminal performs cosine distance processing on the audio features of the original song segment and the positive cover segment to obtain the positive cosine distance between them; it also performs cosine distance processing on the audio features of the original song segment and the negative cover segment to obtain the negative cosine distance; then, based on the difference between the positive and negative cosine distances, the terminal calculates the ternary loss, which is used as the metric learning loss for the audio feature extraction model to be trained. The metric learning ternary loss L can be expressed by the following formula:

[0106] L=max(d(a,p)-d(a,n)+margin,0)

[0107] Where a represents the audio features of the original song segment; p represents the audio features of the positive cover segment; n represents the audio features of the negative cover segment; d() represents the cosine distance function, then d(a,p) represents the positive cosine distance, and d(a,n) represents the negative cosine distance; margin represents an adjustable degree coefficient, which is used to control the inter-class distance of the audio features of the three types of song segments: original song segment, positive cover segment, and negative cover segment.

[0108] In this embodiment, the positive cosine distance between the audio features of the original song segment and the audio features of the positive cover segment, and the negative cosine distance between the audio features of the original song segment and the audio features of the negative cover segment are determined. Then, based on the difference between the positive and negative cosine distances, the metric learning loss of the audio feature extraction model to be trained is obtained. The distance function in the metric learning loss can then be used to train the audio feature extraction model, so that the audio features of the positive cover segment output by the audio feature extraction model continuously approach the audio features of the original song segment, thereby continuously improving the feature extraction capability of the audio feature extraction model for cover segments.

[0109] In one embodiment, step S101 above, which involves performing audio padding on the record segment to be identified based on a preset audio duration, to obtain a padded audio segment of the record segment to be identified; the duration of the padded audio segment is equal to the preset audio duration, specifically includes the following: performing duration detection processing on the record segment to be identified to obtain the duration of the record segment to be identified; if it is detected that the duration of the record segment to be identified does not meet the preset audio duration, zero padding is performed on the first and last segments of the record segment to be identified to obtain a padded audio segment of the record segment to be identified.

[0110] Specifically, in real-world scenarios, the duration of the song segment to be identified, acquired by the terminal, may be less than the preset audio duration. Therefore, the terminal needs to perform duration detection processing on the song segment to be identified to obtain its duration. When the duration of the song segment to be identified is equal to the preset audio duration, the terminal inputs the song segment to be identified as a padded audio segment into the pre-trained audio feature extraction model. When the duration of the song segment to be identified is less than the preset audio duration, the terminal pads the beginning and end of the song segment with zeros, either by padding the beginning and end of the song segment with zeros, so that the duration of the padded audio segment is equal to the preset audio duration. It is understandable that since the preset audio duration is set based on the maximum acquisition time in the song recognition scenario, there is no case where the song segment to be identified is longer than the preset audio duration.

[0111] In this embodiment, the duration of the cover song segment to be identified is obtained by performing duration detection processing. If the duration of the cover song segment to be identified does not meet the preset audio duration, zero-padding is performed on the first and last segments of the cover song segment to be identified to obtain the padded audio segment. This not only makes the duration of the padded audio segment equal to the preset audio duration, so that the pre-trained audio feature extraction model can identify the padded audio segment, but also avoids introducing irrelevant information during the padded process of the cover song segment to be identified, reducing the impact of the original melody information in the cover song segment to be identified.

[0112] In one embodiment, before step S103 above, which selects songs from the music library that meet the preset similarity conditions based on the feature similarity between the cover song audio features and the audio features of each song in the music library, and uses them as the cover song recognition result corresponding to the cover song segment to be identified, the method further includes: acquiring each song in the music library; dividing each song in the music library into equal-length segments according to the preset audio duration to obtain music library song segments for each song in the music library; and sequentially inputting the music library song segments of each song into a pre-trained audio feature extraction model to obtain the audio features of each song.

[0113] Among them, the song fragments in the music library refer to the song fragments obtained by slicing the songs in the music library based on preset audio duration.

[0114] After acquiring a pre-trained audio feature extraction model, the terminal can use it to obtain the audio features of each song. Specifically, the terminal can acquire a music library containing multiple songs, or collect songs of a preset genre to construct the music library. Based on a preset audio duration, the terminal divides each song in the music library into equal-length segments, obtaining song segments for each song in the library; the duration of each song segment is equal to the preset audio duration. The terminal sequentially inputs these song segments into the pre-trained audio feature extraction model, which outputs the embedding features of each song segment as its audio feature. The audio features of a song are then formed by combining the audio features of all its song segments.

[0115] In this embodiment, each song in the music library is divided into segments of equal duration according to a preset audio duration to obtain song segments of each song in the music library. Then, the song segments of each song are sequentially input into a pre-trained audio feature extraction model to obtain the audio features of each song. This realizes the reasonable acquisition of the audio features of each song in the music library, so that the audio features of each song in the music library can be used as the basis for processing to perform the subsequent record segment recognition step.

[0116] In one embodiment, such as Figure 6 As shown, another method for identifying flipped record segments is provided. Taking the application of this method to a terminal as an example, the method includes the following steps:

[0117] Step S601: Perform duration detection processing on the record segment to be identified to obtain the duration of the record segment to be identified.

[0118] Step S602: If the duration of the record segment to be identified does not meet the preset audio duration, zero padding is performed on the first and last segments of the record segment to be identified to obtain the padded audio segment of the record segment to be identified.

[0119] The duration of the supplemented audio segment is equal to the preset audio duration.

[0120] Step S603: Input the completed audio segment into the pre-trained audio feature extraction model to obtain the cover song audio features corresponding to the completed audio segment.

[0121] The pre-trained audio feature extraction model is trained using original song segments, positive cover segments corresponding to the original song segments, and negative cover segments corresponding to the original song segments.

[0122] Step S604: Based on the feature similarity between the cover song audio features and the audio features of each song in the music library, select songs that meet the preset similarity conditions from the music library as the cover song recognition results corresponding to the cover song to be identified.

[0123] The above-mentioned cover song segment recognition method can achieve the following beneficial effects: it eliminates the need to slice the cover song segment to be recognized, and can extract complete melody information from the completed audio fragment of the cover song segment to be recognized, which greatly reduces the number of slicing, feature extraction, and music library retrieval in the cover song recognition process. Compared with the traditional dense slicing and fixed-length slicing methods, it effectively improves the recognition efficiency of the cover song segment to be recognized.

[0124] To more clearly illustrate the record segment recognition method provided in this disclosure, the training process of the pre-trained audio feature extraction model described above will be specifically explained below using a specific embodiment. Figure 7 This diagram illustrates the training process of a pre-trained audio feature extraction model, specifically including the following:

[0125] (1) The terminal obtains the original song, as well as the cover song 1, cover song 2, ..., cover song n corresponding to the original song.

[0126] (2) Assuming the audio acquisition time during the song recognition operation is 0 to 15 seconds, the terminal can divide the original song into 15-second segments of equal length, thus obtaining m segments of the original song with a duration of 15 seconds each. Based on the constraints, the terminal performs random duration slicing on n cover songs, that is, randomly dividing the cover songs into segments of unequal length. The terminal then obtains m initial cover segments for each cover song, where the duration of each initial cover segment must not be less than 1 second to ensure that each initial cover segment contains melodic information. It is understandable that the number of initial cover segments obtained from random slicing of different cover songs may not be equal.

[0127] (3) The terminal randomly acquires Gaussian white noise between -10dB and 10dB and adds it to the initial record-turning segment to obtain the processed record-turning segment. The terminal randomly selects two scene noises of equal duration from the scene noise of the target scene and splices them to the first and last segments of the processed record-turning segment, so that the duration of the spliced ​​record-turning segment is 15 seconds.

[0128] (4) The terminal selects positive and negative cover segments corresponding to the original song segments from the cover segments; then it combines the original song segments, positive cover segments and negative cover segments to construct the corresponding triplet samples.

[0129] (5) The terminal performs frequency domain transformation on the triplet samples to obtain the MFCC features corresponding to the triplet samples, so as to provide richer feature information.

[0130] (6) The terminal constructs an audio feature extraction model to be trained based on a residual neural network. The terminal inputs the MFCC features into the audio feature extraction model to be trained, and performs feature extraction processing on the MFCC features through the audio feature extraction model to be trained. Then the terminal obtains the embedding features of the original song segment, the embedding features of the positive cover song segment, and the embedding features of the negative cover song segment.

[0131] (7) The terminal constructs a ternary loss for the audio feature extraction model to be trained based on the distance between the embedding features of the original song segment and the embedding features of the positive cover song segment, as well as the distance between the embedding features of the original song segment and the embedding features of the negative cover song segment. The terminal iteratively trains the audio feature extraction model to be trained using the ternary loss. When the ternary loss converges, the terminal obtains the pre-trained audio feature extraction model.

[0132] In this embodiment, 1) by adding Gaussian white noise to the initial cover song segment, the processed cover song segment carries noise interference of different intensities, improving the recognition effect of the audio feature extraction model in the presence of noise interference; 2) by splicing the scene noise of the target scene to the processed cover song segment, the corresponding cover song segment is obtained, so that the audio feature extraction model learns the noise information of the real scene during the training process, thereby improving the robustness of the audio feature extraction model and thus improving the recognition effect of the cover song segment; 3) by randomly segmenting the cover song into segments of different durations, the audio feature extraction model can learn the embedding features of cover song segments of different durations. Therefore, the pre-trained audio feature extraction model can accurately identify cover song segments of any duration within 15 seconds, further improving the recognition accuracy of cover song segments.

[0133] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0134] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 8As shown, the computer device includes a processor, memory, input / output interface, communication interface, display unit, and input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interface. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interface is used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When executed by the processor, the computer program implements a method for identifying recorded music segments. The display unit is used to form a visually visible image and can be a display screen, projection device, or virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0135] Those skilled in the art will understand that Figure 8 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0136] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0137] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0138] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0139] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data shall comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0140] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0141] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0142] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for identifying recorded song segments, characterized in that, The method includes: Based on the preset audio duration, the audio segment to be identified is subjected to audio completion processing to obtain the completed audio segment of the segment to be identified; the duration of the completed audio segment is equal to the preset audio duration. The completed audio segment is input into a pre-trained audio feature extraction model to obtain the cover audio features corresponding to the completed audio segment. The pre-trained audio feature extraction model is trained using original song segments, positive cover segments corresponding to the original song segments, and negative cover segments corresponding to the original song segments. The positive cover segments are obtained by completing segments from an initial positive cover segment. The negative cover segments are obtained by completing segments from an initial negative cover segment. The durations of both the positive and negative cover segments are equal to the preset audio duration. The initial positive cover segment is obtained by randomly segmenting the duration of the positive cover segment corresponding to the original song. The initial negative cover segment is obtained by randomly segmenting the duration of the initial negative cover segment corresponding to the negative cover song. Based on the feature similarity between the cover song audio features and the audio features of each song in the music library, songs that meet the preset similarity conditions are selected from the music library and used as the cover song recognition result corresponding to the cover song to be identified.

2. The method according to claim 1, characterized in that, The pre-trained audio feature extraction model is trained in the following manner: Obtain the original song segment, the corresponding positive cover song segment, and the corresponding negative cover song segment; The original song segment, the positive cover song segment, and the negative cover song segment are input into the audio feature extraction model to be trained to obtain the audio features of the original song segment, the audio features of the positive cover song segment, and the audio features of the negative cover song segment. The metric learning loss of the audio feature extraction model to be trained is obtained based on the distance between the audio features of the original song segment and the audio features of the positive cover song segment, and the distance between the audio features of the original song segment and the audio features of the negative cover song segment. The audio feature extraction model to be trained is iteratively trained based on the metric learning loss to obtain the trained audio feature extraction model.

3. The method according to claim 2, characterized in that, The step of performing segment completion processing on the initial positive flip segment and the initial negative flip segment respectively to obtain the positive flip segment and the negative flip segment includes: Based on Gaussian white noise within a preset decibel range, noise is added to the initial positive flip record segment and the initial negative flip record segment respectively to obtain the processed positive flip record segment and the processed negative flip record segment. Based on the scene noise of the target scene, the processed positive flip record segment and the processed negative flip record segment are spliced ​​together to obtain the positive flip record segment corresponding to the processed positive flip record segment and the negative flip record segment corresponding to the processed negative flip record segment.

4. The method according to claim 3, characterized in that, The step of performing segment splicing processing on the processed positive flip record segment and the processed negative flip record segment according to the scene noise of the target scene, to obtain the positive flip record segment corresponding to the processed positive flip record segment and the negative flip record segment corresponding to the processed negative flip record segment, includes: Determine the first duration difference between the processed positive cover song segment and the preset audio duration, and the second duration difference between the processed negative cover song segment and the preset audio duration; The scene noise of the first target duration is spliced ​​to the first and last segments of the processed front-facing flip segment to obtain the front-facing flip segment corresponding to the processed front-facing flip segment; the first target duration is calculated based on the first duration difference of the processed front-facing flip segment; The scene noise of the second target duration is spliced ​​to the first and last segments of the processed negative cover segment to obtain the negative cover segment corresponding to the processed negative cover segment; the second target duration is calculated based on the second duration difference of the processed negative cover segment.

5. The method according to claim 2, characterized in that, The step of obtaining the metric learning loss of the audio feature extraction model to be trained based on the distance between the audio features of the original song segment and the audio features of the positive cover song segment, and the distance between the audio features of the original song segment and the audio features of the negative cover song segment, includes: Determine the frontal cosine distance between the audio features of the original song segment and the audio features of the front-recorded song segment; Determine the negative cosine distance between the audio features of the original song segment and the audio features of the negative cover segment; The metric learning loss of the audio feature extraction model to be trained is obtained based on the difference between the positive cosine distance and the negative cosine distance.

6. The method according to claim 1, characterized in that, The step of performing audio completion processing on the record-covering segment to be identified, based on a preset audio duration, to obtain the completed audio segment of the record-covering segment to be identified includes: The duration of the record segment to be identified is obtained by performing duration detection processing. If the duration of the record segment to be identified does not meet the preset audio duration, zero-padding is performed on the first and last segments of the record segment to be identified to obtain the padded audio segment of the record segment to be identified.

7. The method according to claim 1, characterized in that, Before selecting songs that meet preset similarity conditions from the music library based on the feature similarity between the cover song audio features and the audio features of each song in the music library, and using these songs as the cover song recognition result corresponding to the cover song to be identified, the process further includes: Retrieve each song from the music library; Based on the preset audio duration, each song in the music library is divided into equal duration segments to obtain music library song segments for each song in the music library. The song fragments from the music library of each song are input into the pre-trained audio feature extraction model to obtain the audio features of each song.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images