A song detection method and apparatus

Abstract

The application provides a song detection method and device, wherein the method comprises: obtaining at least one first to-be-detected song and text data corresponding to the first to-be-detected song; converting the text data into a phoneme representation vector and encoding the first to-be-detected song into an acoustic representation vector; performing first similarity calculation on each phoneme representation vector and dividing the first to-be-detected song into at least one first song category; for each first song category, based on the acoustic representation vector corresponding to the second to-be-detected song in the first song category, dividing the second to-be-detected song into at least one second song category, and the songs belonging to the same second song category are repeated songs.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a song detection method and apparatus. Background Technology

[0002] When using existing song detection methods to perform cluster analysis on songs, the songs are usually clustered as a whole. The accuracy of the clustering results is limited, especially for different adapted versions of the same song or different singers performing the same song. The clustering results are unstable and the error rate is high. Summary of the Invention

[0003] In view of this, the purpose of this application is to provide at least one song detection method and apparatus, which converts the text data of a first song to be detected into a phoneme representation vector and performs song encoding on the first song to be detected to obtain an acoustic representation vector, and uses the phoneme representation vector to perform similarity calculation on the first song to be detected to classify it into a first song category, and then classifies a second song to be detected in the first song category into a second song category according to the acoustic representation vector, so as to identify the songs in the second song category as duplicate songs, thereby solving the technical problem of poor accuracy of duplicate song identification in the prior art and achieving the technical effect of increasing the accuracy of duplicate song identification.

[0004] This application mainly includes the following aspects: In a first aspect, embodiments of this application provide a method for detecting duplicate songs. The method includes: acquiring at least one first song to be detected and text data corresponding to the first song to be detected; converting the text data into phoneme representation vectors and encoding the first song to be detected into acoustic representation vectors; performing a first similarity calculation on each phoneme representation vector and classifying the first song to be detected into at least one first song category; for each first song category, based on the acoustic representation vector corresponding to a second song to be detected in the first song category, classifying the second song to be detected into at least one second song category, wherein songs belonging to the same second song category are duplicate songs.

[0005] Optionally, converting the text data into a phoneme representation vector includes: converting the text data of each first song to be detected into several phoneme data; obtaining a phoneme processing model, mapping the phoneme data to a preset dimension space, performing encoding processing on each phoneme data according to the context information and phoneme dependency relationship of the phoneme data, summarizing the encoding results of all phoneme data of each first song to be detected, and obtaining at least one phoneme embedding representation sequence for each first song to be detected; performing fuzzing processing on each phoneme embedding representation sequence to obtain a phoneme representation vector for each first song to be detected.

[0006] Optionally, performing fuzzing on each phoneme embedding representation sequence to obtain the phoneme representation vector of each first song to be detected includes: performing attention mechanism calculation on the at least one phoneme embedding representation sequence, calculating the corresponding attention weight based on the attention score of each phoneme embedding representation sequence, performing phoneme embedding representation sequence fusion based on the attention weight, and decoding and restoring the fused sequence to obtain the phoneme representation vector of each first song to be detected.

[0007] Optionally, the decoding and restoration of the fusion sequence is used to obtain the phoneme representation vector of each first song to be detected, including: for a first song to be detected, the phoneme processing model uses the location information of all phoneme data as the query index, and generates the phoneme probability distribution of the location information corresponding to each phoneme data according to the attention score and the fusion sequence, and uses the phoneme feature with the highest probability as the phoneme feature of each phoneme data to form the phoneme representation vector of a first song to be detected.

[0008] Optionally, the phoneme processing model is trained by: obtaining a song training sample set, which includes at least one sample song and corresponding text sample data for each sample song; converting each text sample data into phoneme sample data, and performing random phoneme processing on the phoneme sample data of each sample song to obtain second phoneme data; having the phoneme processing model predict the phoneme data of the processed portion, and iteratively training at least one model parameter in the phoneme processing model based on the difference between the predicted phoneme vector and the actual phoneme vector of the processed phoneme in the second phoneme data.

[0009] Optionally, the phoneme processing model predicts the phoneme data of the processed portion, and iteratively trains at least one model parameter in the phoneme processing model based on the difference between the predicted phoneme vector and the actual phoneme vector of the processed phoneme in the second phoneme data. This includes: mapping each phoneme in the second phoneme data to a preset dimension space to determine the initial phoneme vector corresponding to each sample song; predicting the predicted phoneme vector based on the initial phoneme vector; establishing a first loss function between the predicted phoneme vector and the actual phoneme vector; and adjusting the model parameters of the phoneme processing model according to the loss value of the first loss function until the phoneme processing model meets the preset training completion conditions.

[0010] Optionally, random phoneme processing is performed on the phoneme sample data of each sample song to obtain second phoneme data, including: randomly selecting multiple candidate phonemes with a preset processing ratio from the phoneme sample data; selecting masked phonemes with a first preset ratio and replaced phonemes with a second preset ratio from the multiple candidate phonemes; masking the masked phonemes in the phoneme sample data of each sample song and replacing the replaced phonemes to obtain the second phoneme data.

[0011] Optionally, the plurality of candidate phonemes may also include a third preset proportion of original phoneme data, wherein the original phoneme data represents data for which no processing is performed, and together with the masked phoneme and the replaced phoneme, constitutes the second phoneme data.

[0012] Optionally, for each first song category, based on the acoustic representation vector corresponding to the second song to be detected in the first song category, the second song to be detected is divided into at least one second song category, including: performing a second similarity calculation on the acoustic representation vector corresponding to each second song to be detected in the first song category, so as to divide the second song to be detected in the first song category into at least one second song category.

[0013] Secondly, embodiments of this application also provide a song duplication detection device, the device comprising: an acquisition module for acquiring at least one first song to be detected and text data corresponding to the first song to be detected; a conversion module for converting the text data into phoneme representation vectors and encoding the first song to be detected into acoustic representation vectors; a first classification module for performing a first similarity calculation on each phoneme representation vector and classifying the first song to be detected into at least one first song category; and a second classification module for classifying the second song to be detected into at least one second song category for each first song category based on the acoustic representation vector corresponding to the second song to be detected in the first song category, wherein songs belonging to the same second song category are duplicate songs.

[0014] This application provides a song detection method and apparatus. The method includes: acquiring at least one first song to be detected and text data corresponding to the first song to be detected; converting the text data into phoneme representation vectors and encoding the first song to be detected into acoustic representation vectors; performing a first similarity calculation on each phoneme representation vector and classifying the first song to be detected into at least one first song category; for each first song category, based on the acoustic representation vector corresponding to a second song to be detected in that first song category, classifying the second song to be detected into at least one second song category, and identifying songs belonging to the same second song category as duplicate songs. This application solves the technical problem of poor accuracy in identifying duplicate songs in the prior art by converting the text data of the first song to be detected into phoneme representation vectors and encoding the first song to be detected into acoustic representation vectors, using the phoneme representation vectors to perform similarity calculations to classify the first song to be detected into first song categories, and then classifying the second songs to be detected in the first song categories into second song categories according to the acoustic representation vectors, so as to identify the songs in the second song categories as duplicate songs.

[0015] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0016] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 A flowchart of a song duplication detection method provided in an embodiment of this application is shown.

[0018] Figure 2 A flowchart illustrating the steps of training a phoneme processing model provided in an embodiment of this application is shown.

[0019] Figure 3 This diagram illustrates the functional block diagram of a song repetition detection device provided in an embodiment of this application.

[0020] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of this application is shown. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the drawings in this application are for illustrative and descriptive purposes only and are not intended to limit the scope of protection of this application. Furthermore, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in this application illustrate operations implemented according to some embodiments of this application. It should be understood that the operations in the flowcharts may not be implemented in sequence, and steps without logical contextual relationships may be reversed or implemented simultaneously. In addition, those skilled in the art, guided by the content of this application, may add one or more other operations to the flowcharts, or remove one or more operations from the flowcharts.

[0022] Furthermore, the described embodiments are merely some, not all, of the embodiments of this application. The components of the embodiments of this application described and illustrated herein can typically be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0023] In existing technologies, music models are trained to automatically generate music. However, this training process requires a large amount of diverse and high-quality songs as training data. If the training data contains many duplicate songs or different covers of the same song, the repetition rate of the same song becomes too high, making it difficult to obtain diverse data during training and resulting in the final generated songs being biased towards repetitive tracks. Furthermore, music copyright holders need to monitor copyrighted music on online platforms to protect their intellectual property rights. Therefore, it is necessary to identify duplicate songs in the training samples of the music model and to identify pirated songs on music playback software. Existing technologies, using song clustering analysis, struggle to identify different adaptations of the same song or different singers performing the same piece, leading to a high error rate.

[0024] Based on this, this application provides a song detection method and apparatus. The method involves converting the text data of a first song to be detected into a phoneme representation vector and then encoding the first song to be detected to obtain an acoustic representation vector. This phoneme representation vector is used to calculate the similarity of the first song to be detected and classify it into a first song category. Then, based on the acoustic representation vector, a second song to be detected within the first song category is classified into a second song category. Songs in the second song category are then identified as duplicate songs. This solves the technical problem of poor accuracy in identifying duplicate songs in the prior art, achieving the technical effect of increasing the accuracy of duplicate song identification. Specifically: Please see Figure 1 , Figure 1 This is a flowchart illustrating a song duplication detection method provided in an embodiment of this application. Figure 1 As shown in the embodiments of this application, the song duplication detection method includes the following steps: S101: Obtain at least one first song to be detected and the text data corresponding to the first song to be detected.

[0025] The first song to be detected may be sample data used to train the music model, or it may be a song on a music playback software. For example, if it is necessary to identify pirated songs on a music playback software, at least one first song to be detected may be obtained by searching for song titles on the music playback software. This application does not limit the method of selecting the first song to be detected.

[0026] Specifically, the text data corresponding to the first song to be detected is obtained in the following ways: the official lyrics text data corresponding to the first song to be detected, which was uploaded in advance, is determined as the text data of the first song to be detected; or, the recognized lyrics text data extracted from the first song to be detected is determined as the text data of the first song to be detected.

[0027] In other words, the official lyrics text data corresponding to the first song to be tested, which is pre-uploaded to the music playback software, can be used directly as the text data, or the lyrics of the first song to be tested can be recognized and the recognized lyrics text data can be used as the text data.

[0028] For example, the text data should at least include the lyrics of the song. These lyrics can be uploaded simultaneously with the official upload of the first song to be tested, or obtained by recognizing the first song to be tested. Specifically, before obtaining the text data corresponding to the first song to be tested, it is determined whether the official lyrics of the first song to be tested were uploaded simultaneously. If the official lyrics were uploaded simultaneously, they are used as the text data. If the official lyrics were not uploaded simultaneously, text recognition is performed on the first song to be tested, and the recognized lyrics are used as the text data. Therefore, the text data of the first song to be tested is either the official lyrics or the recognized lyrics.

[0029] Furthermore, since official lyrics data is uploaded by the official platform, its accuracy is relatively high. However, it may contain information unrelated to the lyrics, such as lyricist and composer names, which can affect subsequent processing. Recognized lyrics data can be extracted from the first song to be detected using a speech-to-text (STT) model. This type of data does not contain irrelevant information such as lyricist and composer names. However, the accuracy of lyrics data extraction depends on audio quality and the clarity of the singer's pronunciation, and is prone to errors, such as misidentifying characters with similar or identical pronunciations.

[0030] Therefore, to prevent lyric recognition errors, this application chooses to convert text data into phoneme data and then determine the phoneme representation vector. Furthermore, the lyrics of each first song to be detected are obtained by acquiring the text data of that song, so that the songs can be segmented from the perspective of the phonemes in the lyrics.

[0031] For example, before extracting the lyrics data of the first song to be detected using a speech-to-text model, the audio file of the first song to be detected can be preprocessed, such as noise reduction and music standardization, to increase the accuracy and efficiency of recognition.

[0032] S102: Convert the text data into a phoneme representation vector and encode the first song to be detected into an acoustic representation vector.

[0033] Specifically, converting the text data into a phoneme representation vector includes: converting the text data of each first song to be detected into several phoneme data; obtaining a phoneme processing model, mapping the phoneme data to a preset dimension space, and for each phoneme data, performing encoding processing on the phoneme data according to the context information and phoneme dependency relationship of the phoneme data, summarizing the encoding results of all phoneme data of each first song to be detected, and obtaining at least one phoneme embedding representation sequence for each first song to be detected; performing fuzzing processing on each phoneme embedding representation sequence to obtain the phoneme representation vector for each first song to be detected.

[0034] For example, the text data of the first song to be detected is input into a text-to-phoneme tool, such as a grapheme-to-phoneme (G2P) model, to convert the lyrics text in the text data into phonemes, thereby obtaining multiple phoneme data of the first song to be detected. The multiple phoneme data are then input into a phoneme processing model to obtain the phoneme representation vector of the first song to be detected.

[0035] The process of performing fuzzing on each phoneme embedding representation sequence to obtain the phoneme representation vector of each first song to be detected includes: performing attention mechanism calculation on the at least one phoneme embedding representation sequence, calculating the corresponding attention weight based on the attention score of each phoneme embedding representation sequence, performing phoneme embedding representation sequence fusion based on the attention weight, and decoding and restoring the fused sequence to obtain the phoneme representation vector of each first song to be detected.

[0036] The text data for the first song to be detected consists of either official lyrics or recognized lyrics, and these two types of data contain different information, resulting in incomplete alignment in dimensions and formats. Furthermore, the recognized lyrics data suffers from a high error rate. To avoid the high classification error rate that results from directly converting the data into phonemes, a phoneme processing model is needed to convert both the official and recognized lyrics data of each song into a phoneme representation vector with a unified dimension.

[0037] For example, a pre-trained phoneme processing model includes at least a phoneme embedding layer, a context encoder, and an attention module. Specifically, for each first song to be detected, each phoneme data of the song is input to the phoneme embedding layer. The phoneme embedding layer maps each phoneme data into a preset-dimensional space to obtain the initial phoneme vector of the song. The preset-dimensional space is a fixed low-dimensional vector space that transforms discrete phonemes in the phoneme data into continuous, learnable low-dimensional vectors and groups phonemes with similar pronunciations together. The initial phoneme vector of the first song to be detected is then input to the context encoder, which encodes the context information corresponding to the position of each phoneme data in the initial phoneme vector to determine the phoneme embedding representation sequence for each song. The phoneme embedding representation sequence encompasses the context information and long-distance dependencies of each phoneme data. Finally, the phoneme embedding representation sequence is input to the attention module, which performs attention mechanism calculations to compress the phoneme embedding representation sequence into a phoneme representation vector that encompasses global context information.

[0038] For example, an attention mechanism is used to calculate the attention score for each phoneme embedding representation sequence. After passing through a normalization layer, the attention scores of each phoneme embedding representation sequence are normalized to obtain the attention weights of each phoneme embedding representation sequence. Then, a weighted average is calculated on each phoneme embedding representation sequence according to the attention weights of each phoneme embedding representation sequence to obtain the fused sequence. Finally, the fused sequence is decoded and restored to obtain the phoneme representation vector of a first song to be detected.

[0039] Specifically, the decoding and restoration of the fusion sequence yields the phoneme representation vector for each first song to be detected. This includes: for a first song to be detected, the phoneme processing model uses the location information of all phoneme data as a query index, and generates a phoneme probability distribution of the location information corresponding to each phoneme data based on the attention score and the fusion sequence. The phoneme feature with the highest probability is then used as the phoneme feature of each phoneme data to form the phoneme representation vector of a first song to be detected.

[0040] In other words, for each first song to be detected, the positional encoding or the phoneme order of each phoneme data in the first song to be detected is used as the positional information of each phoneme data. The fusion sequence is input to the fully connected layer to determine the phoneme probability distribution under each positional information. The phoneme embedding representation sequence corresponding to the phoneme with the highest probability is used as the final representation of the positional information. In this way, the final representations of each phoneme data are fused into the phoneme representation vector of the first song to be detected.

[0041] Please see Figure 2 , Figure 2This is a flowchart illustrating the steps involved in training a phoneme processing model, as provided in an embodiment of this application. Figure 2 As shown, the phoneme processing model in this application embodiment is trained in the following manner: S201: Obtain the training sample set of songs.

[0042] S202: Convert each text sample data into phoneme sample data, and perform random phoneme processing on the phoneme sample data of each sample song to obtain the second phoneme data.

[0043] S203: The phoneme processing model is instructed to predict the phoneme data of the processed portion, and at least one model parameter in the phoneme processing model is iteratively trained based on the difference between the predicted phoneme vector and the actual phoneme vector of the processed phoneme in the second phoneme data.

[0044] The training sample set includes at least one sample song and corresponding text sample data for each sample song. The text sample data for the sample song refers to the official lyrics text data or the recognized lyrics text data mentioned earlier. That is, if the official lyrics text data of the sample song is uploaded beforehand, then the official lyrics text data of the sample song will be used as the text sample data of the sample song; if the official lyrics text data of the sample song is not uploaded beforehand, then the recognized lyrics text data of the sample song will be identified from the sample song using a speech-to-text model, and this recognized lyrics text data will be used as the text sample data of the sample song.

[0045] Therefore, the text sample data of the sample song is input into the G2P model to obtain the phoneme data of the sample song, and then each phoneme in the phoneme data of the sample song is randomly masked to obtain masked phoneme sample data.

[0046] The random phoneme processing for each sample song involves randomly selecting the second phoneme data to be processed from all phoneme sample data of that sample song. Furthermore, the randomly selected second phoneme data needs to be replaced or masked to predict its predicted phoneme vector. The phoneme processing model is then trained by calculating the difference between the actual phoneme vectors of the masked and replaced phonemes and the predicted phoneme vectors.

[0047] Specifically, random phoneme processing is performed on the phoneme sample data of each sample song to obtain second phoneme data, including: randomly selecting multiple candidate phonemes with a preset processing ratio from the phoneme sample data; selecting masked phonemes with a first preset ratio and replaced phonemes with a second preset ratio from the multiple candidate phonemes; masking the masked phonemes in the phoneme sample data of each sample song and replacing the replaced phonemes to obtain the second phoneme data.

[0048] Among the multiple candidate phonemes, there is also a third preset ratio of original phoneme data. The original phoneme data represents data that is not processed in any way, and together with the masked phoneme and the replaced phoneme, it constitutes the second phoneme data.

[0049] In other words, during the training process of the phoneme processing model, multiple candidate phonemes with a preset processing ratio need to be randomly selected from the phoneme sample data of each sample song. In order to filter out the masked phonemes and the replaced phonemes from the multiple candidate phonemes, the original phoneme data other than the masked and replaced phonemes in the multiple candidate phonemes remain unchanged. In this way, the masked phonemes, the replaced phonemes, and the original phoneme data are used as the second phoneme data after random phoneme processing.

[0050] Furthermore, from a plurality of candidate phonemes with a preset processing ratio, candidate phonemes of a first preset ratio are randomly selected as the masked phonemes, and candidate phonemes of a second preset ratio are selected as the replaced phonemes. For example, the preset processing ratio can be controlled between 10% and 15%, the first preset ratio can be selected as 80%, and the second preset ratio can be selected as 10%. In this way, 10% of the candidate phonemes with the preset processing ratio remain unprocessed, and the phonemes other than those with the preset processing ratio and the remaining unprocessed candidate phonemes are used as the original phoneme data that remains unchanged.

[0051] In other words, the product of the preset processing ratio and the first preset ratio is used as the masking ratio of the masked phoneme, and the product of the preset processing ratio and the second preset ratio is used as the replacement ratio of the replaced phoneme. Furthermore, both the masking ratio and the replacement ratio should not be too high, as excessively high masking and replacement ratios can easily lead to difficulties in model prediction; therefore, the preset processing ratio should also not be too high. The masked phoneme can be masked using a special symbol [mask]. During training, the phoneme processing model should predict the masked phonemes and guess whether each phoneme not marked with [mask] is the transformed phoneme, thereby enhancing the model's generalization ability.

[0052] Furthermore, the phonemes to be processed should include both masked and replaced phonemes in the second phoneme data. Since the original phoneme data has not been masked or replaced, the phonemes to be processed do not include the original phoneme data. Therefore, the phoneme processing model should be trained during the training process to recognize replaced phonemes and should predict the original phoneme sample data corresponding to the replaced phonemes.

[0053] Specifically, the phoneme processing model predicts the phoneme data of the processed portion, and iteratively trains at least one model parameter in the phoneme processing model based on the difference between the predicted phoneme vector and the actual phoneme vector of the processed phoneme in the second phoneme data. This includes: mapping each phoneme in the second phoneme data to a preset dimension space to determine the initial phoneme vector corresponding to each sample song; predicting the predicted phoneme vector based on the initial phoneme vector; establishing a first loss function between the predicted phoneme vector and the actual phoneme vector; and adjusting the model parameters of the phoneme processing model according to the loss value of the first loss function until the phoneme processing model meets the preset training completion conditions.

[0054] In other words, after processing the phoneme sample data of each sample song randomly to obtain the second phoneme data, each phoneme in the second phoneme data is mapped to a preset dimensional space to obtain the initial phoneme vector. Furthermore, in order for the model to correctly predict masked and replaced phonemes, the phoneme embedding layer needs to bring phonemes with similar pronunciations closer together in the vector space, thus requiring the learning of vector representations that can effectively distinguish different phonemes. Simultaneously, the mapping to the vector space also needs to contain sufficient semantic and phonetic information so that the subsequent context encoder can understand it.

[0055] Furthermore, the initial phoneme vector of each sample song output from the phoneme embedding layer is input into the context encoder. The context editor generates an implicit sequence representation for each sample song based on the contextual information corresponding to each phoneme in the initial phoneme vector. The implicit sequence representation includes the hidden state of each phoneme in the sample song, and the context editor learns long-distance dependencies during training to ensure that the implicit sequence representation carries sufficient contextual information for subsequent inference of the processed phonemes. Therefore, the context editor needs to learn how to capture long-distance contextual information from unmasked phonemes during training to infer the processed phonemes through this contextual information. A good context encoder can generate high-quality implicit sequence representations containing rich semantics.

[0056] The phoneme processing model also includes a decoder and an output layer. For example, during training, the decoder and output layer are used to generate predicted phoneme vectors for the phonemes being processed, and to map the predicted phoneme vectors to a phoneme probability distribution to calculate the cross-entropy loss, so as to modify the various model parameters through backpropagation.

[0057] Next, the implicit sequence representation of each sample song is input into the attention module. The attention module calculates the global context vector corresponding to each implicit sequence representation by weighted aggregation according to the attention mechanism. Furthermore, the attention module should gradually modify the model parameters during training, learning to focus on key phonemes that are most helpful for prediction accuracy from the macroscopic perspective of the context information corresponding to the phonemes. That is, higher weights are assigned to the parts that are iconic and play a core role in identifying the pronunciation of the entire song, and lower weights are assigned to non-core parts, achieving fuzzy matching through attention weighting. The global context vector is then input into the decoder for decoding. The decoder maps the global context vector to the vector space of the predicted phoneme vector to gradually generate the predicted phoneme vector of the masked phoneme. The output layer calculates the loss value by using a loss function that supports fuzzy phoneme matching, such as cross-entropy loss, and then uses a backpropagation mechanism to propagate the data backward in reverse order to adjust the parameters of each module. Training of the phoneme processing model stops when the loss value meets the preset training completion condition, resulting in the trained phoneme processing model.

[0058] In this context, adjusting the model parameters of the phoneme processing model through backpropagation refers to all learnable parameters of the model. These learnable parameters include the vector space of the phoneme embedding layer, the weights of the context encoder, the weight matrix for calculating attention scores, the decoder weights, and the output layer weights. For example, modifying the vector space guides phonemes in the correct direction, bringing similar phonemes closer together and dissimilar phonemes further apart. In other words, when a prediction error occurs, a large loss value is calculated. This large loss value is propagated to each module through backpropagation, adjusting the model parameters of the phoneme processing model during training to gradually bring the predicted phoneme vectors closer to the actual phoneme vectors. This drives all modules in the model to learn together in an optimization loop, generating high-quality lyric phoneme representation vectors capable of context-aware and fuzzy training.

[0059] return Figure 1 S103: Perform a first similarity calculation on each phoneme representation vector and classify the first song to be detected into at least one first song category.

[0060] The first similarity calculation is performed as follows: the distance between the phoneme representation vectors of every two first songs to be detected is calculated to determine the similarity matrix of all first songs to be detected; and all first songs to be detected are classified into at least one first song category according to the similarity matrix.

[0061] In other words, for all the first songs to be detected, the distance between every two songs is calculated based on their phoneme representation vectors, thus forming a similarity matrix (N×N, where N refers to the total number of songs to be detected). For example, cosine similarity can be used to calculate the distance between every two songs; this application does not restrict the distance algorithm. Therefore, all the songs to be detected are first divided into at least one song category based on the similarity matrix. Hierarchical clustering or density clustering (DBSCAN, Density-Based Spatial Clustering of Applications with Noise) can be selected for clustering; this application also does not restrict the clustering algorithm.

[0062] Therefore, classifying all first-to-be-detected songs using phoneme representation vectors allows for the categorization of all first-to-be-detected songs into at least one first-song category from a lyric perspective. Each first-song category includes at least one second-to-be-detected song.

[0063] S104: For each first song category, based on the acoustic representation vector corresponding to the second song to be detected in the first song category, the second song to be detected is divided into at least one second song category, and songs belonging to the same second song category are duplicate songs.

[0064] Specifically, for each first song category, based on the acoustic representation vector corresponding to the second song to be detected in the first song category, the second song to be detected is divided into at least one second song category, including: performing a second similarity calculation on the acoustic representation vector corresponding to each second song to be detected in the first song category, so as to divide the second song to be detected in the first song category into at least one second song category.

[0065] Specifically, the first song category is divided into at least one second song category in the following way: the distance between each pair of second songs to be detected is calculated according to the acoustic representation vectors of each pair of second songs to be detected in the first song category, so as to determine the similarity matrix of the second songs to be detected in the first song category; the first song category is divided according to the similarity matrix of the second songs to be detected.

[0066] In other words, after classifying all first songs to be detected into at least one first song category, for each first song category, the distance between every two second songs to be detected in that song category is calculated according to the acoustic representation vector of each second song to be detected in that first song category, so as to obtain the similarity matrix of that first song category. Then, the similarity matrix of that first song category is used to classify all second songs to be detected in that first song category, so as to classify the first song category into at least one second song category.

[0067] For example, cosine similarity can be used to calculate the distance between the acoustic representation vectors of every two second songs to be detected. Hierarchical clustering or density clustering can be selected for the second division. This application does not restrict the distance algorithm and clustering algorithm for the second clustering.

[0068] Furthermore, each second song category includes at least one third song to be detected. If a second song category contains multiple third songs to be detected, then multiple third songs belonging to the same second song category are considered duplicate songs. Thus, after the first segmentation from the perspective of lyrics, a second segmentation is performed from the perspective of music, instead of directly performing music-level grouping on all the first songs to be detected. This reduces the computational load and improves the computational efficiency and robustness of the clustering results.

[0069] For example, an acoustic representation vector for each first song to be detected can be generated based on fixed-dimensional musical features, including rhythm, melody, timbre, vocal patterns, and chords. Normalization can be performed after identifying each musical feature to obtain the acoustic representation vector of the first song to be detected. This can typically be achieved by calculating statistical measures for each musical feature, such as mean, variance, peak value, trough value, and peak-to-trough difference, etc., and this application does not impose any limitations on this.

[0070] For example, before determining the acoustic representation vector of the first song to be detected, preprocessing can be performed on the first song to be detected to improve audio quality and increase the accuracy of music feature extraction.

[0071] Furthermore, the technical solution of this application can perform deduplication processing on the training samples of the music model, and can also identify pirated songs on music playback software, thus solving the technical problem of poor accuracy in identifying duplicate songs in the prior art and achieving the technical effect of increasing the accuracy of identifying duplicate songs.

[0072] Based on the same application concept, this application also provides a song duplication detection device corresponding to the song duplication detection method provided in the above embodiments. Since the principle of the device in this application is similar to the song duplication detection method in the above embodiments of this application, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be described again.

[0073] like Figure 3 As shown, Figure 3 This is a functional block diagram of a song duplication detection device provided in an embodiment of this application. The song duplication detection device 10 includes: an acquisition module 101, which acquires at least one first song to be detected and text data corresponding to the first song to be detected; a conversion module 102, which converts the text data into phoneme representation vectors and encodes the first song to be detected into acoustic representation vectors; a first classification module 103, which performs a first similarity calculation on each phoneme representation vector and classifies the first song to be detected into at least one first song category; and a second classification module 104, which, for each first song category, classifies the second song to be detected into at least one second song category based on the acoustic representation vector corresponding to the second song to be detected in the first song category, and songs belonging to the same second song category are duplicate songs.

[0074] Based on the same application concept, see [link / reference] Figure 4 The diagram shown is a structural schematic of an electronic device provided in an embodiment of this application. The electronic device 20 includes a processor 201, a memory 202, and a bus 203. The memory 202 stores machine-readable instructions that can be executed by the processor 201. When the electronic device 20 is running, the processor 201 and the memory 202 communicate through the bus 203. When the machine-readable instructions are executed by the processor 201, the steps of the song repeat detection method described in any of the above embodiments are executed.

[0075] Specifically, when the machine-readable instructions are executed by the processor 201, they can perform the following processing: acquiring at least one first song to be detected and text data corresponding to the first song to be detected; converting the text data into phoneme representation vectors and encoding the first song to be detected into acoustic representation vectors; performing a first similarity calculation on each phoneme representation vector and classifying the first song to be detected into at least one first song category; for each first song category, based on the acoustic representation vector corresponding to the second song to be detected in the first song category, classifying the second song to be detected into at least one second song category, and songs belonging to the same second song category are duplicate songs.

[0076] Based on the same concept, this application also provides a computer-readable storage medium storing a computer program, which, when run by a processor, executes the steps of the song duplication detection method provided in the above embodiments.

[0077] Specifically, the storage medium can be a general-purpose storage medium, such as a portable disk or hard disk. When the computer program on the storage medium is run, it can execute the above-mentioned song duplication detection method. By converting the text data of the first song to be detected into a phoneme representation vector and encoding the first song to be detected to obtain an acoustic representation vector, the similarity of the first song to be detected is calculated using the phoneme representation vector to classify it into a first song category. Then, according to the acoustic representation vector, the second song to be detected in the first song category is classified into a second song category, so that the songs in the second song category are identified as duplicate songs. This solves the technical problem of poor accuracy in duplicate song identification in the prior art and achieves the technical effect of increasing the accuracy of duplicate song identification.

[0078] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems and devices described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. In the several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division; in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Another point is that the displayed or discussed mutual coupling or direct coupling or communication connection may be through some communication interfaces; the indirect coupling or communication connection of devices or units may be electrical, mechanical, or other forms.

[0079] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0080] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0081] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0082] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A song detection method, characterized in that, The method includes: Obtain at least one first song to be detected and the text data corresponding to the first song to be detected. The text data includes the official lyrics text data corresponding to the first song to be detected that was uploaded in advance, or the recognized lyrics text data extracted from the first song to be detected. The text data is converted into a phoneme representation vector, and the first song to be detected is encoded into an acoustic representation vector. Perform a first similarity calculation on each phoneme representation vector and classify the first song to be detected into at least one first song category; For each first song category, based on the acoustic representation vector corresponding to the second song to be detected in that first song category, the second song to be detected is divided into at least one second song category, and songs belonging to the same second song category are duplicate songs; Converting the text data into a phoneme representation vector includes: The text data of each first song to be detected is converted into several phoneme data; A phoneme processing model is obtained, and the phoneme data is mapped to a preset dimension space. For each phoneme data, encoding processing is performed on the phoneme data according to the context information and phoneme dependency relationship of the phoneme data. The encoding results of all phoneme data of each first song to be detected are summarized to obtain at least one phoneme embedding representation sequence for each first song to be detected. Fuzzing is performed on each phoneme embedding sequence to obtain the phoneme representation vector of each first song to be detected.

2. The method according to claim 1, characterized in that, Enigma processing is performed on each phoneme embedding sequence to obtain the phoneme representation vector for each first song to be detected, including: An attention mechanism is performed on the at least one phoneme embedding representation sequence. The corresponding attention weight is calculated based on the attention score of each phoneme embedding representation sequence. The phoneme embedding representation sequences are fused according to the attention weights, and the fused sequence is decoded and restored to obtain the phoneme representation vector of each first song to be detected.

3. The method according to claim 2, characterized in that, Decode and reconstruct the fused sequence to obtain the phoneme representation vector of each first song to be detected, including: For a first song to be detected, the phoneme processing model uses the location information of all phoneme data as the query index, and generates the phoneme probability distribution of the location information corresponding to each phoneme data according to the attention score and fusion sequence. The phoneme embedding representation sequence corresponding to the phoneme with the highest probability is used as the phoneme feature of each phoneme data to form the phoneme representation vector of the first song to be detected.

4. The method according to claim 1, characterized in that, The phoneme processing model is trained in the following manner: Obtain a song training sample set, which includes at least one sample song and the corresponding text sample data for each sample song; Each text sample data is converted into phoneme sample data, and the phoneme sample data of each sample song is processed by random phoneme processing to obtain the second phoneme data. The phoneme processing model is instructed to predict the phoneme data of the processed portion, and at least one model parameter in the phoneme processing model is iteratively trained based on the difference between the predicted phoneme vector and the actual phoneme vector of the processed phoneme in the second phoneme data.

5. The method according to claim 4, characterized in that, The phoneme processing model is instructed to predict the phoneme data of the processed portion, and based on the difference between the predicted phoneme vector and the actual phoneme vector of the processed phoneme in the second phoneme data, at least one model parameter in the phoneme processing model is iteratively trained, including: Each phoneme in the second phoneme data is mapped to a preset dimension space to determine the initial phoneme vector corresponding to each sample song; The predicted phoneme vector is obtained based on the initial phoneme vector; A first loss function is established between the predicted phoneme vector and the actual phoneme vector; The model parameters of the phoneme processing model are adjusted according to the loss value of the first loss function until the phoneme processing model meets the preset training completion conditions.

6. The method according to claim 4, characterized in that, For each sample song, the phoneme sample data is processed using random phoneme processing to obtain the second phoneme data, including: Multiple candidate phonemes with a preset processing ratio are randomly selected from the phoneme sample data; Select a first preset proportion of masked phonemes and a second preset proportion of replaced phonemes from the plurality of candidate phonemes. The masked phonemes in the phoneme sample data of each sample song are masked, and the replaced phonemes are replaced to form the second phoneme data.

7. The method according to claim 6, characterized in that, Among the multiple candidate phonemes, there is also a third preset ratio of original phoneme data. The original phoneme data represents data for which no processing is performed, and together with the masked phoneme and the replaced phoneme, it constitutes the second phoneme data.

8. The method according to claim 1, characterized in that, For each first song category, based on the acoustic representation vector corresponding to the second song to be detected in that first song category, the second song to be detected is divided into at least one second song category, including: A second similarity calculation is performed on the acoustic representation vector corresponding to each second song to be detected in the first song category to classify the second songs to be detected in the first song category into at least one second song category.

9. A song repetition detection device, characterized in that, The device includes: The acquisition module acquires at least one first song to be detected and the text data corresponding to the first song to be detected. The text data includes the official lyrics text data corresponding to the first song to be detected that was uploaded in advance, or the recognized lyrics text data extracted from the first song to be detected. The conversion module converts the text data into a phoneme representation vector and encodes the first song to be detected into an acoustic representation vector; The first classification module performs a first similarity calculation on each phoneme representation vector and classifies the first song to be detected into at least one first song category; The second classification module, for each first song category, divides the second song to be detected into at least one second song category based on the acoustic representation vector corresponding to the second song to be detected in the first song category. Songs belonging to the same second song category are duplicate songs. The conversion module converts the text data into a phoneme representation vector, including: The text data of each first song to be detected is converted into several phoneme data; A phoneme processing model is obtained, and the phoneme data is mapped to a preset dimension space. For each phoneme data, encoding processing is performed on the phoneme data according to the context information and phoneme dependency relationship of the phoneme data. The encoding results of all phoneme data of each first song to be detected are summarized to obtain at least one phoneme embedding representation sequence for each first song to be detected. Fuzzing is performed on each phoneme embedding sequence to obtain the phoneme representation vector of each first song to be detected.

Images