Music similarity detection method, device, electronic device and storage medium

Abstract

The present invention relates to the field of audio processing technology, and provides a method, device, electronic device, and storage medium for detecting music similarity. By preprocessing a music clip to be detected, the CQT features of the music clip to be detected are obtained; the CQT features of the music clip to be detected are input into a preset model to obtain a first feature vector of the music clip to be detected; and similarity detection is then performed on the music clip to be detected based on the first feature vector and a pre-established music library. By reflecting the characteristics of the music clip to be detected through the CQT features and feature vectors, and performing similarity detection in combination with the music library, automatic detection is achieved, shortening the review cycle and improving review efficiency.

Description

Technical Field

[0001] The present invention relates to the field of audio processing technology, and in particular to a music similarity detection method, device, electronic device and storage medium. Background Art

[0002] With the continuous development of technology, users can upload their own works, such as videos or audio, to platforms for publication. Platforms often review the music in user-uploaded content to avoid copyright infringement risks. Currently, manual review is often used, which results in long review cycles and low efficiency. Summary of the Invention

[0003] In view of this, an object of the present invention is to provide a music similarity detection method, device, electronic device and storage medium.

[0004] In order to achieve the above objectives, the technical solutions adopted in the embodiments of the present invention are as follows:

[0005] In a first aspect, the present invention provides a method for detecting music similarity, the method comprising:

[0006] Preprocessing the music clip to be detected to obtain the CQT features of the music clip to be detected;

[0007] Inputting the CQT features of the music clip to be detected into a preset model to obtain a first feature vector of the music clip to be detected;

[0008] A similarity detection is performed on the music segment to be detected based on the first feature vector and a pre-established music library.

[0009] In an optional embodiment, the step of inputting the CQT features of the music segment to be detected into a preset model to obtain a first feature vector of the music segment to be detected includes:

[0010] Inputting the CQT features of the music clip to be detected into the convolutional network of the preset model and performing convolution processing to obtain multiple feature maps;

[0011] Performing feature regularization processing on the multiple feature maps using the feature regularization layer of the preset model to obtain a regularization vector;

[0012] The regularized vector is fully connected using the fully connected layer of the preset model to obtain the first feature vector.

[0013] In an optional embodiment, the step of performing feature regularization processing on the multiple feature maps using the feature regularization layer of the preset model to obtain a regularized vector includes:

[0014] Using the feature regularization layer of the preset model, each of the feature maps is converted into a one-dimensional tensor and concatenated to obtain a concatenated feature map matrix;

[0015] Transposing the concatenated feature map matrix to obtain a transposed matrix;

[0016] Multiplying the concatenated feature map matrix by the transposed matrix to obtain a regularized feature map matrix;

[0017] The largest element is selected from each row of the regularized feature map matrix to obtain the regularized vector.

[0018] In an optional embodiment, the music library includes second feature vectors of a plurality of audio clips;

[0019] The step of performing similarity detection on the music clip to be detected based on the first feature vector and a pre-established music library includes:

[0020] Calculating the cosine similarity between the first eigenvector and each of the second eigenvectors;

[0021] If the cosine similarity is greater than a preset threshold, it is determined that the music clip to be detected has an infringement risk;

[0022] If all cosine similarities are less than the preset threshold, it is determined that the music clip to be detected does not have an infringement risk.

[0023] In an optional embodiment, before the step of preprocessing the music clip to be detected to obtain the CQT features of the music clip to be detected, the method further includes:

[0024] Obtaining work content based on user interaction operations on the content upload interface;

[0025] The music segment to be detected is extracted from the content of the work.

[0026] In an optional embodiment, the music library includes names of multiple music clips, and the method further includes:

[0027] If the music clip to be detected has an infringement risk after similarity detection, obtaining the name of the target music clip that has an infringement relationship with the music clip to be detected;

[0028] The name of the target music clip and prompt information are displayed on the review result interface, and the prompt information is used to remind the user that there is a risk of infringement of the content of the work.

[0029] In an optional embodiment, the preset model is obtained in the following manner:

[0030] Acquire multiple music audio samples, and divide each of the music audio samples into multiple music segment samples in chronological order to obtain each of the music segment samples;

[0031] Wherein, any two adjacent music clip samples in the music audio samples have the same audio frame;

[0032] Preprocessing each of the music clip samples to obtain a CQT feature of each of the music clip samples;

[0033] Performing dimensionality reduction and cluster analysis on the CQT features of all music clip samples belonging to the same music audio sample to obtain a label for each of the music clip samples;

[0034] According to the preset loss function and the label of each music clip sample, the basic model is trained to obtain the preset model.

[0035] In an optional embodiment, the music library is obtained in the following manner:

[0036] Acquire multiple music audios to be stored in the library; each of the music audios to be stored in the library has a name and includes multiple music clips;

[0037] Preprocessing each of the music clips to obtain a CQT feature of each of the music clips;

[0038] Inputting the CQT feature of each of the music clips into the preset model to obtain a second feature vector of each of the music clips;

[0039] Obtaining an identifier of each second feature vector using a vector search tool;

[0040] A correspondence is established between the name of the music audio to be stored to which each music clip belongs, the second feature vector of each music clip, and the identifier of each second feature vector to obtain the music library.

[0041] In a second aspect, the present invention provides a music similarity detection device, the device comprising:

[0042] A preprocessing module, configured to preprocess the music clip to be detected to obtain the CQT features of the music clip to be detected;

[0043] a processing module, configured to input the CQT features of the music segment to be detected into a preset model to obtain a first feature vector of the music segment to be detected;

[0044] The detection module is used to perform similarity detection on the music segment to be detected based on the first feature vector and a pre-established music library.

[0045] In a third aspect, the present invention provides an electronic device, comprising a processor and a memory, wherein the memory stores a computer program, and when the processor executes the computer program, the method described in any one of the aforementioned embodiments is implemented.

[0046] In a fourth aspect, the present invention provides a storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in any one of the aforementioned embodiments.

[0047] The music similarity detection method, device, electronic device, and storage medium provided by embodiments of the present invention preprocess the music clip to obtain CQT features of the music clip to be detected. The CQT features of the music clip to be detected are then input into a preset model to obtain a first feature vector of the music clip to be detected. Similarity detection is then performed on the music clip to be detected based on the first feature vector and a pre-established music library. By using the CQT features and feature vectors to reflect the characteristics of the music clip to be detected and combining them with the music library for similarity detection, automatic detection is achieved, shortening the review cycle and improving review efficiency.

[0048] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without paying any creative work.

[0050] Figure 1 A block diagram of an electronic device provided by an embodiment of the present invention is shown;

[0051] Figure 2 A schematic diagram showing a flow chart of a music similarity detection method provided by an embodiment of the present invention;

[0052] Figure 3 Another schematic flow chart of a method for detecting music similarity according to an embodiment of the present invention is shown;

[0053] Figure 4 An example diagram of a music similarity detection method provided by an embodiment of the present invention is shown;

[0054] Figure 5 Another example diagram of the music similarity detection method provided by an embodiment of the present invention is shown;

[0055] Figure 6 Another schematic flow chart of a method for detecting music similarity according to an embodiment of the present invention is shown;

[0056] Figure 7 Another example diagram of the music similarity detection method provided by an embodiment of the present invention is shown;

[0057] Figure 8 Another example diagram of the music similarity detection method provided by an embodiment of the present invention is shown;

[0058] Figure 9 A functional module diagram of a music similarity detection device provided by an embodiment of the present invention is shown.

[0059] Icon: 100 - electronic device; 110 - bus; 120 - processor; 130 - memory; 150 - I / O module; 170 - communication interface; 300 - music similarity detection device; 310 - preprocessing module; 330 - processing module; 350 - detection module; 370 - display module; 390 - construction module. DETAILED DESCRIPTION

[0060] The following will be combined with the accompanying drawings to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings herein can be arranged and designed in various different configurations.

[0061] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention as claimed, but is merely intended to represent selected embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative work are within the scope of protection of the present invention.

[0062] It should be noted that relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus comprising the element.

[0063] With the continuous development of technology, users can upload their own works, such as videos or audio, to platforms for publication. Platforms usually review the music in user-uploaded content to avoid the risk of copyright infringement. For example, a user can upload their own cover song to the platform. If the platform does not have the copyright to the song, it will constitute copyright infringement. Currently, manual review is commonly used, but this method has a long review cycle, low efficiency, and consumes a lot of labor costs. Therefore, the embodiments of the present invention provide a music similarity detection method to solve the above problems.

[0064] Please refer to Figure 1 , is a block diagram of an electronic device 100 provided by an embodiment of the present invention. The electronic device 100 includes a bus 110 , a processor 120 , a memory 130 , an I / O module 150 , and a communication interface 170 .

[0065] The bus 110 may be a circuit that connects the above elements to each other and transfers communications (eg, control messages) between the above elements.

[0066] The processor 120 may receive commands from the other components (eg, the memory 130 , the I / O module 150 , the communication interface 170 , etc.) via the bus 110 , may interpret the received commands, and may perform calculations or data processing according to the interpreted commands.

[0067] The processor 120 can be an integrated circuit chip with signal processing capabilities. The processor 120 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0068] The memory 130 may store commands or data received from the processor 120 or other elements (eg, the I / O module 150 , the communication interface 170 , etc.) or commands or data generated by the processor 120 or other elements.

[0069] The memory 130 can be, but is not limited to, a random access memory (RAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), or an electrically erasable programmable read-only memory (EEPROM).

[0070] The I / O module 150 can receive commands or data input from the user via input-output means (e.g., sensors, keyboards, touch screens, etc.), and can transmit the received commands or data to the processor 120 or the memory 130 via the bus 110. It is also used to display various information (e.g., multimedia data, text data) received, stored, and processed by the above components, and can display videos, images, data, etc. to the user.

[0071] The communication interface 170 may be used to communicate signaling or data with other node devices.

[0072] It is understandable that Figure 1 The structure shown is only a schematic diagram of the structure of the electronic device 100. The electronic device 100 may also include Figure 1 More or fewer components than shown, or with Figure 1 Different configurations shown. Figure 1 Each component shown in the figure can be implemented by hardware, software or a combination thereof.

[0073] The electronic device provided in the embodiment of the present invention may be a smart phone, a personal computer, a tablet computer, a wearable device, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), etc. The embodiment of the present invention does not impose any limitation on this.

[0074] The above-mentioned electronic device will be used as the execution subject to execute each step of each method provided in the embodiment of the present invention and achieve corresponding technical effects.

[0075] See also Figure 2 , Figure 2 It is a flowchart of a music similarity detection method provided by an embodiment of the present invention.

[0076] Step S202: pre-process the music clip to be detected to obtain the CQT features of the music clip to be detected;

[0077] In this embodiment, a music clip to be detected can be extracted from the music contained in the work content uploaded by the user, such as a video or audio; then the music clip to be detected is preprocessed, such as resampling and constant Q transform, and the CQT feature (Constant Q Transform) of the music clip to be detected is obtained after the constant Q transform.

[0078] It's understandable that all musical notes are composed of several octaves of equal temperament, and these sounds are exponentially distributed. The Constant Q Transform (CQT) can make the center frequencies follow an exponential distribution, meaning that the CQT feature has the same distribution as the scale frequencies. Consequently, using the CQT feature can better express the characteristics of music.

[0079] Step S204: inputting the CQT features of the music clip to be detected into a preset model to obtain a first feature vector of the music clip to be detected;

[0080] In this embodiment, a preset model is trained using a large number of audio clip samples to extract feature vectors for music clips. Based on the obtained CQT features of the music clip to be detected, these features are input into the preset model and feature extraction is performed to obtain a feature vector for the music clip to be detected, i.e., a first feature vector.

[0081] Step S206: performing similarity detection on the music clip to be detected based on the first feature vector and the pre-established music library;

[0082] In this embodiment, a music library can be pre-established, and based on the obtained first feature vector of the audio segment to be detected and the music library, a similarity test is performed on the music segment to be detected, that is, it is determined whether there is a music segment similar to the music segment to be detected in the music library.

[0083] As can be seen from the above steps, the music clip to be tested is preprocessed to obtain its CQT features. These CQT features are then input into a preset model to obtain a first feature vector for the music clip to be tested. Similarity testing is then performed on the music clip to be tested based on this first feature vector and a pre-established music library. By using the CQT features and feature vectors to characterize the music clip to be tested, and combining them with the music library for similarity testing, automated testing is achieved, shortening the review cycle and improving review efficiency.

[0084] Optionally, for the preset model in the above embodiment, the present invention provides a method for obtaining the preset model, please refer to Figure 3 .

[0085] Step S212: Acquire multiple music audio samples, and divide each music audio sample into multiple music segment samples in chronological order to obtain each music segment sample;

[0086] Any two adjacent music clip samples in the music audio samples have the same audio frame.

[0087] In this embodiment, a plurality of music audio samples are obtained, and each music audio sample is divided into a plurality of music segment samples of preset duration in chronological order, and any two adjacent music segment samples in a music audio sample include the same audio frame.

[0088] For example, a music audio sample can be divided into multiple music segment samples with a duration of 28 seconds, and two adjacent music segment samples have the same audio frames, such as a 10-second overlap. Alternatively, if the last segment of the music audio sample is less than 28 seconds, the last 28 seconds of the music audio sample can be used as a music segment sample. It should be understood that the duration of the music segment sample and the overlap duration of two adjacent music segment samples can be set according to actual application and are not limited in this embodiment of the present invention.

[0089] Each music audio sample is divided to obtain each audio segment sample. Each audio segment sample can be marked with a music tag to record the audio segment sample to which each music segment sample belongs.

[0090] Step S214: pre-process each music clip sample to obtain a CQT feature of each music clip sample;

[0091] In this embodiment, for each music clip sample, preprocessing such as resampling, constant Q transformation and feature downsampling is performed on the music clip sample to obtain the CQT feature of each music clip sample.

[0092] Step S216: performing dimensionality reduction processing and cluster analysis on the CQT features of all music clip samples belonging to the same music audio sample to obtain a label for each music clip sample;

[0093] In this embodiment, multiple segment groups can be obtained based on the music tag of each audio segment sample, and one segment group includes all audio segment samples of a music audio sample.

[0094] For all music clip samples belonging to the same music audio sample, that is, each clip, dimensionality reduction processing such as PAC dimensionality reduction and cluster analysis is performed on the CQT features of all music clip samples in this clip group. Then, based on the clustering results of this clip group, a label is obtained for all music clip samples in this clip group. This label is used to indicate the category of the music audio sample to which the music clip sample belongs. By traversing each clip group, the label of each music clip sample is obtained.

[0095] It's understandable that a piece of music, such as a song, typically has a verse and a chorus. The melodies of the verse and chorus are different, and the lyrics of the same part may also be different. Cluster analysis is used to classify and label multiple music clips within a music audio sample. The labels of the music clips can then be used to represent the various parts of the music audio sample.

[0096] Step S218: training the basic model according to the preset loss function and the label of each music clip sample to obtain a preset model;

[0097] In this embodiment, the basic model includes a convolutional network, a feature regularization layer, and a fully connected layer. The preset loss function can be a cross-entropy loss function. For ease of understanding, the embodiment of the present invention provides an example diagram of the structure of the basic model.

[0098] See also Figure 4 The basic model includes a convolutional network, a feature regularization layer, and a fully connected network. The convolutional network includes multiple convolutional blocks, such as four convolutional blocks. Optionally, the number of convolutional blocks can be set according to the actual application. Each convolutional block includes a convolutional layer, a normalization layer, and a pooling layer. The fully connected network includes two fully connected layers.

[0099] The CQT features and labels of each music clip sample are input into the basic model, and the basic model is trained until the preset conditions are met, such as the number of training iterations reaches the preset number or the value of the preset loss function reaches the preset value, and the parameters of the convolutional network, feature regularization layer and fully connected network are obtained, that is, the preset model is obtained.

[0100] Optionally, based on the preset model obtained in the above embodiment, for the above step S204, the embodiment of the present invention provides a possible implementation method.

[0101] Step S204-1: Input the CQT features of the music clip to be detected into a convolutional network of a preset model and perform convolution processing to obtain multiple feature maps;

[0102] In this embodiment, the CQT features of the music clip to be detected are input into the convolutional network of the preset model, and each convolution block in the convolutional network is used to perform convolution processing on the CQT features to obtain each feature map. Figure 4 The convolutional network includes 4 convolution blocks and obtains 4 feature maps.

[0103] Step S204-3, performing feature regularization processing on the multiple feature maps using the feature regularization layer of the preset model to obtain a regularized vector;

[0104] In this embodiment, each feature map output by the convolutional network is input into the feature regularization layer of the preset model, and the feature regularization layer is used to perform feature regularization processing on all feature maps to obtain a regularized vector.

[0105] Feature regularization processing can extract effective features and reduce the order of magnitude of features, which can effectively avoid model overfitting and underfitting, thereby improving the generalization ability of the model and making the model better fit the actual application scenario.

[0106] Step S204-5: Use the fully connected layer of the preset model to perform full connection processing on the regularized vector to obtain a first eigenvector.

[0107] In this embodiment, the regularization vector output by the feature regularization layer is input into the first fully connected layer of the preset model. Figure 4 The fully connected layer 1 shown is used to perform full connection processing on the regularization vector, that is, to obtain the feature vector of the audio segment to be detected, that is, the first feature vector.

[0108] It is understandable that Figure 4 The fully connected layer 2 is used to classify audio clips based on the feature vectors output by the fully connected layer 1. However, in the process of detecting similarity between audio clips, it is not necessary to obtain the category of the audio clip to be detected. In fact, the feature vector of the audio clip to be detected is extracted. In the similarity detection scenario, the output of the fully connected layer 1 is obtained.

[0109] It can be understood that during training, fully connected layer 2 is used to classify the feature vectors obtained after processing by the convolutional network, feature regularization layer, and fully connected layer 1, obtaining a classification result. Based on this classification result and the loss function, parameters are adjusted and the termination criteria for model training are determined. In other words, fully connected layer 2 is necessary during training, but during model use, since the purpose of this invention is similarity detection rather than classification, only the output of fully connected layer 1 is required.

[0110] Optionally, for the above step S204-3, an embodiment of the present invention provides a possible implementation method.

[0111] Step S204-3-1, using the feature regularization layer of the preset model to convert each feature map into a one-dimensional tensor and splice them to obtain a spliced ​​feature map matrix;

[0112] Step S204-3-3, transpose the concatenated feature map matrix to obtain a transposed matrix;

[0113] Step S204-3-5, multiplying the concatenated feature map matrix by the transposed matrix to obtain a regularized feature map matrix;

[0114] Step S204-3-7, select the largest element from each row of the regularized feature map matrix to obtain a regularized vector.

[0115] For ease of understanding, an example diagram is provided in this embodiment of the present invention. Figure 5 , the following will be combined Figure 5 The above steps S204-3-1 to S204-3-7 are explained.

[0116] For example, after convolving the CQT features of the audio clip to be detected through four convolutional blocks, four feature maps are obtained and each feature map is input into the feature regularization layer. First, each feature map is converted into a one-dimensional tensor, and a one-dimensional tensor is used as a row. These four one-dimensional tensors are concatenated to obtain a concatenated feature map matrix. Then, the concatenated feature map matrix is ​​transposed to obtain a transposed matrix. The concatenated feature map matrix and the transposed matrix are matrix-multiplied to obtain a regularized feature map matrix. Finally, the maximum value is selected from each row of the regularized feature map matrix to obtain the regularized vector.

[0117] Optionally, for the music library in the above embodiment, the present invention provides a method for obtaining the music library, see Figure 6 .

[0118] Step S222: Acquire multiple music audios to be stored in the library; each music audio to be stored in the library has a name and includes multiple music clips;

[0119] In this embodiment, multiple music audios to be stored and their names can be obtained, and each music audio to be stored can be divided into multiple music segments of preset length in chronological order, and two adjacent music segments in a music audio to be stored include the same audio frame.

[0120] It is understandable that the duration of the three music clips, namely the music clips to be stored in the music audio, the music clip samples used in the training model, and the music clips to be detected, can be set to the same value, and the overlapping duration of the three music clips can also be set to the same value to improve the accuracy of similarity detection.

[0121] Step S224, pre-processing each music clip to obtain CQT features of each music clip;

[0122] Step S226, inputting the CQT features of each music clip into a preset model to obtain a second feature vector of each music clip;

[0123] In this embodiment, each music clip is preprocessed by performing resampling, constant Q transform, and feature downsampling on the music clip sample to obtain a CQT feature for each music clip. The CQT feature for each music clip is then input into a preset model to obtain a feature vector for each music clip, namely, a second feature vector.

[0124] Step S228, obtaining the identifier of each second eigenvector through a vector search tool;

[0125] In this embodiment, the vector search tool may be milvus, and each second eigenvector is input into the vector search tool. The vector search tool assigns a unique identifier to each second eigenvector, thereby obtaining the identifier of each second eigenvector.

[0126] Step S230: establishing a correspondence between the name of the music audio to be stored to which each music clip belongs, the second feature vector of each music clip, and the identifier of each second feature vector, thereby obtaining a music library;

[0127] In this embodiment, the name of the music audio to be stored in the library to which each music clip belongs is used as the name of the music clip, and then a correspondence is established between the name of each music clip, the second feature vector of each music clip, and the identifier of each second feature vector, thereby obtaining a music library.

[0128] For ease of understanding, this embodiment of the present invention uses the song "True Hero" as an example of music audio to be stored. First, the song is divided into 20 music segments, such as seg1, seg2, ..., seg20. Each music segment is 28 seconds long, and the overlap between two adjacent music segments is 10 seconds. These 20 music segments are then preprocessed to obtain CQT features for each music segment. These CQT features are then input into a preset model to obtain a second feature vector for each music segment, such as emb1, emb2, ..., emb20.

[0129] Then, using a vector search tool, we obtain the identifiers of each second eigenvector, such as id1, id2, ..., id20. Finally, we create the name of each audio clip, "True Hero," and establish a corresponding relationship between the second eigenvector of each music clip and its identifier, namely, emb1-id1-True Hero, emb2-id2-True Hero, ..., emb20-id20-True Hero. This completes the addition of the song "True Hero" to the library. It is understood that a similar addition operation is performed for each piece of music to be added to the library to create a music library.

[0130] Optionally, based on the music library obtained in the above embodiment, for the above step S206, the embodiment of the present invention provides a possible implementation method.

[0131] Step S206-1, calculating the cosine similarity between the first eigenvector and each second eigenvector;

[0132] In this embodiment, the music library includes a second feature vector for each music clip. Based on the obtained first feature vector of the audio clip to be detected, the cosine similarity between the first feature vector and each second feature vector in the music library is calculated, i.e., the cosine angle between the two vectors, to obtain each cosine similarity. The cosine similarity indicates the degree of similarity between the music clip to be detected and the music clips in the music library.

[0133] Step S206-3: If the cosine similarity is greater than a preset threshold, it is determined that the music clip to be detected has an infringement risk;

[0134] In this embodiment, if the cosine similarity is greater than a preset threshold, it means that there is a music clip in the music library that is highly similar to the music clip to be detected, and the similarity between the two is higher than the set threshold, then it is determined that the music clip to be detected has an infringement risk.

[0135] Step S206-5: If all cosine similarities are less than the preset threshold, it is determined that the music clip to be detected does not have an infringement risk.

[0136] In this embodiment, if all cosine similarities are not greater than the preset threshold, it means that the similarity between all music clips in the music library and the music clip to be detected is low, and it is determined that the music clip to be detected does not have an infringement risk.

[0137] Optionally, before the above step 202, the following steps may be included: obtaining the content of the work according to the interactive operation of the user on the content upload interface; and extracting the music clip to be detected from the content of the work.

[0138] For ease of understanding, an example diagram is provided for the implementation of the present invention. Figure 7(a) The user performs an interactive operation such as a click operation on the content upload interface, and obtains a local media file list based on the click operation, which includes video and audio; the user selects video A in the media file list, and the following is displayed: Figure 7 In the interface shown in (b), the user enters detailed information of the work content such as name, category and introduction, and clicks the publish control to obtain the work content uploaded by the user, namely video A.

[0139] Then extract the audio from video A and identify it as a segment every 10 seconds. If the segment is recognized as not music, continue to obtain the next segment. If the segment is recognized as music, obtain the last 18 seconds of the current segment, and obtain a music segment with a duration of 28 seconds, which is the music segment to be detected.

[0140] Based on the obtained music clip to be tested, similarity testing is performed on the music clip to be tested according to the above steps S202 and S206. After step 206, the following steps are further included: if the music clip to be tested has an infringement risk after the similarity test, the name of the target music clip that has an infringement relationship with the music clip to be tested is obtained; and the name of the target music clip and a prompt message are displayed on the review result interface, the prompt message being used to inform the user that the work content has an infringement risk.

[0141] For example, if the cosine similarity between the first feature vector of the music clip to be detected and the second feature vector of music clip 1 in the music library is greater than a preset threshold, the music clip to be detected is determined to have an infringement risk, and music clip 1 is the target music clip with an infringement relationship with the music clip to be detected. Alternatively, if there are multiple music clips with cosine similarities greater than the preset threshold, the music clip with the largest cosine similarity is used as the target music clip.

[0142] Then get the name of the music clip 1, such as C, and display the name and prompt information of the music clip 1 on the audit result interface, such as Figure 8 As shown. Users can use the re-upload control in this interface to re-upload the modified work for re-review.

[0143] Optionally, the timestamp corresponding to the first audio frame of the music segment to be detected in video A can be obtained and displayed in the prompt information to facilitate users to quickly locate the segment with infringement risk, thereby improving user experience.

[0144] It is understandable that the music similarity detection method provided by the embodiment of the present invention can also be used in scenarios assisting manual review. If it is determined that the music clip to be detected has an infringement risk, the name of the target music clip that has an infringement relationship with the music clip to be detected can be obtained after step 206, and the timestamp corresponding to the first audio frame of the music clip to be detected in the work content can be obtained; the name and timestamp of the target music clip are fed back to the reviewer, and the reviewer does not need to review the entire content, but can quickly locate the part with infringement risk for review, thereby narrowing the review scope and improving the review efficiency.

[0145] In order to execute the corresponding steps in the above embodiments and various possible methods, a method for implementing a music similarity detection device is given below. Figure 9 , Figure 9 This is a functional block diagram of a music similarity detection device 300 provided in an embodiment of the present invention. It should be noted that the basic principles and technical effects of the music similarity detection device 300 provided in this embodiment are the same as those of the above-mentioned embodiments. For the sake of brevity, any parts not mentioned in this embodiment can be referred to the corresponding contents of the above-mentioned embodiments. The music similarity detection device 300 includes:

[0146] A preprocessing module 310 is used to preprocess the music clip to be detected to obtain the CQT features of the music clip to be detected;

[0147] The processing module 330 is configured to input the CQT features of the music segment to be detected into a preset model to obtain a first feature vector of the music segment to be detected;

[0148] The detection module 350 is configured to perform similarity detection on the music segment to be detected based on the first feature vector and a pre-established music library.

[0149] Optionally, the processing module 330 is specifically used to: input the CQT features of the music clip to be detected into the convolutional network of the preset model and perform convolution processing to obtain multiple feature maps; use the feature regularization layer of the preset model to perform feature regularization processing on the multiple feature maps to obtain a regularized vector; use the fully connected layer of the preset model to perform full connection processing on the regularized vector to obtain a first feature vector.

[0150] Optionally, the processing module 330 is specifically used to: use the feature regularization layer of the preset model to convert each feature map into a one-dimensional tensor and splice it to obtain a spliced ​​feature map matrix; transpose the spliced ​​feature map matrix to obtain a transposed matrix; multiply the spliced ​​feature map matrix by the transposed matrix to obtain a regularized feature map matrix; select the largest element from each row of the regularized feature map matrix to obtain a regularized vector.

[0151] Optionally, the detection module 350 is specifically used to: calculate the cosine similarity between the first eigenvector and each second eigenvector; if there is a cosine similarity greater than a preset threshold, it is determined that the music clip to be detected has an infringement risk; if all cosine similarities are not greater than the preset threshold, it is determined that the music clip to be detected does not have an infringement risk.

[0152] Optionally, the music similarity detection device 300 further includes a display module 370, which is used to: obtain the content of the work according to the user's interactive operation on the content upload interface; and extract the music segment to be detected from the content of the work.

[0153] Optionally, the display module 370 is also used to: if the music clip to be detected has an infringement risk after similarity detection, obtain the name of the target music clip that has an infringement relationship with the music clip to be detected; display the name of the target music clip and prompt information on the review result interface, and the prompt information is used to remind the user that the content of the work has an infringement risk.

[0154] Optionally, the music similarity detection device 300 also includes a construction module 390, which is used to obtain multiple music audio samples, and divide each music audio sample into multiple music segment samples in chronological order to obtain each music segment sample; wherein any two adjacent music segment samples in the music audio sample have the same audio frame; each music segment sample is preprocessed to obtain the CQT features of each music segment sample; the CQT features of all music segment samples belonging to the same music audio sample are subjected to dimensionality reduction processing and cluster analysis to obtain the label of each music segment sample; according to the preset loss function and the label of each music segment sample, the basic model is trained to obtain the preset model.

[0155] Optionally, the construction module 390 is also used to: obtain multiple music audios to be stored in the library; each music audio to be stored has a name and includes multiple music clips; pre-process each music clip to obtain the CQT features of each music clip; input the CQT features of each music clip into a preset model to obtain the second feature vector of each music clip; obtain the identifier of each second feature vector through a vector retrieval tool; establish a correspondence between the name of the music audio to be stored to which each music clip belongs, the second feature vector of each music clip and the identifier of each second feature vector to obtain a music library.

[0156] An embodiment of the present invention further provides an electronic device, including a processor 120 and a memory 130. The memory 130 stores a computer program. When the processor executes the computer program, the music similarity detection method disclosed in the above embodiment is implemented.

[0157] The embodiment of the present invention further provides a storage medium on which a computer program is stored. When the computer program is executed by the processor 120, the music similarity detection method disclosed in the embodiment of the present invention is implemented.

[0158] In the several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architectures, functions, and operations of the devices, methods, and computer program products according to multiple embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or part of the code, which contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions marked in the boxes can also occur in an order different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, as well as the combination of boxes in the block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified functions or actions, or can be implemented using a combination of dedicated hardware and computer instructions.

[0159] In addition, the functional modules in the various embodiments of the present invention may be integrated together to form an independent part, or each module may exist independently, or two or more modules may be integrated to form an independent part.

[0160] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0161] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.

Claims

1. A music similarity detection method, characterized in that: The method comprises: Preprocessing the music clip to be detected to obtain the CQT features of the music clip to be detected; Inputting the CQT features of the music clip to be detected into a convolutional network of a preset model and performing convolution processing to obtain multiple feature maps; Using the feature regularization layer of the preset model, each of the feature maps is converted into a one-dimensional tensor and concatenated to obtain a concatenated feature map matrix; Transposing the concatenated feature map matrix to obtain a transposed matrix; Multiplying the concatenated feature map matrix by the transposed matrix to obtain a regularized feature map matrix; Selecting the largest element from each row of the regularized feature map matrix to obtain the regularized vector; Performing full connection processing on the regularized vector using the fully connected layer of the preset model to obtain a first eigenvector; A similarity detection is performed on the music segment to be detected based on the first feature vector and a pre-established music library.

2. The method according to claim 1, characterized in that The music library includes second feature vectors of a plurality of audio clips; The step of performing similarity detection on the music clip to be detected based on the first feature vector and a pre-established music library includes: Calculating the cosine similarity between the first eigenvector and each of the second eigenvectors; If the cosine similarity is greater than a preset threshold, it is determined that the music clip to be detected has an infringement risk; If all cosine similarities are less than the preset threshold, it is determined that the music clip to be detected does not have an infringement risk.

3. The method according to claim 1, characterized in that Before the step of preprocessing the music clip to be detected to obtain the CQT features of the music clip to be detected, the method further includes: Obtaining work content based on user interaction operations on the content upload interface; The music segment to be detected is extracted from the content of the work.

4. The method according to claim 3, characterized in that The music library includes names of a plurality of music clips, and the method further includes: If the music clip to be detected has an infringement risk after similarity detection, obtaining the name of the target music clip that has an infringement relationship with the music clip to be detected; The name of the target music clip and prompt information are displayed on the review result interface, and the prompt information is used to remind the user that there is a risk of infringement of the content of the work.

5. The method according to claim 1, wherein The preset model is obtained in the following manner: Acquire multiple music audio samples, and divide each of the music audio samples into multiple music segment samples in chronological order to obtain each of the music segment samples; Wherein, any two adjacent music clip samples in the music audio samples have the same audio frame; Preprocessing each of the music clip samples to obtain a CQT feature of each of the music clip samples; Performing dimensionality reduction and cluster analysis on the CQT features of all music clip samples belonging to the same music audio sample to obtain a label for each of the music clip samples; According to the preset loss function and the label of each music clip sample, the basic model is trained to obtain the preset model.

6. The method according to claim 1, characterized in that The music library is obtained in the following manner: Acquire multiple music audios to be stored in the library; each of the music audios to be stored in the library has a name and includes multiple music clips; Preprocessing each of the music clips to obtain a CQT feature of each of the music clips; Inputting the CQT feature of each of the music clips into the preset model to obtain a second feature vector of each of the music clips; Obtaining an identifier of each second feature vector using a vector search tool; A correspondence is established between the name of the music audio to be stored to which each music clip belongs, the second feature vector of each music clip, and the identifier of each second feature vector to obtain the music library.

7. A music similarity detection device, characterized in that: The device comprises: A preprocessing module, configured to preprocess the music clip to be detected to obtain the CQT features of the music clip to be detected; A processing module is configured to input the CQT features of the music clip to be detected into a convolutional network of a preset model and perform convolution processing to obtain a plurality of feature maps; use the feature regularization layer of the preset model to convert each of the feature maps into a one-dimensional tensor and perform splicing to obtain a spliced ​​feature map matrix; transpose the spliced ​​feature map matrix to obtain a transposed matrix; multiply the spliced ​​feature map matrix by the transposed matrix to obtain a regularized feature map matrix; select the largest element from each row of the regularized feature map matrix to obtain the regularized vector; use the fully connected layer of the preset model to perform full connection processing on the regularized vector to obtain a first feature vector; The detection module is used to perform similarity detection on the music segment to be detected based on the first feature vector and a pre-established music library.

8. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores a computer program, and when the processor executes the computer program, the method according to any one of claims 1 to 6 is implemented.

9. A storage medium, characterized in that: The storage medium stores a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 6.

Images