Target music retrieval method based on big data

By extracting and matching the melody and audio features of humming songs, a similarity matrix is ​​constructed, which solves the impact of humming errors and music library size on the search results, and improves the accuracy and distinction of music search.

CN118939829AInactive Publication Date: 2024-11-12LAIWU VOCATIONAL & TECHNICAL COLLEGE
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411141412.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-20
Publication Date
2024-11-12
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

In the prior art, humming search methods are susceptible to user humming errors and music library size, resulting in insufficient accuracy and distinction of search results.

Method used

The target music retrieval method based on big data is adopted to obtain the audio of humming songs, extract melody features and audio features, build a melody similarity and audio similarity matrix, sort and match, and reduce the impact of the music library size on the search results.

Benefits of technology

It improves the accuracy and distinction of music search, reduces the impact of humming errors and music library size on the search results, and achieves more accurate music matching.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118939829A_ABST
    Figure CN118939829A_ABST
Patent Text Reader

Abstract

The invention discloses a target music retrieval method based on big data, and belongs to the technical field of music retrieval, and the method specifically comprises the following steps: obtaining a humming song audio input by a target object, extracting melody features of a humming song fragment, comparing the melody features with melody features of songs in a song library to obtain melody similarity, and retrieving the melody similarity according to the melody similarity. Screening candidate songs from the song library according to the melody similarity X to obtain a music candidate set; analyzing each frame of audio data by adopting a Mel-frequency cepstrum transform technology, extracting audio features, constructing three-dimensional audio vectors according to the audio features, sequentially constructing an audio data similarity matrix of the humming song and all audios in the music candidate set according to the distance between the audio vectors, performing similarity calculation on the candidate songs, and obtaining an audio data similarity matrix of the humming song and all the audios in the music candidate set; obtaining audio similarity; sorting the music candidate set according to the melody similarity and the audio similarity to obtain a retrieval result, and improving the accuracy of target music retrieval.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of music retrieval, and in particular to a target music retrieval method based on big data. Background Art

[0002] With the advent of the information age, we rely on the Internet to obtain the information we need, but the information on it is growing explosively. How to effectively filter out useful information about the required music from massive music information has become a key technical issue.

[0003] In the prior art, humming retrieval is a content-based audio retrieval method, which performs retrieval by humming music clips by users. It is an extension of traditional text retrieval and brings great convenience to users. However, users will inevitably make various errors in the humming process, such as extra notes, missing notes, inaccurate pitch and length, out of tune, humming too fast or too slow, etc., that is, humming errors affect the accuracy of retrieval results. At the same time, as the size of the music library increases, the distinguishability of melody features between songs weakens, resulting in the accuracy of actual music retrieval results being easily affected by the size of the music library. Summary of the invention

[0004] The purpose of the present invention is to provide a target music retrieval method based on big data to solve the following technical problems: However, users will inevitably make various errors in the humming process, such as extra notes, missing notes, inaccurate pitch and length, out of tune, humming too fast or too slow, etc., that is, humming errors affect the accuracy of the retrieval results. At the same time, as the size of the music library expands, the distinguishability of the melody features between songs weakens, resulting in the accuracy of the actual music retrieval results being easily affected by the size of the music library.

[0005] The purpose of the present invention can be achieved through the following technical solutions: A target music retrieval method based on big data comprises the following steps: S1, for obtaining the humming song audio input by the target object, extracting the melody features of the humming song segment, comparing the melody features with the melody features of songs in a song library to obtain melody similarity, and screening candidate songs from the song library according to the melody similarity X to obtain a music candidate set; S2, the humming song audio is divided into frames according to a preset frame length, and a three-dimensional audio coordinate system is constructed with the frequency sequence variance at the maximum peak of the frame spectrum of the humming song audio as the horizontal axis, the logarithm of the sequence variance of the power sum of each frame as the vertical axis, and the average value of the power sum as the vertical axis to obtain a three-dimensional audio vector of the humming song, and an audio data similarity matrix S of the humming song and all audio in the music candidate set is constructed in turn according to the distance between the audio vectors, and the audio similarity X' is calculated; S3, sorting the music candidate set according to the melody similarity and the audio similarity, and outputting the target object according to the sorting result.

[0006] As a further solution of the present invention: in S1, the process of extracting the melody feature is as follows: The humming song audio is denoised, the fundamental frequency sequence of the humming song after denoising is extracted, and the energy threshold method is used to perform note cutting to obtain the audio sequence of the humming song.

[0007] As a further solution of the present invention: In S1, the specific process of comparison is: S11, calculating the similarity between the note sequence of the humming song segment and each sub-note sequence in the note sequence of the song to be compared in the song library and selecting the one with the largest similarity as the similarity X of the song to be compared, to obtain a first melody similarity; S12, calculating the confidence C of the candidate song according to the first melody similarity, and if C is greater than or equal to a preset threshold, outputting the candidate song result; if C is less than the preset threshold, using a linear scaling algorithm to correct the humming song segment, calculating the similarity between the note sequence of the corrected song audio and each sub-note sequence in the note sequence of the song to be compared in the song library, and selecting the one with the largest similarity as the similarity X of the song to be compared, to obtain a second melody similarity; S13, calculating the confidence C of the candidate song according to the second melody similarity, if the confidence C is greater than or equal to a preset threshold, outputting the candidate song result; if the confidence C is less than the preset threshold, repeating S12.

[0008] 4. According to the target music retrieval method based on big data as claimed in claim 3, it is characterized in that the specific calculation process of the confidence C is:

[0009] Among them, X1 is the similarity of the candidate result ranked first, N is the preset candidate song threshold, and Xi is the similarity of the candidate result ranked i.

[0010] As a further solution of the present invention: In S1, the specific process of selecting candidate songs is as follows: Select the top N songs to be compared that are ranked in similarity from the song library, select the sub-note sequence in the note sequence of any song to be compared that is most similar to the note sequence of the humming song clip, and determine the start and end time points of the sub-note sequence, perform audio extraction on the songs to be matched based on the start and end time points of the sub-note sequence, and obtain a music candidate set.

[0011] As a further solution of the present invention: In S2, the specific process of extracting audio features is: The Mel frequency cepstrum transform technique is used to generate the spectrum corresponding to each audio frame. The frequency corresponding to the maximum peak in each spectrum is selected to obtain the frequency sequence p1, p2, p3, ..., pn. The variance of the frequency sequence is calculated using the variance formula. All amplitude values ​​of the spectrum corresponding to any audio frame are selected, and the power sum is calculated using the formula power = (amplitude value) 2 , calculate the power sum corresponding to any audio frame, obtain the power sum sequence w1,w2,w3,...,wn, calculate the variance logarithm and power sum mean of the power sum sequence.

[0012] As a further solution of the present invention: in S2, the specific calculation process of the audio similarity is: The number of three-dimensional audio vectors of the humming song audio is obtained and marked as n, and a similarity matrix between the humming song audio and any audio in the music candidate set is constructed according to the data corresponding to the n points. The similarity matrix is: ; in is the difference distance value between the humming song audio B and any audio in the music candidate set; the matrix shortest cumulative path distance D between any audio in each music candidate set and the humming song audio B is calculated in turn dtw (B, Am), a similarity function is constructed based on the distance matrix using the dynamic time warping method. The similarity function calculation formula is:

[0013] X' is the audio similarity, and the curve comparison similarity values ​​X1, X2, ..., Xm corresponding to all sub-curves are calculated in sequence.

[0014] As a further solution of the present invention: in S3, the specific process of sorting is: According to the calculation formula W=b*X+d*X', where b and d are preset coefficients, the recommendation score of any candidate song is calculated, the candidate songs are sorted from large to small according to the recommendation score, and songs are recommended to the target object according to the sorting result to obtain the music retrieval result.

[0015] Beneficial effects of the present invention: The present invention extracts melody features from the acquired humming songs, performs preliminary search on the music song library according to the extracted melody features, and obtains a music candidate set. Since the accuracy of the retrieval result of the retrieval method based on song melody feature matching is easily affected by the scale of the music library, resulting in inaccurate retrieval results, the present invention identifies the subtle differences between melodious audios in different dimensions through audio similarity, enhances the distinction between different audios to accurately identify and match similar music clips, thereby obtaining the overall similarity between the humming songs and the music candidate set, sorts the music candidate set according to the melody similarity and the overall similarity, obtains the retrieval result and outputs the target object, reduces the influence of the scale of the music library on the target music retrieval result, and improves the accuracy of the target music retrieval. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The present invention will be further described below in conjunction with the accompanying drawings.

[0017] Figure 1 It is a flow chart of a target music retrieval method based on big data of the present invention. DETAILED DESCRIPTION

[0018] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0019] See also Figure 1 As shown, the present invention is a target music retrieval method based on big data, comprising the following steps: S1, for obtaining the humming song audio input by the target object, extracting the melody features of the humming song segment, comparing the melody features with the melody features of songs in a song library to obtain melody similarity, and screening candidate songs from the song library according to the melody similarity X to obtain a music candidate set; S2, the humming song audio is divided into frames according to a preset frame length, and a three-dimensional audio coordinate system is constructed with the frequency sequence variance at the maximum peak of the frame spectrum of the humming song audio as the horizontal axis, the logarithm of the sequence variance of the power sum of each frame as the vertical axis, and the average value of the power sum as the vertical axis to obtain a three-dimensional audio vector of the humming song, and an audio data similarity matrix S of the humming song and all audio in the music candidate set is constructed in turn according to the distance between the audio vectors, and the audio similarity X' is calculated; S3, sorting the music candidate set according to the melody similarity and the audio similarity, obtaining the retrieval result and outputting the target object.

[0020] The present invention extracts melody features from the acquired humming songs, performs preliminary search on the music song library according to the extracted melody features, and obtains a music candidate set. Since the accuracy of the retrieval result of the retrieval method based on song melody feature matching is easily affected by the scale of the music library, resulting in inaccurate retrieval results, the present invention identifies the subtle differences between melodious audios in different dimensions through audio similarity, enhances the distinction between different audios to accurately identify and match similar music clips, thereby obtaining the overall similarity between the humming songs and the music candidate set, sorts the music candidate set according to the melody similarity and the overall similarity, obtains the retrieval result and outputs the target object, reduces the influence of the scale of the music library on the target music retrieval result, and improves the accuracy of the target music retrieval.

[0021] In a preferred implementation of the present invention, in S1, the process of extracting melody features is as follows: The humming song audio is denoised, the fundamental frequency sequence of the humming song after denoising is extracted, and the energy threshold method is used to perform note cutting to obtain the audio sequence of the humming song.

[0022] In another preferred implementation of the present invention, in S1, the specific process of comparison is: S11, calculating the similarity between the note sequence of the humming song segment and each sub-note sequence in the note sequence of the song to be compared in the song library and selecting the one with the largest similarity as the similarity X of the song to be compared, to obtain a first melody similarity; S12, calculating the confidence C of the candidate song according to the first melody similarity, and if C is greater than or equal to a preset threshold, outputting the candidate song result; if C is less than the preset threshold, using a linear scaling algorithm to correct the humming song segment, calculating the similarity between the note sequence of the corrected song audio and each sub-note sequence in the note sequence of the song to be compared in the song library, and selecting the one with the largest similarity as the similarity X of the song to be compared, to obtain a second melody similarity; S13, calculating the confidence C of the candidate song according to the second melody similarity, if the confidence C is greater than or equal to a preset threshold, outputting the candidate song result; if the confidence C is less than the preset threshold, repeating S12.

[0023] It can be understood that the melody features of the song clip hummed by the user are matched with the melody features of the songs in the song library, and some songs with high similarity are selected from the song library as candidate songs based on the matching results, for example, a set number of songs are selected as candidate songs in descending order of melody similarity. The DTW algorithm can be used to calculate the similarity between the melody features of the user's humming song fragment and the melody features of the songs in the song library. However, since the humming rate of most users is close to the normal rate of the original song, it is not ruled out that the humming speed of some users is faster or slower than the normal rate of the original song. Therefore, the confidence C of the candidate song is calculated according to the first melody similarity. If the confidence C is less than the preset threshold, it means that the current candidate song does not fully meet the similarity. Since the user will inevitably make various errors in the humming process, such as humming too fast or too slow, etc., resulting in the current candidate song not fully meeting the similarity, the humming song fragment is linearly scaled, and the duration of the humming song is dynamically adjusted by the preset scaling factor γ while keeping the frequency of the audio unchanged. The similarity is calculated according to the scaled humming song until the confidence C is greater than or equal to the preset threshold, and the candidate song result is output, thereby reducing the wrong matching caused by the difference in humming rate and achieving accurate matching of the target music retrieval.

[0024] In another preferred implementation of the present invention, the specific calculation process of the confidence C is:

[0025] Among them, X1 is the similarity of the candidate result ranked first, N is the preset candidate song threshold, and Xi is the similarity of the candidate result ranked i.

[0026] In another preferred implementation of the present invention, in S1, the specific process of selecting candidate songs is as follows: Select the top N songs to be compared that are ranked in similarity from the song library, select the sub-note sequence in the note sequence of any song to be compared that is most similar to the note sequence of the humming song clip, and determine the start and end time points of the sub-note sequence, perform audio extraction on the songs to be matched based on the start and end time points of the sub-note sequence, and obtain a music candidate set.

[0027] In another preferred implementation of the present invention, in S2, the specific process of extracting audio features is: The Mel frequency cepstrum transform technique is used to generate the spectrum corresponding to each audio frame. The frequency corresponding to the maximum peak in each spectrum is selected to obtain the frequency sequence p1, p2, p3, ..., pn. The variance of the frequency sequence is calculated using the variance formula. All amplitude values ​​of the spectrum corresponding to any audio frame are selected, and the power sum is calculated using the formula power = (amplitude value) 2, calculate the power sum corresponding to any audio frame, obtain the power sum sequence w1,w2,w3,...,wn, calculate the variance logarithm and power sum mean of the power sum sequence.

[0028] It can be understood that database music matching through the above-mentioned melody features avoids the influence of inaccurate retrieval results caused by inaccurate target object tone length, out of tune, too fast or too slow humming rate, etc., but when the database scale is large, the distinguishability of melody features between songs is weakened, resulting in the accuracy of actual music retrieval results being easily affected by the scale of the music library, resulting in inaccurate retrieval results. Therefore, a spectrogram is generated through Mel-frequency cepstrum transform technology, and frequency sequences and power sum sequences are extracted from the spectrogram. The energy characteristics of the audio can characteristically show the volume and intensity of the song, and the frequency characteristics of the audio can show the pitch of the song. The spectral characteristics of the song are obtained according to the logarithm of the variance of the power sum sequence and the mean of the power sum, and three-dimensional coordinates are constructed. A similarity matrix is ​​constructed based on the three-dimensional coordinates, which can identify subtle differences between melodious audios in different dimensions, enhance the distinction between different audios, and help to more accurately identify and match similar music clips, thereby obtaining the overall similarity between the hummed song and the music candidate set, thereby reducing the influence of the target music retrieval result on the scale of the music library and improving the accuracy of the target music retrieval.

[0029] In another preferred implementation of the present invention, in S2, the specific calculation process of the audio similarity is: The number of three-dimensional audio vectors of the humming song audio is obtained and marked as n, and a similarity matrix between the humming song audio and any audio in the music candidate set is constructed according to the data corresponding to the n points. The similarity matrix is: ; in is the difference distance value between the humming song audio B and any audio in the music candidate set; the matrix shortest cumulative path distance D between any audio in each music candidate set and the humming song audio B is calculated in turn dtw (B, Am), a similarity function is constructed based on the distance matrix using the dynamic time warping method. The similarity function calculation formula is:

[0030] X' is the audio similarity, and the curve comparison similarity values ​​X1, X2, ..., Xm corresponding to all sub-curves are calculated in sequence.

[0031] In another preferred implementation of the present invention, in S3, the specific process of sorting is: According to the calculation formula W=b*X+d*X', where b and d are preset coefficients, the recommendation score of any candidate song is calculated, the candidate songs are sorted from large to small according to the recommendation score, and songs are recommended to the target object according to the sorting result to obtain the music retrieval result.

[0032] It is understandable that as the size of the music library expands, the distinguishability of the melody features between songs weakens. The accuracy of the retrieval results of the existing retrieval method based on song melody feature matching is easily affected by the size of the music library, resulting in inaccurate retrieval results. Therefore, audio similarity is used to identify the subtle differences between melodious audios in different dimensions, which enhances the distinction between different audios and more accurately identifies and matches similar music clips, thereby obtaining the overall similarity between the hummed song and the music candidate set, thereby reducing the impact of the size of the music library on the target music retrieval results and improving the accuracy of the target music retrieval.

[0033] The above is a detailed description of an embodiment of the present invention, but the content is only a preferred embodiment of the present invention and cannot be considered to limit the scope of implementation of the present invention. All equivalent changes and improvements made within the scope of the present invention should still fall within the scope of the patent coverage of the present invention.

Claims

1. A target music retrieval method based on big data, characterized in that: The following steps are involved: S1, for obtaining the humming song audio input by the target object, extracting the melody features of the humming song segment, comparing the melody features with the melody features of songs in a song library to obtain melody similarity, and screening candidate songs from the song library according to the melody similarity X to obtain a music candidate set; S2, the humming song audio is divided into frames according to a preset frame length, and a three-dimensional audio coordinate system is constructed with the frequency sequence variance at the maximum peak of the frame spectrum of the humming song audio as the horizontal axis, the logarithm of the sequence variance of the power sum of each frame as the vertical axis, and the average value of the power sum as the vertical axis to obtain a three-dimensional audio vector of the humming song, and an audio data similarity matrix S of the humming song and all audio in the music candidate set is constructed in turn according to the distance between the audio vectors, and the audio similarity X' is calculated; S3, sorting the music candidate set according to the melody similarity and the audio similarity, obtaining the retrieval result and outputting the target object.

2. A target music retrieval method based on big data according to claim 1, characterized in that: In S1, the process of extracting melody features is as follows: The humming song audio is denoised, the fundamental frequency sequence of the humming song after denoising is extracted, and the energy threshold method is used to perform note cutting to obtain the audio sequence of the humming song.

3. A target music retrieval method based on big data according to claim 1, characterized in that: In S1, the specific process of comparison is: S11, calculating the similarity between the note sequence of the humming song segment and each sub-note sequence in the note sequence of the song to be compared in the song library and selecting the one with the largest similarity as the similarity X of the song to be compared, to obtain a first melody similarity; S12, calculating the confidence C of the candidate song according to the first melody similarity, and if C is greater than or equal to a preset threshold, outputting the candidate song result; If C is less than a preset threshold, a linear scaling algorithm is used to correct the humming song segment, and the similarity between the note sequence of the corrected song audio and each sub-note sequence in the note sequence of the song to be compared in the song library is calculated, and the one with the largest similarity is selected as the similarity X of the song to be compared, to obtain the second melody similarity; S13, calculating the confidence C of the candidate song according to the second melody similarity, and if the confidence C is greater than or equal to a preset threshold, outputting the candidate song result; If the confidence C is less than the preset threshold, S12 is repeated.

4. A target music retrieval method based on big data according to claim 3, characterized in that: The specific calculation process of confidence C is: Among them, X1 is the similarity of the candidate result ranked first, N is the preset threshold of the number of candidate songs, and Xi is the similarity of the candidate result ranked i.

5. The target music retrieval method based on big data according to claim 1, characterized in that: In S1, the specific process of selecting songs is as follows: Select the top N songs to be compared that are ranked in similarity from the song library, select the sub-note sequence in the note sequence of any song to be compared that is most similar to the note sequence of the humming song clip, and determine the start and end time points of the sub-note sequence, perform audio extraction on the songs to be matched based on the start and end time points of the sub-note sequence, and obtain a music candidate set.

6. A target music retrieval method based on big data according to claim 1, characterized in that: In S2, the specific process of extracting audio features is as follows: The Mel frequency cepstrum transform technique is used to generate the spectrum corresponding to each audio frame. The frequency corresponding to the maximum peak in each spectrum is selected to obtain the frequency sequence p1, p2, p3, ..., pn. The variance of the frequency sequence is calculated using the variance formula. All amplitude values ​​of the spectrum corresponding to any audio frame are selected, and the power sum is calculated using the formula power = (amplitude value) 2 , calculate the power sum corresponding to any audio frame, obtain the power sum sequence w1,w2,w3,...,wn, calculate the variance logarithm and power sum mean of the power sum sequence.

7. A target music retrieval method based on big data according to claim 1, characterized in that: In S2, the specific calculation process of the audio similarity is: The number of three-dimensional audio vectors of the humming song audio is obtained and marked as n, and a similarity matrix between the humming song audio and any audio in the music candidate set is constructed according to the data corresponding to the n points. The similarity matrix is: ; in is the difference distance value between the humming song audio B and any audio in the music candidate set; the matrix shortest cumulative path distance D between any audio in each music candidate set and the humming song audio B is calculated in turn dtw (B, Am), a similarity function is constructed based on the distance matrix using the dynamic time warping method. The similarity function calculation formula is: ; X' is the audio similarity, and the curve comparison similarity values ​​X1, X2, ..., Xm corresponding to all sub-curves are calculated in sequence.

8. The target music retrieval method based on big data according to claim 1, characterized in that: In S3, the specific process of sorting is as follows: According to the calculation formula W=b*X+d*X', where b and d are preset coefficients, the recommendation score of any candidate song is calculated, the candidate songs are sorted from large to small according to the recommendation score, and songs are recommended to the target object according to the sorting result to obtain the music retrieval result.