Music structure analysis method, electronic device and storage medium

Abstract

The present application discloses a music structure analysis method, which obtains lyrics text of target music and calculates text line similarity of the lyrics text; wherein the text line similarity includes the similarity between any two lyrics text lines in the lyrics text; generates a self-similarity matrix of the lyrics text according to the text line similarity, and uses the self-similarity matrix to query repeated lyrics segments in the lyrics text; wherein each repeated lyrics segment includes at least one lyrics text line; and divides the music structure of the target music according to the position of the repeated lyrics segment. The present application can improve the accuracy of music structure analysis. The present application also discloses an electronic device and a storage medium, which have the above beneficial effects.

Description

Technical Field

[0001] The present application relates to the technical field of audio processing, and in particular to a music structure analysis method, an electronic device and a storage medium. Background Art

[0002] In order to quickly find a song or try to listen to a song, users usually want to listen to only a certain part of the song (such as the chorus, the main song or the prelude, etc.). At this time, it is necessary to analyze the music structure of the song so as to quickly find the corresponding segment.

[0003] In the related art, music structure analysis is mainly performed by extracting audio features from audio files. However, since audios with the same music structure in audio files may be quite different, the accuracy of the above-mentioned scheme for structural analysis based on audio features is low.

[0004] Therefore, how to improve the accuracy of music structure analysis is a technical problem that technical personnel in this field currently need to solve. Summary of the invention

[0005] The purpose of this application is to provide a music structure analysis method, an electronic device and a storage medium, which can improve the accuracy of music structure analysis.

[0006] In order to solve the above technical problems, the present application provides a music structure analysis method, which includes:

[0007] Obtaining lyrics text of target music, and calculating text line similarity of the lyrics text; wherein the text line similarity includes the similarity between any two lyrics text lines in the lyrics text;

[0008] Generate a self-similarity matrix of the lyrics text according to the text line similarity, and use the self-similarity matrix to query repeated lyrics segments in the lyrics text; wherein each of the repeated lyrics segments includes at least one of the lyrics text lines;

[0009] The music structure of the target music is divided according to the positions of the repeated lyrics segments.

[0010] Optionally, calculating the text line similarity of the lyrics text includes:

[0011] Calculate the text line semantic similarity and / or text line structural similarity of the lyrics text.

[0012] Optionally, generating a self-similarity matrix of the lyrics text according to the text line similarity includes:

[0013] If the text line similarity includes the text line semantic similarity and the text line structural similarity, a weighted calculation is performed on the text line semantic similarity and the text line structural similarity to generate a self-similarity matrix of the lyrics text.

[0014] Optionally, calculating the text line structure similarity of the lyrics text includes:

[0015] Extracting the structural sequence of the lyrics text, and calculating the text line structural similarity of the lyrics text according to the structural sequence;

[0016] The structure sequence includes any one or a combination of any several of a sentence word number structure sequence, a pinyin structure sequence and a part-of-speech structure sequence.

[0017] Optionally, after dividing the music structure of the target music according to the positions of the repeated lyrics segments, the method further includes:

[0018] Optionally, after dividing the music structure of the target music according to the positions of the repeated lyrics segments, the method further includes:

[0019] Dividing the target music into multiple music clips according to the music structure, and adding corresponding music structure tags to all the music clips;

[0020] If a music singing instruction is received, the music structure tag to be sung is determined according to the music singing instruction, and a picture to be shared is generated; wherein the picture to be shared includes a music playback link corresponding to the music structure tag to be sung;

[0021] The picture to be shared is sent to a target device so that the target device plays the music clip corresponding to the music structure tag to be sung through the music play link in the picture to be shared and records the user's singing voice.

[0022] Optionally, generating the picture to be shared includes:

[0023] Determine the music structure tag of the sung music according to the music singing instruction, and generate lyrics pictures corresponding to the music structure tag of the sung music and the music structure tag to be sung;

[0024] Add the QR code of the music play link to the lyrics picture to obtain the picture to be shared.

[0025] Optionally, using the self-similarity matrix to query repeated lyrics segments in the lyrics text includes:

[0026] Marking elements in the self-similarity matrix that are greater than a similarity cutoff value as similar elements;

[0027] Marking elements in the self-similarity matrix that are less than or equal to the similarity cutoff value as non-similar elements;

[0028] The lyrics division boundary is determined according to the distribution of the similar elements and the non-similar elements in the self-similarity matrix, and the repeated lyrics segments in the lyrics text are determined using the lyrics division boundary.

[0029] Optionally, determining the lyrics division boundary according to the distribution of the similar elements and the non-similar elements in the self-similarity matrix includes:

[0030] If m(i, j) is a similar element and m(i+1, j+1) is a non-similar element, then the j-th row of lyrics is marked as the end of the paragraph; wherein m is an element in the self-similar matrix, i is the number of rows of element m in the self-similar matrix, and j is the number of columns of element m in the self-similar matrix;

[0031] And / or, if m(i, j) is a non-similar element and m(i+1, j+1) is a similar element, then mark the jth line of lyrics as the end-of-paragraph lyrics;

[0032] And / or, if m(i, j) and m(i+1, j+1) are both similar elements, then the lyrics in the i-th row and the i+1-th row are marked as repeated lyrics;

[0033] And / or, if m(i, i+1) is a similar element, the lyrics in the i-th line and the i+1-th line are marked as the same paragraph of lyrics;

[0034] and / or, if m(i, 1) is a similar element, then the i-1th line of lyrics is marked as the end-of-paragraph lyrics;

[0035] And / or, if m(i, n) is a similar element, then the i-th line of lyrics is marked as the end-of-paragraph lyrics; wherein n is the total number of lines of lyrics in the lyrics text;

[0036] All the lyrics text lines are clustered according to the distribution of target type lyrics to obtain the lyrics division boundary; wherein the target type lyrics include any one or a combination of any one of the paragraph ending lyrics, repeated lyrics and the same paragraph lyrics.

[0037] The present application also provides a storage medium on which a computer program is stored, and when the computer program is executed, the steps of the above-mentioned music structure analysis method are implemented.

[0038] The present application also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above-mentioned music structure analysis method when calling the computer program in the memory.

[0039] The present application provides a music structure analysis method, comprising: obtaining lyrics text of target music, and calculating text line similarity of the lyrics text; wherein the text line similarity includes the similarity between any two lyrics text lines in the lyrics text; generating a self-similarity matrix of the lyrics text according to the text line similarity, and using the self-similarity matrix to query repeated lyrics segments in the lyrics text; wherein each of the repeated lyrics segments includes at least one lyrics text line; and dividing the music structure of the target music according to the positions of the repeated lyrics segments.

[0040] The present application obtains the lyrics text of the target music and calculates the similarity between any two lines of lyrics text in the lyrics text, and then generates a self-similarity matrix of the lyrics text according to the text line similarity. Since the lyrics text has the same structure as the target music, and there are repeated lyrics fragments in the lyrics text, the music structure of the target music can be divided based on the position of the repeated lyrics fragments in the lyrics text. The above process divides the music structure based on the lyrics of the target music, the repeatability of the lyrics is relatively consistent, and the division process does not depend on the audio features in the audio file, which can improve the accuracy of the music structure analysis. The present application also provides an electronic device and a storage medium, which have the above-mentioned beneficial effects and are not repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0042] Figure 1 An architecture diagram of a music structure analysis system provided in an embodiment of the present application;

[0043] Figure 2 A flowchart of a music structure analysis method provided in an embodiment of the present application;

[0044] Figure 3 A flowchart of a music playing method provided in an embodiment of the present application;

[0045] Figure 4 A flowchart of a method for determining repeated lyrics segments provided in an embodiment of the present application;

[0046] Figure 5 A schematic diagram of the comparison relationship between a lyric text and a self-similarity matrix provided in an embodiment of the present application;

[0047] Figure 6 A flowchart of an audio recording solution provided in an embodiment of the present application;

[0048] Figure 7 A structural diagram of an electronic device provided in an embodiment of the present application;

[0049] Figure 8 A picture to be shared provided in an embodiment of the present application. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0051] Music has a complex structural hierarchy. For example, a single note has different timbre, pitch, duration and pronunciation. The next level has the theme, segment and chapter of the music, and there are more hierarchical structures. Generally, this field classifies audio content into meaningful structural classifications. For example, pop music is composed of intro, verse, prechorus (the connecting paragraph between the verse and the chorus), chorus, bridge (the transition section inserted between the repeated verse and the chorus, the style is often different from the original song), and outro. The main purpose of music structure analysis is to distinguish the corresponding parts of a song, and some parts may appear many times. For example, common pop songs are composed of intro, verse 1, chorus 1, verse 2, chorus 2, bridge and chorus 3. The verse or chorus is the most basic unit of the song. The verse conveys the background story of a song and is the backbone of the whole song. The chorus is often the most impressive segment for the audience and is often the climax of the song. It will be repeated many times relative to the verse to deepen the audience's memory.

[0052] In the related art, various audio features (such as speech feature parameters MFCC, etc.) are usually extracted from audio files, and then music structure analysis is performed based on the audio features. But in fact, most audio is a mixture of human voice and accompaniment. If the background accompaniment of the music structure of similar fragments is quite different, or the singing style of similar song fragments changes, the audio corresponding to the same music structure will be quite different, so it is impossible to effectively distinguish the structure, which easily causes errors in structural analysis. Since audio files are used for audio structure analysis, when performing structural division, the end point of each lyric cannot be found accurately, and the end point may be divided into the middle of the sentence, resulting in errors in structural analysis. In order to solve the technical problems existing in the above-mentioned music structure analysis, the present application provides a new music structure analysis method through the following embodiments, which can improve the accuracy of music structure analysis.

[0053] In order to facilitate understanding of the music structure analysis solution provided by this application, the system used is introduced below. Figure 1 , Figure 1 The architecture diagram of a music structure analysis system provided in an embodiment of the present application includes a client 101, a computing device 102, and a database 103. The client 101 can send a structure analysis request to the computing device, and the computing device 102 determines the target music to be analyzed by parsing the structure analysis request. The computing device 102 obtains the lyrics text corresponding to the target music from the database 103, and analyzes the music structure of the target music based on the lyrics text.

[0054] See below Figure 2 , Figure 2 A flowchart of a music structure analysis method provided in an embodiment of the present application.

[0055] Specific steps may include:

[0056] S201: Obtain lyrics text of target music, and calculate text line similarity of the lyrics text;

[0057] Before this step, there may be an operation of receiving a structural analysis request, so as to determine the target music that needs to be structurally analyzed according to the structural analysis request. The music structural analysis mentioned in this embodiment refers to the division of various types of segments in the music, for example, a piece of music can be divided into multiple parts such as prelude, verse, lead song, chorus, bridge, chorus and outro.

[0058] After determining the target music that needs to be structurally analyzed, this embodiment can obtain the lyrics text of the target music, which can be the text of the entire content sung by the singer during the playback of the target music. The lyrics text includes multiple lyrics text lines. This step can calculate the similarity between any two lyrics text lines, and then obtain the text line similarity. The above text line similarity includes the similarity between any two lyrics text lines in the lyrics text.

[0059] S202: generating a self-similarity matrix of the lyrics text according to the text line similarity, and using the self-similarity matrix to query repeated lyrics segments in the lyrics text;

[0060] Among them, based on the obtained text line similarity, this embodiment can generate a self-similarity matrix of the lyrics text according to the text line similarity. Each element in the self-similarity matrix represents the similarity between two lines of lyrics text, for example, the element m(i, j) represents the similarity between the i-th line of lyrics text and the j-th line of lyrics text.

[0061] It is understandable that the lyrics fragments corresponding to the same music structure in the lyrics text are repetitive, for example, the lyrics content and sentence structure of the chorus are basically the same, so the repeated lyrics fragments in the lyrics text can be determined by using the self-similarity matrix of the lyrics text, and each of the repeated lyrics fragments includes at least one of the lyrics text lines. Repeated lyrics fragments refer to two lyrics fragments with a repeated pattern, and the above-mentioned repeated pattern can meet the lyrics pattern of preset constraints, and the preset constraints include any one or a combination of any several of the same number of lines, the same semantics and the same structural sequence. There can be multiple groups of repeated lyrics fragments in the lyrics text, and the repeated lyrics fragments in the same group are similar, and the repeated lyrics fragments in different groups are not similar. Taking the music structure "prelude, verse 1, chorus 1, verse 2, chorus 2, chorus 3 and outro" as an example, verse 1 and verse 2 are repeated lyrics fragments in one group, and chorus 1, chorus 2 and chorus 3 are repeated lyrics fragments in another group.

[0062] S203: Divide the music structure of the target music according to the positions of the repeated lyrics segments.

[0063] After obtaining the repeated lyrics segment, the lyrics text can be divided into multiple lyrics paragraphs according to the position of the repeated lyrics segment, and then the target music can be divided into multiple music segments according to the music playing time corresponding to the lyrics segment, so as to obtain the music structure of the target music. As a feasible implementation method, this embodiment can use the first line and the last line of lyrics of the repeated lyrics segment as the lyrics division boundary, and then divide the lyrics text into multiple lyrics paragraphs according to the lyrics boundary.

[0064] Music structure analysis is an important field of music understanding research, and music structure analysis helps computer-aided music understanding and analysis. After obtaining the music structure of the target music in this embodiment, it can be combined with music theory rules to identify the form, style and type of music. Furthermore, after obtaining the music structure in this embodiment, the chorus of the target music can be used as a song summary, so that the user can quickly preview the chorus part of a song, or sing only the chorus part when singing.

[0065] Since the present embodiment analyzes the music structure based on the lyrics, the music structure division process can be accurate to each sentence, and the boundaries of the music structure will not be divided in the sentence. Moreover, for most songs, the repetitiveness of the lyrics is relatively consistent, and its consistency is better than the similarity of the extracted audio features. The scheme of the present embodiment is more conducive to accurate division. Furthermore, if a certain piece of music is structurally analyzed, the existing technology may not be able to perform structural division because the selected piece cannot find a similar piece. However, the scheme of the present embodiment can achieve effective structural division by performing similarity analysis on the lyrics text.

[0066] This embodiment obtains the lyrics text of the target music and calculates the similarity between any two lines of lyrics text in the lyrics text, and then generates a self-similarity matrix of the lyrics text according to the text line similarity. Since the lyrics text has the same structure as the target music, and there are repeated lyrics fragments in the lyrics text, the music structure of the target music can be divided based on the position of the repeated lyrics fragments in the lyrics text. The above process divides the music structure based on the lyrics of the target music, the repeatability of the lyrics is relatively consistent, and the division process does not depend on the audio features in the audio file, which can improve the accuracy of music structure analysis.

[0067] As for Figure 2 For further introduction of the corresponding embodiment, the above-mentioned text line similarity may include text line semantic similarity and / or text line structural similarity. Therefore, the self-similarity matrix can be generated using the text line semantic similarity of the lyrics text, the self-similarity matrix can be generated using the text line structural similarity of the lyrics text, and the self-similarity matrix can be generated using the text line semantic similarity of the lyrics text.

[0068] The above-mentioned text line semantic similarity refers to: the similarity between any two lines of lyrics text lines in the lyrics text in the semantic dimension. In this embodiment, the text line semantic similarity can be calculated through the open source tool gensim. The specific process may include: processing the lyrics text line through the jieba word segmenter to form a two-dimensional array; generating a dictionary corresponding to the two-dimensional array, and passing the two-dimensional array through the doc2bow sparse vector to form a corpus. Use the TF model algorithm to calculate the TF-IDF value of each word in the corpus. Obtain the feature number of the dictionary token2id; calculate the sparse matrix similarity and establish an index. Calculate the sparse vectors of the two lines of lyrics to be compared through doc2bow, and calculate the text line semantic similarity based on the sparse vectors of the two lines of lyrics to be compared. Of course, this embodiment can also use other known semantic similarity comparison methods to calculate the text line semantic similarity, which will not be repeated here. For example, the open source tool gensim can be used to calculate the text line semantic similarity, and its value range is 0 to 1. The larger the value, the more similar it is, and the smaller the value, the less similar it is.

[0069] The above-mentioned text line structural similarity refers to: the similarity between any two lines of lyrics text in the lyrics text in the structural dimension. The above-mentioned structural dimension may include any one or a combination of any several of the sentence word count structural dimension, the pinyin structure dimension and the part-of-speech structure dimension. The similarity of the above-mentioned sentence word count structural dimension refers to the similarity of the number of words in each sentence of the lyrics, the similarity of the above-mentioned pinyin structure dimension refers to the similarity of the initial consonants and / or finals in the lyrics, and the similarity of the above-mentioned part-of-speech structure dimension refers to the similarity of the subject-predicate-verb structure. Specifically, the present embodiment may calculate the text line structural similarity of the lyrics text in the following manner: extracting the structural sequence of the lyrics text, and calculating the text line structural similarity of the lyrics text according to the structural sequence; wherein the structural sequence includes any one or a combination of any several of the sentence word count structural sequence, the pinyin structure sequence and the part-of-speech structure sequence.

[0070] According to the above method for calculating text line similarity, it can be known that the present embodiment can determine text line similarity from one or more dimensions. If the above embodiment determines text line similarity from only one dimension, the text line similarity can be directly used to generate the self-similarity matrix of the lyrics text. If the above embodiment determines text line similarity from multiple dimensions, the text line similarity of each dimension can be weighted to generate the self-similarity matrix of the lyrics text.

[0071] For example, if the text line similarity includes the text line semantic similarity and the text line structural similarity, the text line semantic similarity and the text line structural similarity are weighted to generate the self-similarity matrix of the lyrics text. In the self-similarity matrix SSM, SSM(i, j) = w1*A(i, j) + w2*B(i, j), A(i, j) is the text line semantic similarity between the i-th line of lyrics and the j-th line of lyrics, B(i, j) is the text line structural similarity between the i-th line of lyrics and the j-th line of lyrics, w1 is the weight of the text line semantic similarity, and w2 is the weight of the text line structural similarity.

[0072] For example, if the text line similarity includes the text line semantic similarity and the text line structural similarity, and the text line structural similarity includes the sentence word number structure sequence similarity, the pinyin structure sequence similarity and the part-of-speech structure sequence similarity, different weights can be set for the sentence word number structure sequence similarity, the pinyin structure sequence similarity and the part-of-speech structure sequence similarity to generate a self-similarity matrix SSM, SSM(i, j) = w1*A(i, j)+w21*B1(i, j)+w22*B2 (i, j)+w23*B3(i, j), B1(i, j) is the sentence word structure sequence similarity between the i-th line of lyrics and the j-th line of lyrics, B2(i, j) is the pinyin structure sequence similarity between the i-th line of lyrics and the j-th line of lyrics, B3(i, j) is the part-of-speech structure sequence similarity between the i-th line of lyrics and the j-th line of lyrics, w21 is the weight of the sentence word structure sequence similarity, w22 is the weight of the pinyin structure sequence similarity, and w23 is the weight of the part-of-speech structure sequence similarity.

[0073] See also Figure 3 , Figure 3 A flowchart of a music playing method provided in an embodiment of the present application. Figure 2 The further description after determining the music structure of the target music in the corresponding embodiment can be compared with Figure 2 The corresponding embodiments are combined to obtain a further implementation method. This embodiment may include the following steps:

[0074] S301: Divide the target music into multiple music clips according to the music structure, and add corresponding music structure tags to all the music clips;

[0075] After obtaining the music structure of the target music, the start and end time of each part can be determined, and then the target music can be divided into multiple music segments according to the start and end time.

[0076] This embodiment can compare the arrangement of each music clip in the target music with the preset template, and determine the music structure label of each music clip according to the template with the highest similarity. For example, there is a first template "verse 1, chorus 1, verse 2, chorus 2 and chorus 3", and a second template "verse 1, chorus 1, verse 2, chorus 2, chorus 3 and chorus 4"; if the target music includes 5 music clips, and the first and third clips are a group of repeated music clips, and the second, fourth and fifth clips are another group of repeated music clips, then it is determined that the target music meets the arrangement of the music structure label in the first template.

[0077] S302: If a music singing instruction is received, determining a music structure tag to be sung according to the music singing instruction, and generating a picture to be shared;

[0078] Among them, after adding corresponding music structure tags to all music clips, music clips corresponding to any music structure can be played according to the music structure tags. During the song singing and recording process, the user can first select a music clip corresponding to a certain music structure tag to play and record the singing content. After the user finishes singing, he can send a song singing invitation to other users by issuing a music singing instruction. The music singing instruction may include the music structure tags corresponding to the music clips that the user has sung, and the music structure tags of the music clips to be sung that need to be sung by other users. This embodiment can generate a picture to be shared according to the music structure tags to be sung, and the above-mentioned picture to be shared includes the music playback link corresponding to the music structure tags to be sung.

[0079] Specifically, this embodiment can generate the picture to be shared in the following manner: determine the music structure tag of the sung music according to the music connection instruction, and generate lyrics pictures corresponding to the music structure tag of the sung music and the music structure tag to be sung; add the QR code of the music playback link to the lyrics picture to obtain the picture to be shared. Figure 8 , Figure 8 A picture to be shared provided in an embodiment of the present application.

[0080] S303: Send the picture to be shared to the target device, so that the target device plays the music clip corresponding to the music structure tag to be sung through the music play link in the picture to be shared and records the user's singing voice.

[0081] The above embodiment discloses a music playing method based on music structure, which can perform structured analysis on song files by using music structure, so as to locate the chorus, main song and other segments of the song. For example, in a karaoke scene, it is possible to sing only the chorus, or skip the prelude and the coda, which not only saves the user's time, but also lowers the threshold for singing and attracts users to experience. In a song listening scene, it is also possible to allow users to listen to only the chorus segment, etc., so that users can quickly sing the song.

[0082] See also Figure 4 , Figure 4 A flowchart of a method for determining a repeated lyrics segment provided in an embodiment of the present application. Figure 2 For further description of the process of determining repeated lyrics segments in the corresponding embodiment, this embodiment can be compared with Figure 2 The corresponding embodiments are combined to obtain a further implementation method. This embodiment may include the following steps:

[0083] S401: marking the elements in the self-similarity matrix that are greater than the similarity threshold as similar elements;

[0084] S402: marking elements in the self-similarity matrix that are less than or equal to the similarity threshold as non-similar elements;

[0085] Among them, a pair of lyrics text lines whose similarity is greater than the similarity cutoff value can be determined based on the similar elements, and a pair of lyrics text lines whose similarity is less than or equal to the similarity cutoff value can be determined based on the dissimilar elements.

[0086] S403: Determine lyrics division boundaries according to the distribution of the similar elements and the non-similar elements in the self-similarity matrix, and use the lyrics division boundaries to determine repeated lyrics segments in the lyrics text.

[0087] As a feasible implementation method, this embodiment can cluster all the lyrics text lines according to the distribution of target type lyrics, and then obtain the lyrics division boundary; wherein the above target type lyrics include any one or a combination of any one of the end-of-paragraph lyrics, repeated lyrics, and lyrics in the same paragraph. Specifically, this embodiment can cluster according to the following principles to obtain the lyrics division boundary: (1) The end-of-paragraph lyrics are the lyrics division boundary; (2) There are lyrics division boundaries between non-adjacent repeated lyrics; (3) There are no lyrics division boundaries between lyrics in the same paragraph.

[0088] In this embodiment, the target type of lyrics can be determined by any one or a combination of methods 1 to 6:

[0089] Method 1: If m(i, j) is a similar element and m(i+1, j+1) is a non-similar element, then the jth line of lyrics is marked as the end-of-paragraph lyrics; wherein m is an element in the self-similar matrix, i is the number of rows of element m in the self-similar matrix, and j is the number of columns of element m in the self-similar matrix. The end-of-paragraph lyrics are the last line of lyrics of a corresponding segment of a music structure (such as a verse or chorus).

[0090] Method 2: If m(i, j) is a non-similar element and m(i+1, j+1) is a similar element, then mark the jth line of lyrics as the end of the paragraph.

[0091] Method 3: If m(i, j) and m(i+1, j+1) are both similar elements, the lyrics in the i-th row and the i+1-th row are marked as repeated lyrics.

[0092] Method 4: If m(i, i+1) is a similar element, the lyrics in the i-th line and the i+1-th line are marked as the same paragraph. The same paragraph of lyrics refers to two lines of lyrics belonging to the same segment of a music structure.

[0093] Method 5: If m(i, 1) is a similar element, then mark the i-1th line of lyrics as the end of the paragraph.

[0094] Method 6: If m(i, n) is a similar element, then the i-th line of lyrics is marked as the end-of-paragraph lyrics; wherein n is the total number of lines of lyrics in the lyrics text.

[0095] It is understandable that the target type lyrics may also include any one or a combination of paragraph start lyrics, repeated lyrics and same paragraph lyrics. The target type lyrics may also include any one or a combination of paragraph start lyrics, short circuit end lyrics, repeated lyrics and same paragraph lyrics.

[0096] See also Figure 5 , Figure 5 A schematic diagram of the comparison relationship between a lyric text and a self-similarity matrix provided in an embodiment of the present application, Figure 5 The figure includes the correspondence between the lyrics of the song "Nanping Evening Bell" and the self-similarity matrix. The color of each element in the self-similarity matrix represents the similarity between the corresponding two lines of lyrics. The larger the gray value of the element, the higher the similarity.

[0097] like Figure 5 As shown, this embodiment can use the value m(i, j) in the self-similarity matrix SSM to represent the similarity between the lyrics in the i-th row and the lyrics in the j-th row, and then determine the lyrics division boundary. Specifically, this embodiment can determine the lyrics division boundary through the following formulas:

[0098] s1:g(i)={j|(m(i,j)-λ)<0or(m(i+1,j+1)-λ)<0};

[0099] s2:c(i)={j|m(i,j)>λand m(i+1,j+1)>λ};

[0100] s3:f()={i|m(i,i+1)>λ};

[0101] s4:b()={i|m(i,1)>λ};

[0102] s5:e()={i|m(i,n)>λ}; where n refers to the total number of lines of lyrics, and the threshold λ represents the demarcation value of similarity, which can be set to 0.7. g() represents the lyrics division boundary determined by formula s1, c() represents the repeated lyrics determined by formula s2, f() represents the lyrics of the same paragraph determined by formula s3, b() represents the lyrics division boundary determined by formula s4, and e() represents the lyrics division boundary determined by formula s5.

[0103] s1 means that one of m(i,j) and m(i+1,j+1) is greater than λ and the other is less than λ, which means that the lyrics line i is similar to line j and line i+1 is not similar to line j+1 (or that the lyrics line i is not similar to line j and line i+1 is similar to line j+1), that is, for the i-th line, find the corresponding lyrics division boundary j. Figure 5 , g()={8,22,25}, the 8th, 22nd and 25th lines of lyrics are the boundaries of lyrics division.

[0104] s2 means that the lyrics line i+1 is similar to line j+1, so line i and line i+1 are repeated segments in the lyrics. Figure 5 , c()={1,19}, the lyrics in the 1st and 19th lines are repeated lyrics.

[0105] s3 means that the i+1th adjacent line of the lyrics is similar to the i+1th line, which is suitable to be classified as the same paragraph.

[0106] s4 means that the lyrics line i is similar to line 1, and the i-1th line can be used as the dividing boundary. Figure 5 , b()={1,5,19}, so the 4th and 18th lines of lyrics can be used as lyrics division boundaries. Among them, the first line is the starting line, and the lyrics division boundary in this embodiment can be the paragraph ending line except the last line of lyrics.

[0107] s5 indicates that the ith line of lyrics is similar to the last line, and the ith line can be used as the dividing boundary. Figure 5 , e()={18,32}, so the 18th line of lyrics can be used as the lyrics division boundary.

[0108] The following is an example of an actual application to illustrate the process described in the above embodiment. Figure 6 , Figure 6 A flowchart of an audio recording solution provided for an embodiment of the present application. Since users have a lot of fragmented time, and a large part of a song that is more impressive or can be sung is the climax part, only allowing users to sing the climax part can greatly reduce the user's usage threshold and improve user participation. For example, the singing function in the WeChat applet of Kugou can achieve chorus by structurally segmenting the most impressive or widely sung segments of the song, and user A and user B sing the upper and lower segments respectively. The above method can help users complete the interactive closed loop within the applet. This embodiment enables users to freely record specific music clips. The specific process of this embodiment is as follows: WeChat searches for music-related keywords; users enter the applet and select the recording mode; prioritize the recording of short audio chorus; guide sharing to the relationship chain scene; the sharee enters the applet from the relationship chain scene, and the two users complete the short audio chorus of the chorus part, and finally guide users to share the chorus work.

[0109] The present application also provides a music structure analysis device, which may include:

[0110] A similarity calculation module is used to obtain the lyrics text of the target music and calculate the text line similarity of the lyrics text; wherein the text line similarity includes the similarity between any two lyrics text lines in the lyrics text;

[0111] A repeated segment query module generates a self-similarity matrix of the lyrics text according to the text line similarity, and uses the self-similarity matrix to query repeated lyrics segments in the lyrics text; wherein each of the repeated lyrics segments includes at least one of the lyrics text lines;

[0112] The music structure determination module is used to divide the music structure of the target music according to the position of the repeated lyrics segment.

[0113] This embodiment obtains the lyrics text of the target music and calculates the similarity between any two lines of lyrics text in the lyrics text, and then generates a self-similarity matrix of the lyrics text according to the text line similarity. Since the lyrics text has the same structure as the target music, and there are repeated lyrics fragments in the lyrics text, the music structure of the target music can be divided based on the position of the repeated lyrics fragments in the lyrics text. The above process divides the music structure based on the lyrics of the target music, the repeatability of the lyrics is relatively consistent, and the division process does not depend on the audio features in the audio file, which can improve the accuracy of music structure analysis.

[0114] Furthermore, the similarity calculation module is used to calculate the text line semantic similarity and / or text line structural similarity of the lyrics text.

[0115] Furthermore, the repeated segment query module is used to perform weighted calculation on the text line semantic similarity and the text line structural similarity if the text line similarity includes the text line semantic similarity and the text line structural similarity to generate a self-similarity matrix of the lyrics text.

[0116] Furthermore, the process of the similarity calculation module calculating the text line structure similarity of the lyrics text includes: extracting the structure sequence of the lyrics text, and calculating the text line structure similarity of the lyrics text based on the structure sequence; wherein the structure sequence includes any one or a combination of any several of a sentence word number structure sequence, a pinyin structure sequence and a part-of-speech structure sequence.

[0117] Furthermore, it also includes:

[0118] A label adding module, used for dividing the target music into a plurality of music segments according to the music structure, and adding corresponding music structure labels to all the music segments;

[0119] The picture generation module is used for determining the music structure tag to be sung according to the music connection instruction if a music connection instruction is received, and generating a picture to be shared; wherein the picture to be shared includes a music playback link corresponding to the music structure tag to be sung;

[0120] The picture sending module is used to send the picture to be shared to the target device, so that the target device plays the music clip corresponding to the music structure tag to be sung through the music playing link in the picture to be shared and records the user's singing sound.

[0121] Furthermore, the process of the picture generation module generating the picture to be shared includes: determining the music clip that has been sung according to the music connection instruction, and generating a lyrics picture of the music clip that has been sung; determining the structure tag of the music that has been sung according to the music connection instruction, and generating a lyrics picture corresponding to the structure tag of the music that has been sung and the structure tag of the music to be sung; adding the QR code of the music playback link to the lyrics picture to obtain the picture to be shared.

[0122] Furthermore, the repeated fragment query module includes:

[0123] An element query unit, used to mark elements in the self-similarity matrix that are greater than a similarity cutoff value as similar elements; and also used to mark elements in the self-similarity matrix that are less than or equal to the similarity cutoff value as non-similar elements;

[0124] The boundary determination unit is used to determine the lyrics division boundary according to the distribution of the similar elements and the non-similar elements in the self-similarity matrix, and use the lyrics division boundary to determine the repeated lyrics segments in the lyrics text.

[0125] Further, the boundary determination unit includes:

[0126] The first query unit is used for marking the jth line of lyrics as the end of the paragraph if m(i, j) is a similar element and m(i+1, j+1) is a non-similar element; wherein m is an element in the self-similar matrix, i is the number of rows of element m in the self-similar matrix, and j is the number of columns of element m in the self-similar matrix;

[0127] and / or, a second query unit, for marking the jth line of lyrics as the end-of-paragraph lyrics if m(i, j) is a non-similar element and m(i+1, j+1) is a similar element;

[0128] and / or, a third query unit, for marking the lyrics in the i-th row and the i+1-th row as repeated lyrics if m(i, j) and m(i+1, j+1) are both similar elements;

[0129] and / or, a fourth query unit, for marking the lyrics in the i-th line and the i+1-th line as the same paragraph of lyrics if m(i, i+1) is a similar element;

[0130] and / or, a fifth query unit, for marking the i-1th line of lyrics as the end-of-paragraph lyrics if m(i, 1) is a similar element;

[0131] and / or, a sixth query unit, for marking the i-th line of lyrics as the end-of-paragraph lyrics if m(i, n) is a similar element; wherein n is the total number of lines of lyrics in the lyrics text;

[0132] A clustering unit is used to cluster all the lyrics text lines according to the distribution of target type lyrics to obtain the lyrics division boundary; wherein the target type lyrics include any one or a combination of any several of the lyrics at the end of a paragraph, repeated lyrics and lyrics in the same paragraph.

[0133] Since the embodiments of the apparatus part correspond to the embodiments of the method part, please refer to the description of the embodiments of the method part for the embodiments of the apparatus part, which will not be repeated here.

[0134] The present application also provides a storage medium on which a computer program is stored, and when the computer program is executed, the steps provided in the above embodiment can be implemented. The storage medium may include: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

[0135] The present application also provides an electronic device, see Figure 7 , a structural diagram of an electronic device provided in an embodiment of the present application, such as Figure 7 As shown, a processor 710 and a memory 720 may be included.

[0136] Among them, the processor 710 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 710 can be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 710 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 710 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 710 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0137] The memory 720 may include one or more computer-readable storage media, which may be non-transitory. The memory 720 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In this embodiment, the memory 720 is at least used to store the following computer program 721, wherein, after the computer program is loaded and executed by the processor 710, the relevant steps in the music structure analysis method disclosed in any of the aforementioned embodiments can be implemented. In addition, the resources stored in the memory 720 may also include an operating system 722 and data 723, etc., and the storage method may be temporary storage or permanent storage. Among them, the operating system 722 may include Windows, Linux, Android, etc.

[0138] In some embodiments, the electronic device may further include a display screen 730 , an input / output interface 740 , a communication interface 750 , a sensor 760 , a power source 770 , and a communication bus 780 .

[0139] certainly, Figure 7 The structure of the electronic device shown does not constitute a limitation on the electronic device in the embodiments of the present application. In actual applications, the electronic device may include Figure 7 More or fewer components than shown, or combinations of certain components.

[0140] The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same and similar parts between the various embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of this application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the claims of this application.

[0141] It should also be noted that, in this specification, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprises", "comprising" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the statement "comprising a ..." does not exclude the presence of other identical elements in the process, method, article or device including the element.

Claims

1. A music structure analysis method, characterized in that: include: Obtaining lyrics text of target music, and calculating text line similarity of the lyrics text; wherein the text line similarity includes the similarity between any two lyrics text lines in the lyrics text; A self-similarity matrix of the lyrics text is generated according to the text line similarity, and the self-similarity matrix is ​​used to query repeated lyrics segments in the lyrics text; wherein each repeated lyrics segment includes at least one lyrics text line; the repeated lyrics segments are two lyrics segments with a repeated pattern, and the repeated pattern is a lyrics pattern that meets preset constraints, and the preset constraints include any one or a combination of any several of the same number of lines, the same semantics, and the same structural sequence; Dividing the music structure of the target music according to the positions of the repeated lyrics segments; Wherein, using the self-similarity matrix to query repeated lyrics segments in the lyrics text includes: Marking elements in the self-similarity matrix that are greater than a similarity cutoff value as similar elements; Marking elements in the self-similarity matrix that are less than or equal to the similarity cutoff value as non-similar elements; The lyrics division boundary is determined according to the distribution of the similar elements and the non-similar elements in the self-similarity matrix, and the repeated lyrics segments in the lyrics text are determined using the lyrics division boundary.

2. The music structure analysis method according to claim 1, characterized in that: Calculating the text line similarity of the lyrics text includes: Calculate the text line semantic similarity and / or text line structural similarity of the lyrics text.

3. The music structure analysis method according to claim 2, characterized in that: Generating a self-similarity matrix of the lyrics text according to the text line similarity includes: If the text line similarity includes the text line semantic similarity and the text line structural similarity, a weighted calculation is performed on the text line semantic similarity and the text line structural similarity to generate a self-similarity matrix of the lyrics text.

4. The music structure analysis method according to claim 2, characterized in that: Calculating the text line structure similarity of the lyrics text includes: Extracting the structural sequence of the lyrics text, and calculating the text line structural similarity of the lyrics text according to the structural sequence; The structure sequence includes any one or a combination of any several of a sentence word number structure sequence, a pinyin structure sequence and a part-of-speech structure sequence.

5. The music structure analysis method according to claim 1, characterized in that: After dividing the music structure of the target music according to the positions of the repeated lyrics segments, the method further includes: Dividing the target music into multiple music clips according to the music structure, and adding corresponding music structure tags to all the music clips; If a music singing instruction is received, the music structure tag to be sung is determined according to the music singing instruction, and a picture to be shared is generated; wherein the picture to be shared includes a music playback link corresponding to the music structure tag to be sung; The picture to be shared is sent to a target device so that the target device plays the music clip corresponding to the music structure tag to be sung through the music play link in the picture to be shared and records the user's singing voice.

6. The music structure analysis method according to claim 5, characterized in that: The generating of the picture to be shared includes: Determine the music structure tag of the sung music according to the music singing instruction, and generate lyrics pictures corresponding to the music structure tag of the sung music and the music structure tag to be sung; Add the QR code of the music play link to the lyrics picture to obtain the picture to be shared.

7. The music structure analysis method according to claim 1, characterized in that: Determining the lyrics division boundary according to the distribution of the similar elements and the non-similar elements in the self-similarity matrix includes: If m(i, j) is a similar element and m(i+1, j+1) is a non-similar element, then the j-th row of lyrics is marked as the end of the paragraph; wherein m is an element in the self-similar matrix, i is the number of rows of element m in the self-similar matrix, and j is the number of columns of element m in the self-similar matrix; And / or, if m(i, j) is a non-similar element and m(i+1, j+1) is a similar element, then mark the jth line of lyrics as the end-of-paragraph lyrics; And / or, if m(i, j) and m(i+1, j+1) are both similar elements, then the lyrics in the i-th row and the i+1-th row are marked as repeated lyrics; And / or, if m(i, i+1) is a similar element, the lyrics in the i-th line and the i+1-th line are marked as the same paragraph of lyrics; and / or, if m(i, 1) is a similar element, then the i-1th line of lyrics is marked as the end-of-paragraph lyrics; And / or, if m(i, n) is a similar element, then the i-th line of lyrics is marked as the end-of-paragraph lyrics; wherein n is the total number of lines of lyrics in the lyrics text; All the lyrics text lines are clustered according to the distribution of target type lyrics to obtain the lyrics division boundary; wherein the target type lyrics include any one or a combination of any one of the paragraph ending lyrics, repeated lyrics and the same paragraph lyrics.

8. An electronic device, characterized in that: It comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the music structure analysis method as claimed in any one of claims 1 to 7 when calling the computer program in the memory.

9. A storage medium, characterized in that: The storage medium stores computer executable instructions, and when the computer executable instructions are loaded and executed by the processor, the steps of the music structure analysis method as described in any one of claims 1 to 7 are implemented.

Images