Song teaching template generation method and system, equipment and program product

Abstract

The invention discloses a song teaching template generation method and system, equipment and a program product. The method comprises the following steps: acquiring song audio information of a target song audio; wherein the song audio information comprises lyrics, lyric timestamps, time points corresponding to paragraphs and audio representation information; performing various intra-sentence teaching feature analysis on the lyric timestamps and the audio representation information of the target song audio, and marking various analyzed intra-sentence teaching features on lyrics; wherein the in-sentence teaching features comprise at least one of accent, treble, long tone, tremolo, gliding and breathing points; the singing difficulty feature comprises at least one of high pitch and long pitch; the skill features comprise at least one of tremolo and glide; on the basis of the time point corresponding to each paragraph and the audio representation information, performing in-segment teaching feature marking on the loudness gradient part of each paragraph in the lyrics of the target song audio; and generating a song teaching template based on the marked lyrics of the target song audio and outputting the song teaching template.

Description

Technical Field

[0001] The present application relates to the technical field of audio data processing, and in particular to a method and system, device, and program product for generating a song teaching template. Background Art

[0002] Currently, many users practice singing and recording songs through audio playback software, and are no longer limited to listening to songs. Therefore, simply providing the lyrics of the songs is no longer enough to meet the needs. Therefore, other information that can guide singing is added to the lyrics, and teaching templates are generated and provided to users.

[0003] The current method of generating a singing teaching template is mainly to generate the pitch information and lyrics information of the lyrics based on the audio data of the song, and then combine the lyrics and pitch information of the song to generate a teaching template for the song. When the song is played, the lyrics and pitch information can be synchronously displayed through the teaching template of the song, making it convenient for users to sing the song according to the pitch.

[0004] However, since the singing teaching template generated by the existing method only includes the lyrics and pitch information of the song, the guidance it can provide can only meet the general singing needs, but cannot meet the needs of professional music teaching and creation. Therefore, the traditional singing teaching template for music teaching or creation is still manually annotated, which is very inconvenient. Summary of the invention

[0005] Based on the above-mentioned deficiencies in the prior art, the present application provides a song teaching template generation method and system, device, and program product to solve the problem that the teaching templates generated by the prior art cannot meet professional needs.

[0006] In order to achieve the above objectives, this application provides the following technical solutions:

[0007] The first aspect of the present application provides a method for generating a song teaching template, comprising:

[0008] Acquire song audio information of the target song audio; wherein the song audio information includes lyrics, lyrics timestamps, time points corresponding to each paragraph, and audio representation information;

[0009] Performing various types of intra-sentence teaching feature analyses on the lyrics timestamp and the audio representation information of the target song audio respectively, and marking each feature mark of the analyzed various types of intra-sentence teaching features on the lyrics of the target song audio; wherein the intra-sentence teaching features include at least one of stress, singing difficulty features, skill features, and breathing points; the singing difficulty features include at least one of high notes and long notes; the skill features include at least one of vibrato and glissando;

[0010] Based on the time points corresponding to the respective paragraphs and the audio representation information, the loudness gradient part of the respective paragraphs in the lyrics of the target song audio is marked with intra-segment teaching features;

[0011] A song teaching template for the target song audio is generated based on the marked lyrics of the target song audio and outputted.

[0012] Optionally, in the above-mentioned song teaching singing template generation method, the step of obtaining song audio information of the target song audio includes:

[0013] Get the target song audio;

[0014] Performing lyrics recognition on the target song audio through speech recognition technology to obtain the lyrics and lyrics timestamps of the target song audio, and performing verse and chorus detection on the target song audio to obtain the time points corresponding to each paragraph of the target song audio;

[0015] Perform audio analysis on the target song audio to obtain audio representation information of the target song audio.

[0016] Optionally, in the above-mentioned song teaching template generation method, the audio analysis of the target song audio to obtain the audio representation information of the target song audio includes:

[0017] The target song audio is separated into tracks, the separated vocal audio is extracted, and the pitch of the target song audio is recognized using automatic transcription technology to generate a MIDI file of the target song audio.

[0018] Optionally, in the above-mentioned song teaching template generation method, when the intra-sentence teaching feature is a high pitch, the lyrics timestamp and the audio representation information of the target song audio are respectively analyzed for various types of intra-sentence teaching features, and each feature mark of each type of intra-sentence teaching feature analyzed is marked on the lyrics of the target song audio, including:

[0019] Extracting the sound signal of each word in the lyrics of the target song audio from the vocal audio of the target song audio;

[0020] The sound signal of each word is analyzed and processed through the A-Weight curve, and the loudness value of each word;

[0021] Determine the duration coefficient corresponding to each word based on the lyrics timestamp;

[0022] The loudness value of each word is adjusted using the duration coefficient corresponding to each word to obtain the energy value of each word;

[0023] The words in the lyrics of the target song audio whose energy values ​​are greater than the energy mean of the paragraph they belong to are marked as strong tones, and the words in the lyrics of the target song audio whose energy values ​​are not greater than the energy mean of the paragraph they belong to are marked as light tones; wherein the energy mean of a paragraph is the average of the energy values ​​of all the words in the paragraph.

[0024] Optionally, in the above-mentioned song teaching template generation method, when the sentence teaching feature is a singing difficulty feature, the lyrics timestamp and the audio representation information of the target song audio are respectively analyzed for various types of sentence teaching features, and each feature mark of each type of sentence teaching feature analyzed is marked on the lyrics of the target song audio, including:

[0025] Based on the MIDI file of the target song audio, mark the notes whose pitch is greater than the pitch threshold as high notes at the position where the lyrics of the target song audio are located, and mark the notes whose duration is greater than the duration threshold as long notes at the position where the lyrics of the target song audio are located;

[0026] Based on the lyrics timestamp, the speed change rates between the lyrics are detected respectively, and the speed mutation is marked between every two lyrics where the speed change rate is greater than the change rate threshold.

[0027] Optionally, in the above-mentioned song teaching template generation method, when the sentence teaching feature is a breathing point, the lyrics timestamp and the audio representation information of the target song audio are respectively analyzed for various types of sentence teaching features, and each feature mark of each type of sentence teaching feature analyzed is marked on the lyrics of the target song audio, including:

[0028] The MIDI file of the target song audio is identified by using a mutual detection algorithm, and every two notes in the lyrics of the target song audio whose interval is greater than a preset interval are obtained and the breathing points are marked;

[0029] The lyrics timestamps are identified using a mutual detection algorithm to obtain every two words in the lyrics of the target song audio whose interval is greater than the preset interval time and mark the breathing points.

[0030] Optionally, in the above-mentioned song teaching template generation method, when the sentence teaching feature is a skill feature, the lyrics timestamp and the audio representation information of the target song audio are respectively analyzed for various types of sentence teaching features, and each feature mark of each type of sentence teaching feature analyzed is marked on the lyrics of the target song audio, including:

[0031] Extracting the melody fundamental frequency of each word of the lyrics of the target song audio from the vocal audio based on the lyrics timestamp;

[0032] The melody fundamental frequency of each word is calculated using fast Fourier transform to obtain the vibration frequency of each word;

[0033] Mark the words on the lyrics of the target song audio with a vibration frequency within a preset vibration frequency range and a duration greater than a preset vibration duration as vibrato;

[0034] Based on the lyrics timestamp and the vocal audio, audio detection is performed on each line of the lyrics of the target song audio, and the glissando of each line of the lyrics of the target song audio is obtained and marked.

[0035] Optionally, in the above-mentioned song teaching template generation method, based on the time points corresponding to the respective paragraphs and the audio representation information, the loudness gradient part of the respective paragraphs in the lyrics of the target song audio is marked with intra-segment teaching features, including:

[0036] Based on the time points corresponding to the respective paragraphs, respectively, loudness change analysis is performed on the audio of the sentences in each of the paragraphs in the original audio to obtain loudness change information of the sentences in each of the paragraphs;

[0037] Based on the sentence loudness variation information in each of the paragraphs, the lyrics of the part with gradually increasing loudness and the lyrics of the part with decreasing loudness in each of the paragraphs are marked.

[0038] Optionally, in the above-mentioned song teaching template generation method, the audio analysis of the target song audio to obtain the audio representation information of the target song audio includes:

[0039] The target song audio is input into an audio feature extraction model to extract audio representation information of the target song audio; wherein the audio representation information of the target song audio includes feature information of multiple dimensions of each frame in each audio channel.

[0040] Optionally, in the above-mentioned song teaching template generation method, the lyrics timestamp and the audio representation information of the target song audio are respectively analyzed for various types of intra-sentence teaching features, and each feature mark of each type of intra-sentence teaching features analyzed is marked on the lyrics of the target song audio, including:

[0041] Generate lyrics timestamp input information based on the lyrics time; wherein the lyrics timestamp input information includes the start time point and the end time point of each word in the lyrics of the target song audio;

[0042] The audio representation information of the target song audio and the lyrics timestamp input information are input into a pre-trained intra-sentence recognition model, and feature processing is performed through each feature processing layer in the intra-sentence model to obtain processing features, and the scores corresponding to each type of intra-sentence teaching features in the output layer of the intra-sentence model are used to output the processing features to obtain recognition results of each type of intra-sentence teaching features;

[0043] According to the recognition results of each type of the sentence teaching features, each feature mark of the each type of the sentence teaching features is marked on the lyrics of the target song audio.

[0044] Optionally, in the above-mentioned song teaching template generation method, based on the time points corresponding to the respective paragraphs and the audio representation information, the loudness gradient part of the respective paragraphs in the lyrics of the target song audio is marked with intra-segment teaching features, including:

[0045] The time point corresponding to the paragraph and the audio representation information are input into a pre-trained intra-segment recognition model to identify the intra-segment teaching features in each of the paragraphs, and the identified intra-segment teaching features of each of the paragraphs are marked in the lyrics of the target song audio.

[0046] Optionally, in the above-mentioned song teaching template generation method, before outputting the marked lyrics of the target song audio as the song teaching template of the target song audio, it also includes:

[0047] The various marks that are in the same position and belong to the same type combination are merged into corresponding merged marks, and the various marks that are in the same position but do not belong to the same merged type group are displayed according to the priority of the category they belong to; wherein each of the type combinations includes at least two mark types that are required to be merged.

[0048] The second aspect of the present application provides a song teaching template generation system, comprising:

[0049] An information acquisition unit, used to acquire song audio information of the target song audio; wherein the song audio information includes lyrics, lyrics timestamps, time points corresponding to each paragraph, and audio representation information;

[0050] The intra-sentence skill marking unit is used to respectively analyze various intra-sentence teaching features on the lyrics timestamp and the audio representation information of the target song audio, and mark each feature mark of the analyzed various intra-sentence teaching features on the lyrics of the target song audio; wherein the intra-sentence teaching features include stress, singing difficulty features, skill features, and breathing points; the singing difficulty features include at least high notes and long notes; the skill features include vibrato and glissando;

[0051] An intra-segment skill marking unit, for marking intra-segment teaching features on the loudness gradient part of each of the paragraphs in the lyrics of the target song audio based on the time points corresponding to each of the paragraphs and the audio representation information;

[0052] The template output unit is used to generate and output a song teaching template for the target song audio based on the marked lyrics of the target song audio.

[0053] A third aspect of the present application provides a computer device, including a memory and a processor;

[0054] The memory is used to store computer programs;

[0055] The processor is used to execute the computer program to implement the song teaching template generation method provided in any one of the first aspects of the present application.

[0056] A fourth aspect of the present application provides a computer program product, comprising executable computer instructions;

[0057] When the computer instructions are executed by the processor, they are used to implement the song teaching template generation method provided in any one of the first aspects of the present application.

[0058] The present application provides a method for generating a song teaching singing template, which first obtains the song audio information of the target song audio. Among them, the song audio information includes lyrics, lyrics timestamps, time points corresponding to each paragraph, and audio representation information. Then, various types of intra-sentence teaching feature analysis are performed on the lyrics timestamps and audio representation information of the target song audio, and various feature marks of various types of intra-sentence teaching features analyzed are marked on the lyrics of the target song audio. Among them, the intra-sentence teaching features include at least one of stress, singing difficulty features, skill features, and breathing points. The singing difficulty features include at least one of high notes and long notes. The skill features include at least one of vibrato and glissando. Therefore, multiple types of skills are marked for each lyric, and based on the time points corresponding to each paragraph and the audio representation information, the loudness gradient part of each paragraph in the lyrics of the target song audio is marked with intra-segment teaching features, thereby also marking the skills of the paragraph, and finally generating the song teaching singing template of the target song audio based on the marked target song audio lyrics and outputting it, thereby obtaining a song teaching singing template with comprehensive singing skill marks, which can meet professional music teaching and creation needs, etc. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0060] Figure 1 A flow chart of a method for generating a song teaching template provided in an embodiment of the present application;

[0061] Figure 2 A schematic diagram of an example of a song teaching template provided in an embodiment of the present application;

[0062] Figure 3 A flowchart of a method for marking high notes provided in an embodiment of the present application;

[0063] Figure 4 A flowchart of a method for marking skill features provided in an embodiment of the present application;

[0064] Figure 5 A logic diagram of a method for generating a song teaching template provided in an embodiment of the present application;

[0065] Figure 6 A flowchart of a method for marking teaching features within a sentence provided in an embodiment of the present application;

[0066] Figure 7 A logic diagram of another method for generating a song teaching template provided in an embodiment of the present application;

[0067] Figure 8 A schematic diagram of the architecture of a song teaching template generation system provided in an embodiment of the present application;

[0068] Fig. 9 A schematic diagram of the architecture of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0069] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0070] In this application, relational terms such as first and second, etc. are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the statement "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device including the element.

[0071] The present application embodiment provides a method for generating a song teaching template, such as Figure 1 As shown, the following steps are included:

[0072] S101, obtaining song audio information of a target song audio.

[0073] The target song audio is the audio data of the song for which the teaching singing template is currently generated. The song audio information includes lyrics, lyrics timestamps, time points corresponding to each paragraph, and audio representation information.

[0074] It should be noted that since the lyrics need to be displayed in the teaching singing template and the singing skills of each part of the lyrics need to be marked, the lyrics of the target song audio need to be identified. In order to locate and mark each word or each part of the lyrics, and to analyze the corresponding singing skills based on the time length corresponding to the lyrics, the lyrics timestamp, that is, the timestamp of the lyrics of the entire song, is identified. Specifically, it can include the timestamp of each sentence of lyrics and the timestamp of each word.

[0075] Since the verse, chorus, and bridge of a song are very different in singing, the singing atmosphere of each section needs to be analyzed, so the target song audio needs to be tested for the verse and chorus to obtain the time points corresponding to each section of the target song audio. The time points corresponding to each section need to correspond to the timestamps of each line of lyrics, so that it can be determined which paragraphs of lyrics belong to the verse, chorus, etc.

[0076] The song audio itself has many features that determine the singing skills, such as loudness, pitch, etc., so it is also necessary to perform audio analysis on the target song audio as needed to obtain the audio representation information of the target song audio.

[0077] Optionally, when it is necessary to generate a target song audio teaching target, the target song audio is first obtained, and then the lyrics of the target song audio are recognized by speech recognition technology to obtain the lyrics and lyrics timestamps of the target song audio, and the main and chorus of the target song audio are detected to obtain the time points corresponding to each paragraph of the target song audio. In addition, the audio of the target song audio is analyzed to obtain the audio representation information of the target song audio.

[0078] Optionally, the target song audio may be obtained from a database, downloaded online, or uploaded by a user, etc. The lyrics may be recognized by automatic speech recognition technology (ASR) and a lyrics timestamp may be generated.

[0079] S102, respectively performing various intra-sentence teaching feature analyses on the lyrics timestamp and audio representation information of the target song audio, and marking each feature mark of the analyzed various intra-sentence teaching features on the lyrics of the target song audio.

[0080] Since the position and duration of each word in the lyrics can be determined based on the lyrics timestamp, and the interval between each word can also be determined, and the audio representation information feeds back the audio features of each word, based on the lyrics timestamp and audio representation information, the teaching features corresponding to each word in a sentence of lyrics can be analyzed and marked on the lyrics. In addition, since the teaching singing template not only shows the lyrics but also the pitch, Figure 2 As shown, not only the lyrics but also the pitch information can be marked.

[0081] Among them, the intra-sentence teaching features are the features related to teaching in the sentence dimension of the lyrics, which may specifically include at least one of stress, singing difficulty features, technical features, and breathing points. The singing difficulty features include at least one of high notes and long notes. The technical features include at least one of vibrato and glissando.

[0082] Specifically, based on the characteristics of various intra-sentence teaching features, the lyrics timestamps and audio representation information of the target song audio can be analyzed, that is, the lyrics timestamps and audio representation information at which position in the lyrics meet the characteristics of which type of intra-sentence teaching features, and corresponding marking is performed at that position.

[0083] Alternatively, if Figure 2 As shown, in order to distinguish different intra-sentence teaching features and let users know the meanings of various intra-sentence teaching features, the intra-sentence teaching features can be marked with text and corresponding symbols.

[0084] S103: Based on the time points corresponding to each paragraph and the audio representation information, the loudness gradient part of each paragraph in the lyrics of the target song audio is marked with intra-segment teaching features.

[0085] It should be noted that the intra-segment feature is the overall feature of a song. However, the gradual change of loudness in a song is relatively difficult to be noticed by the user and thus ignored. Therefore, in the embodiment of the present application, the intra-segment teaching feature mainly marks the part of the gradual change of loudness, so that the user can know which part of the song needs to change its loudness gradually when singing.

[0086] Therefore, each paragraph is determined according to the time point corresponding to each paragraph, and based on the loudness in the audio representation information corresponding to each paragraph, the loudness gradual change part is analyzed and marked. Among them, the loudness gradual change part includes the part where the loudness gradually increases and the part where the loudness gradually decreases.

[0087] S104. Generate a song teaching template for the target song audio based on the marked lyrics of the target song audio and output it.

[0088] Optionally, the marked target song audio lyrics and the target song audio pitch information can be combined to form a song teaching template for the target song audio and output it. Figure 2 The song teaching template shown, the upper part is the pitch information of the target song audio displayed synchronously with the song playing synchronization time, and the lower part is the lyrics of the marked target song audio displayed synchronously with the song playing synchronization time.

[0089] Optionally, in order to make the mark of the song teaching template more concise and clear, in another embodiment of the present application, after executing step S103 and before executing step S104, the following may be further performed:

[0090] The individual tags at the same position and belonging to the same type combination are merged into corresponding merged tags, and the individual tags at the same position but not belonging to the same merged type group are displayed according to the priority of the category they belong to.

[0091] Each type combination includes at least two types of marks that need to be merged, that is, multiple types of marks that need to be merged are defined as a type combination, for example, high notes and accents are defined as a type combination. When a certain position is marked as high notes and accents at the same time, the two marks can be merged into a defined merged case, such as merging into "high intensity".

[0092] For those that cannot be merged, they need to be displayed according to the priority of the pre-defined various marks. The specific settings can be made according to the needs, that is, the more important the mark of the skill is, the higher the priority is defined. For example, if the mark of the test item overlaps with the mark of the skill item, it is usually marked as a skill first. For example, when the long tone in the difficulty and the trill in the skill appear in the same word at the same time, the word will be marked as trill first.

[0093] Optionally, in order to continuously improve the accuracy of feature extraction and labeling, after outputting the song teaching template, it is also possible to receive user feedback or expert annotations, medical data, and optimize the feature extraction and labeling algorithms, thereby continuously improving the quality of the template.

[0094] A method for generating a song teaching singing template provided in an embodiment of the present application first obtains the song audio information of the target song audio. The song audio information includes lyrics, lyrics timestamps, time points corresponding to each paragraph, and audio representation information. Then, various types of intra-sentence teaching feature analysis are performed on the lyrics timestamps and audio representation information of the target song audio, and various feature marks of various types of intra-sentence teaching features analyzed are marked on the lyrics of the target song audio. The intra-sentence teaching features include at least one of stress, singing difficulty features, skill features, and breathing points. The singing difficulty features include at least one of high notes and long notes. The skill features include at least one of vibrato and glissando. Therefore, multiple types of skills are marked for each lyric, and based on the time points corresponding to each paragraph and the audio representation information, the loudness gradient part of each paragraph in the lyrics of the target song audio is marked with intra-segment teaching features, thereby also marking the skills of the paragraph, and finally generating and outputting the song teaching singing template of the target song audio based on the marked target song audio lyrics, thereby obtaining a song teaching singing template with comprehensive singing skill marks, which can meet professional music teaching and creation needs.

[0095] Optionally, in another embodiment of the present application, a specific implementation method of performing audio analysis on the target song audio to obtain audio representation information of the target song audio includes:

[0096] The target song audio is separated by audio tracks, and the separated vocal audio is extracted. The target song audio is pitch recognized by using automatic transcription technology to generate a MIDI file of the target song audio.

[0097] Among them, MIDI files refer to files that comply with the MIDI (Musical Instrument Digital Interface) digital interface standard for connecting electronic musical instruments and computers. Therefore, by generating a MIDI file of the target song audio, the target song audio is converted into a format that can be processed by a computer.

[0098] It should be noted that, since the teaching singing template is to guide the user to sing, in the embodiment of the present application, the target song audio is separated into tracks, that is, the vocals and the accompaniment are separated, and then the separated vocal audio is extracted for analysis. In order to analyze the pitch of the target song audio and subsequently display the pitch information, the automatic transcription technology is used to identify the pitch information and generate a MIDI file.

[0099] Accordingly, in the embodiment of the present application, when the teaching feature within the sentence is a high tone, a specific implementation of step S102 is an implementation of marking the high tone, such as Figure 3 As shown, the following steps are included:

[0100] S301. Extract the sound signal of each word in the lyrics of the target song audio from the vocal audio of the target song audio.

[0101] S302, analyzing and processing the sound signal of each word through the A-Weight curve to obtain the loudness value of each word.

[0102] It should be noted that the A-Weight curve simulates the sensitivity of the human ear to different frequencies, so it can also obtain a loudness value that is more in line with the human ear. Specifically, the weighted time domain signal is obtained through frequency weighting and inverse transformation, and finally the loudness value of each word can be calculated.

[0103] S303: Determine the duration coefficient corresponding to each word based on the lyrics timestamp.

[0104] Specifically, the singing duration of each word is determined from the lyrics timestamp, and then the corresponding duration system is matched according to the duration.

[0105] S304: adjusting the loudness value of each word using the duration coefficient corresponding to each word to obtain the energy value of each word.

[0106] Since the energy value output by a word depends not only on its responsiveness but also on the duration of continuous output, it is necessary to use the duration coefficient corresponding to each word to adjust the loudness value of each word to obtain the energy value of each word.

[0107] S305. Mark the words in the lyrics of the target song audio whose energy values ​​are greater than the energy mean of the paragraph to which they belong as strong tones, and mark the words in the lyrics of the target song audio whose energy values ​​are not greater than the energy mean of the paragraph to which they belong as light tones.

[0108] The energy mean of a paragraph is the mean of the energy values ​​of all the words in the paragraph.

[0109] Specifically, peak detection is performed on the energy value, and words whose energy values ​​are greater than the energy mean of the paragraph to which they belong are marked as strong tones, and words in the lyrics of the target song audio whose energy values ​​are not greater than the energy mean of the paragraph to which they belong are marked as light tones.

[0110] Accordingly, in the embodiment of the present application, when the sentence teaching feature is a singing difficulty feature, a specific implementation of step S102, that is, an implementation of marking the singing difficulty feature, includes:

[0111] Based on the MIDI file of the target song audio, the notes with pitch greater than the pitch threshold are marked as high notes at the position of the lyrics of the target song audio, and the notes with duration greater than the duration threshold are marked as long notes at the position of the lyrics of the target song audio. Based on the lyrics timestamp, the speed change rate between the phrases of the lyrics is detected respectively, and the speed mutation is marked between every two lyrics where the speed change rate is greater than the change rate threshold.

[0112] Optionally, since the division of male high pitch and female high pitch is different, the pitch threshold of high pitch can be set for male and female voices respectively, for example, if the male voice is higher than F and the female voice is higher than D, the position of the note is marked. The position of the note can be determined according to the timestamp of the word corresponding to the note.

[0113] Accordingly, in the embodiment of the present application, when the intra-sentence teaching feature is a breathing point, a specific implementation of step S102, that is, an implementation of marking the breathing point, includes:

[0114] The MIDI file of the target song audio is identified by using a mutual detection algorithm to obtain every two notes in the lyrics of the target song audio whose interval is greater than a preset interval, and a breathing point is marked between the two notes. The lyrics timestamp is identified by using a mutual detection algorithm to obtain every two words in the lyrics of the target song audio whose interval is greater than a preset interval, and a breathing point is marked between the two words.

[0115] Since the breathing point is where you need to pause and breathe, we analyze and mark the breathing points from the dimensions of notes and words. For example, if the interval between words is greater than 250ms, mark the breathing points according to the lyrics. If the interval between notes is greater than 250ms, mark the breath points according to MIDI. Among them, you can mark the points between notes first, and then mark the points between words.

[0116] Accordingly, in the embodiment of the present application, when the sentence teaching feature is a skill feature, a specific implementation of step S102 is an implementation of marking the skill feature, such as Figure 4 As shown, including:

[0117] S401. Extract the melody fundamental frequency of each word of the lyrics of the target song audio from the vocal audio based on the lyrics timestamp.

[0118] Specifically, the vocal audio may be cropped at the word level in combination with the lyrics timestamp, thereby cropping the audio of each sub-song, and then extracting the melody fundamental frequency from the audio of each sub-song.

[0119] S402, using fast Fourier transform to calculate the melody fundamental frequency of each word to obtain the vibration frequency of each word.

[0120] S403, marking vibrato on the lyrics of the target song audio whose vibration frequency is within a preset vibration frequency range and whose duration is greater than a preset vibration duration.

[0121] Optionally, a vibration comment with a frequency between 4 and 8 Hz and a duration of more than 3 seconds may be specifically marked as a vibrato.

[0122] S404: Based on the lyrics timestamp and the vocal audio, perform audio detection on each line of the lyrics in the target song audio, obtain the glissando of each line of the lyrics in the target song audio and mark them.

[0123] Specifically, the human voice audio can be segmented according to the start and end time of the sentence to obtain audio of multiple lyrics, and the sentence-level audio is detected to obtain the portamento and marked. Optionally, HMM can be used to model and decode the portamento.

[0124] Accordingly, in the embodiment of the present application, a specific implementation of step S103 includes:

[0125] Based on the time points corresponding to each paragraph, the loudness change analysis is performed on the audio of the sentences in each paragraph in the original audio to obtain the sentence loudness change information in each paragraph, and based on the sentence loudness change information in each paragraph, the lyrics of the part with gradually increasing loudness and the lyrics of the part with decreasing loudness in each paragraph are marked.

[0126] Therefore, in the embodiment of the present application, the logic of generating the song teaching template can be as follows: Figure 5 As shown, the song audio is preprocessed first. Specifically, the song audio is transcribed to obtain vocal music, that is, a MIDI file; the lyrics, words and lyrics timestamps of the song audio are recognized through singing ASR; the vocal audio is separated through vocal accompaniment separation technology; the main and chorus of the song audio are detected to achieve main and chorus segmentation. Then, the data obtained from these preprocessing are analyzed and marked to obtain difficulty marks, breathing point marks, stress marks, skill marks, and paragraph crescendo and decrescendo marks. Finally, the marked data is verified and integrated, and a teaching singing template can be generated.

[0127] Due to limited machine resources, in another embodiment of the present application, a trained model is used to generate a teaching singing template. Since the analysis of the audio representation information of the target song audio is to extract the features of the target song audio, a feature extraction model can be used to perform audio features. Therefore, optionally, in another embodiment of the present application, another specific implementation method of performing audio analysis on the target song audio to obtain the audio representation information of the target song audio includes:

[0128] The target song audio is input into the audio feature extraction model to extract the audio representation information of the target song audio.

[0129] Among them, the audio representation information of the target song audio includes feature information of multiple dimensions of each frame in each audio channel. That is to say, in an embodiment of the present application, the audio representation information is a three-dimensional array. For example, the audio representation information extracted using the general audio representation extraction model-MERT can be expressed as (25, T, 1024). Among them, 25 represents the number of audio channels, so the audio representation information will include data for 25 audio channels. And T represents the number of time frames, that is, the number of frames of each audio channel. And 1024 is the feature dimension of each frame, that is, 1024 dimensions of features will be extracted for each frame in the audio representation information. Therefore, the total number of features in the audio representation information is: 25*T*1024.

[0130] Optionally, the training data of the model is obtained by the method of the previous embodiment to train the model.

[0131] Accordingly, in the embodiment of the present application, a specific implementation of step S102 is as follows: Figure 6 As shown, the following steps are included:

[0132] S601, generating lyrics timestamp input information based on lyrics time.

[0133] Optionally, since the teaching feature analysis is performed within a sentence, each line of lyrics can be processed separately, that is, lyrics timestamp input information of each line of lyrics is generated separately.

[0134] The lyrics timestamp input information includes the start time and end time of each word in the lyrics of the target song audio. That is, the lyrics timestamp input information is a two-dimensional element group (word_num, 2), where "word_num" represents the order of a word in a sentence of lyrics, that is, the number of words, and "2" represents the features of two dimensions, that is, the start time and end time of the word. For example, for the lyrics "I love China", its lyrics timestamp input information is expressed as: [

[0136] [0.0, 0.5], #The start and end time of "I"

[0137] [0.5, 1.0], #The start and end time of "love"

[0138] [1.0, 1.3], #The start and end time of "middle"

[0139] [1.3, 1.8], #The start and end time of "country"

[0140] ].

[0141] Optionally, considering that there may be breathing points between sentences, that is, there may be breathing points at the beginning of a sentence, a timestamp of a virtual position is added between the first words in the lyrics timestamp input information of each of the lyrics generated except the first one. For example, for the lyrics "I love China" in the above example, the lyrics timestamp input information is expressed as: [

[0143] [-0.25, 0.0], #virtual breathing point position

[0144] [0.0, 0.5], #The start and end time of "I"

[0145] [0.5, 1.0], #The start and end time of "love"

[0146] [1.0, 1.3], #The start and end time of "middle"

[0147] [1.3, 1.8], #The start and end time of "country"

[0148] ].

[0149] S602. The audio representation information and lyrics timestamp input information of the target song audio are input into a pre-trained intra-sentence recognition model, and feature processing is performed through each feature processing layer in the intra-sentence model to obtain processing features. The processing features are outputted using the scores corresponding to each type of intra-sentence teaching features in the output layer of the intra-sentence model to obtain recognition results of each type of intra-sentence teaching features.

[0150] The analysis of the intra-sentence teaching features is to analyze the points or words in the data that meet the various intra-sentence teaching features, so in the embodiment of the present application, the model is also used for processing.

[0151] Specifically, the audio representation information and lyrics timestamp input information are input into the input layer of the pre-trained intra-sentence recognition model, and then feature extraction is performed through the feature extraction layer. In the feature extraction process, it is first processed through a series of convolutional layers. The function of these convolutional layers is to extract local features from the input audio representation information and capture important features in the audio signal through multiple filters. After the convolution processing, the output feature map will be reduced in dimension by the maximum pooling layer to retain important information and reduce computational complexity. Next, the output of the convolutional layer is flattened into a one-dimensional array for easy input to the subsequent sequence processing layer. At this stage, the model makes full use of the local features of the audio signal to prepare for time series analysis.

[0152] The flattened feature data is then processed through the sequence processing layer. Optionally, the sequence processing layer can use a long short-term memory network (LSTM), which can capture long-distance dependencies in time series. In this layer, the model can learn the temporal dynamic changes of the audio signal. It is then processed through the attention mechanism, which helps the model focus more on the audio information at key moments by calculating the importance score of each word in the audio feature. Through weighted processing, the model can prioritize those features that have a greater impact on the task results when performing feature recognition.

[0153] It should be noted that after feature processing is performed through each feature processing layer in the internal model, the processed features can be obtained, and at this time, it is possible to further analyze which of these processed features are consistent with various types of intra-sentence teaching features. The characteristics of each intra-sentence teaching feature are different, so in the embodiment of the present application, corresponding branches are set for each type of intra-sentence teaching feature, so the processed features are output to each branch of the output layer for output processing, and the recognition results of various types of intra-sentence teaching features are obtained.

[0154] Optionally, for the stress output branch, a softmax activation function can be used, which outputs whether each word is a stress word. The specific output recognition result can be expressed as (batch_size, word_num). For example, if the model predicts that only "I" is a stress word in a sentence, then only the output value corresponding to this word in the output result is 1, and the output value of the remaining words is 0. The high-pitched output branch can also use the softmax activation function, and its recognition result is also in the form of (batch_size, word_num), which is used to identify which words have a higher pronunciation. The long sound output branch can use the sigmoid activation function. Since there is usually only one long sound in a sentence, its recognition result can be in the form of (batch_size, 1). The vibrato output branch can also use the sigmoid activation function, so its output result can also be in the form of (batch_size, word_num), which is used to identify whether each word contains vibrato features to ensure that the model can accurately capture these phonetic changes. The glide output branch also uses the sigmoid activation function, and the output shape is (batch_size, word_num), which is used to identify whether each word contains the glide feature. Since the glide can only appear on words, the model can accurately determine which words have the glide feature. For the breath point output branch, the sigmoid activation function can also be used, but since it is between two words, the output result is represented in the shape of (batch_size, (word_num-1)+word_num).

[0155] Correspondingly, corresponding loss functions and training configurations need to be configured for each type of intra-sentence teaching feature configuration. Optionally, the cross entropy loss function can be used for tasks such as stress, high pitch, vibrato, and glissando. For the recognition task of long sounds, the binary cross entropy loss function can be used. The breathing point also uses the sigmoid activation function and the binary cross entropy loss function to identify the breathing points at the beginning of the sentence and between words.

[0156] In terms of training configuration, the model can use the Adam optimizer. Specifically, the initial learning rate is set to 0.001. The batch size can be adjusted to 32 or 64 according to actual conditions, the training cycle is set to 50 epochs, and the learning rate and other hyperparameters are dynamically adjusted according to the performance of the validation set.

[0157] S603. According to the recognition results of each type of intra-sentence teaching features, mark each feature mark of each type of intra-sentence teaching features on the lyrics of the target song audio.

[0158] Since the recognition results of various types of intra-sentence teaching features include the specific positions of the identified intra-sentence teaching features, such as the number of words in a line of lyrics, or between two words, the corresponding feature markers of the intra-sentence teaching features can be marked at the corresponding positions on the lyrics of the target song audio based on the recognition results of various types of intra-sentence teaching features.

[0159] Correspondingly, in the embodiment of the present application, the teaching characteristics within the segment can also be analyzed with the same model, so a specific implementation of step S103 includes:

[0160] The time points and audio representation information corresponding to the paragraphs are input into a pre-trained intra-segment recognition model to identify the intra-segment teaching features in each paragraph, and the identified intra-segment teaching features of each paragraph are marked in the lyrics of the target song audio.

[0161] It should be noted that the input form of the time point corresponding to the paragraph is consistent with the lyrics timestamp input information, which only includes the start time point and the section number time point of each paragraph. In addition, the model converts the timestamp of the paragraph into the number of time frames to achieve paragraph segmentation, ensuring that the input features can accurately reflect the paragraph information.

[0162] Optionally, the time point corresponding to the paragraph and the audio representation information can be input into the input layer of the segment recognition model, and then processed by the convolutional neural network (CNN) and the long short-term memory network (LSTM). A fully connected layer is added after the LSTM layer to extract high-level features. The final output layer uses a sigmoid activation function with an output value of 0 or 1 to determine whether the input paragraph is gradually increasing or decreasing. Similarly, the model is trained using a loss function and an optimizer.

[0163] Therefore, the logic of the song teaching template generated by the embodiment of the present application is as follows: Figure 7 As shown in the figure, the song audio is first preprocessed, that is, the timestamps of the lyrics, words and sentences of the song audio are recognized by the singing ASR; the song audio is segmented by the main chorus detection, and the audio representation information of the song audio is extracted by the audio standard model. Then, the results of the singing ASR recognition and the audio representation information are analyzed by the internal model to obtain the difficulty mark, breathing point mark, stress mark, and skill mark. In addition, the segmentation result and the audio representation information are analyzed by the intra-segment recognition model to obtain the paragraph crescendo and decrescendo marks.

[0164] Another embodiment of the present application provides a song teaching template generation system, such as Figure 8 As shown, including:

[0165] The information acquisition unit 801 is used to acquire the song audio information of the target song audio.

[0166] The song audio information includes lyrics, lyrics timestamps, time points corresponding to each paragraph, and audio representation information.

[0167] The intra-sentence skill marking unit 802 is used to analyze various intra-sentence teaching features on the lyrics timestamp and audio representation information of the target song audio, and mark each feature mark of the analyzed various intra-sentence teaching features on the lyrics of the target song audio. Among them, the intra-sentence teaching features include at least one of stress, singing difficulty features, skill features, and breathing points. The singing difficulty features include at least one of high notes and long notes. The skill features include at least one of vibrato and glissando.

[0168] The intra-segment skill marking unit 803 is used to mark the intra-segment teaching features of the loudness gradient part of each paragraph in the lyrics of the target song audio based on the time points corresponding to each paragraph and the audio representation information.

[0169] The template output unit 804 is used to generate and output a song teaching template for the target song audio based on the marked lyrics of the target song audio.

[0170] Optionally, in a song teaching template generation system provided in another embodiment of the present application, the information acquisition unit includes:

[0171] The song acquisition unit is used to acquire the target song audio.

[0172] The lyrics recognition unit is used to recognize the lyrics of the target song audio through speech recognition technology to obtain the lyrics and lyrics timestamp of the target song audio.

[0173] The paragraph detection unit is used to detect the main and chorus of the target song audio and obtain the time points corresponding to each paragraph of the target song audio.

[0174] The audio feature analysis unit is used to perform audio analysis on the target song audio to obtain audio representation information of the target song audio.

[0175] Optionally, in a song teaching template generation system provided in another embodiment of the present application, the audio feature analysis unit includes:

[0176] The first audio feature analysis unit is used to separate the audio tracks of the target song audio, extract the separated vocal audio, and use the automatic transcription technology to perform pitch recognition on the target song audio to generate a MIDI file of the target song audio.

[0177] Optionally, in the song teaching singing template generation system provided in another embodiment of the present application, when the sentence teaching feature is a high pitch, the sentence skill marking unit performs various sentence teaching feature analyses on the lyrics timestamp and audio representation information of the target song audio, and marks each feature mark of the analyzed various sentence teaching features on the lyrics of the target song audio, for:

[0178] The sound signal of each word in the lyrics of the target song audio is extracted from the vocal audio of the target song audio.

[0179] The sound signal of each word is analyzed and processed through the A-Weight curve to obtain the loudness value of each word.

[0180] Determine the duration coefficient corresponding to each word based on the lyrics timestamp.

[0181] The loudness value of each word is adjusted using the duration coefficient corresponding to each word to obtain the energy value of each word.

[0182] The words in the lyrics of the target song audio whose energy value is greater than the energy mean of the paragraph to which they belong are marked as strong tones, and the words in the lyrics of the target song audio whose energy value is not greater than the energy mean of the paragraph to which they belong are marked as light tones. The energy mean of a paragraph is the average of the energy values ​​of all the words in the paragraph.

[0183] Optionally, in the song teaching singing template generation system provided by another embodiment of the present application, when the sentence teaching feature is a singing difficulty feature, the sentence skill marking unit performs various sentence teaching feature analyses on the lyrics timestamp and audio representation information of the target song audio, and marks each feature mark of the analyzed various sentence teaching features on the lyrics of the target song audio, for:

[0184] Based on the MIDI file of the target song audio, the notes whose pitch is greater than the pitch threshold are marked as high notes at the position of the lyrics of the target song audio, and the notes whose duration is greater than the duration threshold are marked as long notes at the position of the lyrics of the target song audio.

[0185] Based on the lyrics timestamp, the speed change rate between the lyrics is detected respectively, and the speed mutation is marked between every two lyrics when the speed change rate is greater than the change rate threshold.

[0186] Optionally, in the song teaching singing template generation system provided in another embodiment of the present application, when the sentence teaching feature is a breathing point, the sentence skill marking unit performs various sentence teaching feature analyses on the lyrics timestamp and audio representation information of the target song audio, and marks each feature mark of the analyzed various sentence teaching features on the lyrics of the target song audio, for:

[0187] The MIDI file of the target song audio is identified using a mutual detection algorithm to obtain every two notes in the lyrics of the target song audio whose interval is greater than the preset interval and mark the breathing points.

[0188] The lyrics timestamps are identified using a mutual detection algorithm to obtain the target song audio lyrics with intervals greater than the preset interval for every two words and mark the breathing points.

[0189] Optionally, in the song teaching singing template generation system provided in another embodiment of the present application, when the sentence teaching feature is a skill feature, the sentence skill marking unit performs various sentence teaching feature analyses on the lyrics timestamp and audio representation information of the target song audio, and marks each feature mark of the analyzed various sentence teaching features on the lyrics of the target song audio, for:

[0190] Based on the lyrics timestamp, the melody fundamental frequency of each word of the lyrics of the target song audio is extracted from the vocal audio.

[0191] The melody fundamental frequency of each word is calculated using fast Fourier transform to obtain the vibration frequency of each word.

[0192] The words on the lyrics of the target song audio whose vibration frequency is within the preset vibration frequency range and whose duration is greater than the preset vibration duration are marked with vibrato.

[0193] Based on the lyrics timestamp and vocal audio, the audio of each lyric in the target song audio is detected, and the glissando of each lyric in the target song audio is obtained and marked.

[0194] Optionally, in the song teaching singing template generation system provided in another embodiment of the present application, the intra-segment skill marking unit includes:

[0195] The loudness analysis unit is used to perform loudness change analysis on the audio of the sentences in each paragraph in the original audio based on the time points corresponding to each paragraph, so as to obtain the loudness change information of the sentences in each paragraph.

[0196] The intra-segment skill marking sub-unit is used to mark the lyrics of the part with increasing loudness and the lyrics of the part with decreasing loudness in each paragraph based on the sentence loudness change information in each paragraph.

[0197] Optionally, in a song teaching template generation system provided in another embodiment of the present application, the audio feature analysis unit includes:

[0198] The second audio feature analysis unit is used to input the target song audio into the audio feature extraction model to extract the audio representation information of the target song audio, wherein the audio representation information of the target song audio includes feature information of multiple dimensions of each frame in each audio channel.

[0199] Optionally, in the song teaching template generation system provided in another embodiment of the present application, the sentence skill marking unit includes:

[0200] The input generation unit is used to generate lyrics time stamp input information based on lyrics time. The lyrics time stamp input information includes the start time point and end time point of each word in the lyrics of the target song audio.

[0201] The intra-sentence skill recognition unit inputs the audio representation information and lyrics timestamp input information of the target song audio into a pre-trained intra-sentence recognition model, performs feature processing through each feature processing layer in the intra-sentence model to obtain processing features, and uses the scores corresponding to each type of intra-sentence teaching features in the output layer of the intra-sentence model to output the processing features to obtain the recognition results of various types of intra-sentence teaching features.

[0202] The first marking unit is used to mark each feature mark of each type of intra-sentence teaching feature on the lyrics of the target song audio according to the recognition results of each type of intra-sentence teaching feature.

[0203] Optionally, in the song teaching singing template generation system provided in another embodiment of the present application, the intra-segment skill marking unit includes:

[0204] The intra-segment skill recognition unit is used to input the time point corresponding to the paragraph and the audio representation information into a pre-trained intra-segment recognition model, identify the intra-segment teaching features in each paragraph, and mark the identified intra-segment teaching features of each paragraph in the lyrics of the target song audio.

[0205] Optionally, in the song teaching template generation system provided in another embodiment of the present application, it also includes:

[0206] The integration unit is used to merge the marks in the same position and belonging to the same type combination into corresponding merged marks, and to display the marks in the same position but not belonging to the same merged type group according to the priority of the category they belong to. Each type combination includes at least two mark types that are required to be merged.

[0207] It should be noted that the specific working process of each unit provided in the above embodiments of the present application can refer to the implementation process of the corresponding steps in the above method embodiments, which will not be repeated here.

[0208] The present application also provides a computer device, such as Fig. 9 As shown, the electronic device may include a memory 901 and a processor 902 .

[0209] The memory 901 is used to store computer programs.

[0210] Processor 902 is used to execute the above-mentioned computer program to implement the song teaching template generation method provided in any embodiment of the present application.

[0211] An embodiment of the present application also provides a computer program product, which may include executable computer instructions; when the computer instructions are executed by a processor (for example, by the above-mentioned processor 502), they are used to implement the song teaching template generation method provided in any embodiment of the present application.

[0212] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0213] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for generating a song teaching template, characterized in that: include: Acquire song audio information of the target song audio; wherein the song audio information includes lyrics, lyrics timestamps, time points corresponding to each paragraph, and audio representation information; Performing various types of intra-sentence teaching feature analyses on the lyrics timestamp and the audio representation information of the target song audio respectively, and marking each feature mark of the analyzed various types of intra-sentence teaching features on the lyrics of the target song audio; wherein the intra-sentence teaching features include at least one of stress, singing difficulty features, skill features, and breathing points; the singing difficulty features include at least one of high notes and long notes; the skill features include at least one of vibrato and glissando; Based on the time points corresponding to the respective paragraphs and the audio representation information, the loudness gradient part of the respective paragraphs in the lyrics of the target song audio is marked with intra-segment teaching features; A song teaching template for the target song audio is generated based on the marked lyrics of the target song audio and outputted.

2. The method according to claim 1, characterized in that The step of obtaining the song audio information of the target song audio includes: Get the target song audio; Performing lyrics recognition on the target song audio through speech recognition technology to obtain the lyrics and lyrics timestamps of the target song audio, and performing verse and chorus detection on the target song audio to obtain the time points corresponding to each paragraph of the target song audio; Perform audio analysis on the target song audio to obtain audio representation information of the target song audio.

3. The method according to claim 2, characterized in that The step of performing audio analysis on the target song audio to obtain audio representation information of the target song audio includes: The target song audio is separated into tracks, the separated vocal audio is extracted, and the pitch of the target song audio is recognized using automatic transcription technology to generate a MIDI file of the target song audio.

4. The method according to claim 3, characterized in that When the intra-sentence teaching feature is a high pitch, performing various intra-sentence teaching feature analyses on the lyrics timestamp and the audio representation information of the target song audio respectively, and marking each feature mark of the analyzed various intra-sentence teaching features on the lyrics of the target song audio, including: Extracting the sound signal of each word in the lyrics of the target song audio from the vocal audio of the target song audio; The sound signal of each word is analyzed and processed through the A-Weight curve, and the loudness value of each word; Determine the duration coefficient corresponding to each word based on the lyrics timestamp; The loudness value of each word is adjusted using the duration coefficient corresponding to each word to obtain the energy value of each word; The words in the lyrics of the target song audio whose energy values ​​are greater than the energy mean of the paragraph they belong to are marked as strong tones, and the words in the lyrics of the target song audio whose energy values ​​are not greater than the energy mean of the paragraph they belong to are marked as light tones; wherein the energy mean of a paragraph is the average of the energy values ​​of all the words in the paragraph.

5. The method according to claim 3, characterized in that: When the intra-sentence teaching feature is a singing difficulty feature, the lyrics timestamp and the audio representation information of the target song audio are respectively analyzed for various intra-sentence teaching features, and each feature mark of each type of intra-sentence teaching feature analyzed is marked on the lyrics of the target song audio, including: Based on the MIDI file of the target song audio, mark the notes whose pitch is greater than the pitch threshold as high notes at the position where the lyrics of the target song audio are located, and mark the notes whose duration is greater than the duration threshold as long notes at the position where the lyrics of the target song audio are located; Based on the lyrics timestamp, the speed change rates between the lyrics are detected respectively, and the speed mutation is marked between every two lyrics where the speed change rate is greater than the change rate threshold.

6. The method according to claim 3, characterized in that When the intra-sentence teaching feature is a breathing point, performing various intra-sentence teaching feature analyses on the lyrics timestamp and the audio representation information of the target song audio respectively, and marking each feature mark of the analyzed various intra-sentence teaching features on the lyrics of the target song audio, including: The MIDI file of the target song audio is identified by using a mutual detection algorithm, and every two notes in the lyrics of the target song audio whose interval is greater than a preset interval are obtained and the breathing points are marked; The lyrics timestamps are identified using a mutual detection algorithm to obtain every two words in the lyrics of the target song audio whose interval is greater than the preset interval time and mark the breathing points.

7. The method according to claim 3, characterized in that When the intra-sentence teaching feature is a skill feature, performing various types of intra-sentence teaching feature analyses on the lyrics timestamp and the audio representation information of the target song audio, and marking each feature mark of the analyzed various types of intra-sentence teaching features on the lyrics of the target song audio, including: Extracting the melody fundamental frequency of each word of the lyrics of the target song audio from the vocal audio based on the lyrics timestamp; The melody fundamental frequency of each word is calculated using fast Fourier transform to obtain the vibration frequency of each word; Mark the words on the lyrics of the target song audio with a vibration frequency within a preset vibration frequency range and a duration greater than a preset vibration duration as vibrato; Based on the lyrics timestamp and the vocal audio, audio detection is performed on each line of the lyrics of the target song audio, and the glissando of each line of the lyrics of the target song audio is obtained and marked.

8. The method according to claim 3, characterized in that The step of marking the intra-segment teaching features of the loudness gradient portion of each paragraph in the lyrics of the target song audio based on the time point corresponding to each paragraph and the audio representation information includes: Based on the time points corresponding to the respective paragraphs, respectively, loudness change analysis is performed on the audio of the sentences in each of the paragraphs in the original audio to obtain loudness change information of the sentences in each of the paragraphs; Based on the sentence loudness variation information in each of the paragraphs, the lyrics of the part with gradually increasing loudness and the lyrics of the part with decreasing loudness in each of the paragraphs are marked.

9. The method according to claim 2, characterized in that: The step of performing audio analysis on the target song audio to obtain audio representation information of the target song audio includes: The target song audio is input into an audio feature extraction model to extract audio representation information of the target song audio; wherein the audio representation information of the target song audio includes feature information of multiple dimensions of each frame in each audio channel.

10. The method according to claim 9, characterized in that The method of respectively performing various types of intra-sentence teaching feature analyses on the lyrics timestamp and the audio representation information of the target song audio, and marking each feature mark of the various types of intra-sentence teaching features analyzed on the lyrics of the target song audio, includes: Generate lyrics timestamp input information based on the lyrics time; wherein the lyrics timestamp input information includes the start time point and the end time point of each word in the lyrics of the target song audio; The audio representation information of the target song audio and the lyrics timestamp input information are input into a pre-trained intra-sentence recognition model, and feature processing is performed through each feature processing layer in the intra-sentence model to obtain processing features, and the scores corresponding to each type of intra-sentence teaching features in the output layer of the intra-sentence model are used to output the processing features to obtain recognition results of each type of intra-sentence teaching features; According to the recognition results of each type of the sentence teaching features, each feature mark of the each type of the sentence teaching features is marked on the lyrics of the target song audio.

11. The method according to claim 9, characterized in that The step of marking the intra-segment teaching features of the loudness gradient portion of each paragraph in the lyrics of the target song audio based on the time point corresponding to each paragraph and the audio representation information includes: The time point corresponding to the paragraph and the audio representation information are input into a pre-trained intra-segment recognition model to identify the intra-segment teaching features in each of the paragraphs, and the identified intra-segment teaching features of each of the paragraphs are marked in the lyrics of the target song audio.

12. The method according to claim 1, characterized in that Before outputting the marked target song audio lyrics as a song teaching template for the target song audio, the method further includes: The various marks that are in the same position and belong to the same type combination are merged into corresponding merged marks, and the various marks that are in the same position but do not belong to the same merged type group are displayed according to the priority of the category they belong to; wherein each of the type combinations includes at least two mark types that are required to be merged.

13. A song teaching template generation system, characterized in that: include: An information acquisition unit, used to acquire song audio information of the target song audio; wherein the song audio information includes lyrics, lyrics timestamps, time points corresponding to each paragraph, and audio representation information; The intra-sentence skill marking unit is used to respectively analyze various intra-sentence teaching features on the lyrics timestamp and the audio representation information of the target song audio, and mark each feature mark of the analyzed various intra-sentence teaching features on the lyrics of the target song audio; wherein the intra-sentence teaching features include at least one of stress, singing difficulty features, skill features, and breathing points; the singing difficulty features include at least one of high notes and long notes; the skill features include at least one of vibrato and glissando; An intra-segment skill marking unit, for marking intra-segment teaching features on the loudness gradient part of each of the paragraphs in the lyrics of the target song audio based on the time points corresponding to each of the paragraphs and the audio representation information; The template output unit is used to generate and output a song teaching template for the target song audio based on the marked lyrics of the target song audio.

14. A computer device, characterized in that: including memory and processor; The memory is used to store computer programs; The processor is used to execute the computer program to implement the song teaching template generation method as described in any one of claims 1 to 12.

15. A computer program product, characterized in that includes executable computer instructions; When the computer instructions are executed by the processor, they are used to implement the song teaching template generation method as described in any one of claims 1 to 12.

Images