Note cutting method, cutting system and sight-singing evaluation method

Abstract

The present invention provides a note cutting method, comprising the following steps: dividing an audio file into a plurality of audio segments; performing silent frame detection on the plurality of audio segments, marking each silent frame in the plurality of audio segments, and defining each audio segment except the silent frame as a non-silent frame audio segment; performing pitch detection and pitch conversion on each non-silent frame audio segment, and obtaining a converted pitch sequence; performing note starting point detection on each non-silent frame audio segment, and obtaining a note starting point sequence composed of the note starting points in all non-silent frame audio segments; dividing the converted pitch sequence according to the note starting points in the note starting point sequence and obtaining a plurality of pitch groups; taking the pitch with the largest number in each pitch group as the note of the pitch group, and forming a sight-singing note sequence with the notes of the plurality of pitch groups.

Description

Technical Field

[0001] The present invention relates to the field of audio data processing, and in particular to a note cutting method, a cutting system and a sight-singing evaluation method. Background Art

[0002] Music scoring is a technology that provides real-time feedback on the user's personal singing. It can evaluate the accuracy of the singer's singing, bringing a novel experience to the user while also improving the singing level. This method is common in entertainment venues such as KTV, which can greatly enhance the interactivity between the singer and the song he sings. Therefore, it is deeply loved by young people today.

[0003] For example, the invention patent application with application number 202110610610.3 mentions a singing scoring system, including a data acquisition module, a controller, a display and playback module, a data transmission module, a user terminal and a cloud server. The data acquisition module is used to collect user data information and data information during the user's singing process. The controller is used to process the data information collected by the data acquisition module to obtain the singing evaluation results. The display and playback module is used to display the singing evaluation results and the data information collected by the data acquisition module. The user terminal and the cloud server communicate with the controller through the data transmission module. The present invention can realize a comprehensive evaluation of the singing audio and singing emotions to obtain the singing evaluation results. At the same time, users can interact with other users through mobile terminals.

[0004] The above system can realize the interaction between scoring and singers, but since it does not improve the specific evaluation method much, it is impossible to give corresponding evaluation scores more objectively.

[0005] Therefore, in order to better and more finely cut the notes and to provide a more objective and fair evaluation system, the present application proposes a note cutting method and a sight-singing evaluation method based on the method. Summary of the invention

[0006] In view of the problems existing in the background technology, the purpose of the present invention is to provide a note cutting method and a sight-singing evaluation method based on the method, which can complete refined note cutting of audio. At the same time, since the audio is aligned through audio adjustment after cutting, it can be compatible with the advance and lag of the overall rhythm. Moreover, due to the refined cutting of the audio, when performing audio-visual evaluation, the influence of adjacent sounds is minimized, so high-precision recognition scoring effects can be maintained under various timbres.

[0007] In order to achieve the above-mentioned object, the present invention provides a note cutting method and a sight-singing evaluation method based on the method.

[0008] According to one aspect of the present application, the present application provides a note cutting method, comprising the following steps:

[0009] Split the audio file into multiple audio segments;

[0010] Performing silent frame detection on the multiple audio frames, marking each silent frame in the multiple audio frames, and defining each audio frame except the silent frame as a non-silent frame audio segment;

[0011] Performing pitch detection and pitch conversion on each non-silent frame audio segment, and obtaining a converted pitch sequence;

[0012] Perform note starting point detection on each non-silent frame audio segment, and obtain a note starting point sequence consisting of note starting points in all non-silent frame audio segments;

[0013] Segmenting the converted pitch sequence according to the note starting points in the note starting point sequence to obtain a plurality of pitch groups;

[0014] The pitch with the largest number in each pitch group is used as the note of the pitch group, and the notes of the multiple pitch groups form a sight-reading note sequence.

[0015] Preferably, dividing the audio file into multiple audio segments comprises the steps of:

[0016] Setting audio parameters of each frame of the audio clip, wherein the audio parameters include channel type and sampling frequency;

[0017] The audio file is divided into multiple audio frame segments according to the audio parameters.

[0018] Preferably, performing silent frame detection on the multiple audio frames, marking each silent frame in the multiple audio frames, and defining each audio frame except the silent frame as a non-silent frame audio segment, comprises the steps of:

[0019] Set the energy threshold of silent frames;

[0020] Calculate the energy value of each frame of audio clip;

[0021] Compare the energy value of each frame of the audio clip with the energy threshold, and mark the audio clip with an energy value less than or equal to the energy threshold as a silent frame;

[0022] The audio segments of each frame except the silent frame are defined as the non-silent frame audio segments.

[0023] Preferably, the energy threshold of the silence frame is set to 0-0.1.

[0024] Preferably, performing pitch detection and pitch conversion on each non-silent frame audio segment and obtaining a converted pitch sequence comprises the steps of:

[0025] A constant Q transform is used to perform frequency analysis on each non-silent frame audio segment, and a frequency spectrum feature of each non-silent frame audio segment is obtained;

[0026] A pitch extraction algorithm is used to extract the fundamental frequency of each non-silent frame audio segment, and the fundamental frequency of each non-silent frame audio segment is used as the pitch of the non-silent frame audio segment, thereby obtaining an initial pitch sequence composed of all pitches of all non-silent frame audio segments;

[0027] Preprocessing the initial pitch sequence to obtain a preprocessed pitch sequence;

[0028] The pitches in the preprocessed pitch sequence are converted with the standard pitches of the piano, thereby obtaining a converted pitch sequence.

[0029] Preferably, the initial pitch sequence is preprocessed in the following manner:

[0030] When the pitches of the preceding and following frames are both zero, the preceding and following frames are merged into one frame with the pitch being zero;

[0031] When the pitch deviation value of the two frames before and after exceeds a preset threshold, the pitches of the two frames before and after are set to the average pitch value of the two frames before and after.

[0032] Preferably, the conversion formula for converting the pitches in the preprocessed pitch sequence to the standard notes of the piano is: Note=69+12*log(f / 440) / log(2), wherein f is the frequency of each pitch in the preprocessed pitch sequence, and Note is the piano pitch.

[0033] Preferably, performing note starting point detection on each non-silent frame audio segment and obtaining a note starting point sequence consisting of note starting points in all non-silent frame audio segments comprises the steps of:

[0034] Calculate the energy value and phase value of each non-silent frame audio segment;

[0035] Find all candidate starting points according to the energy value and phase value of each non-silent frame audio segment;

[0036] Number all candidate starting points and calculate the probability of each candidate starting point being the starting point of a note;

[0037] The probability value of each candidate starting point is compared with the set probability threshold. When the probability value of a candidate starting point is greater than the probability threshold, the candidate starting point is determined as a note starting point, thereby obtaining a note starting point sequence consisting of all note starting points.

[0038] Preferably,

[0039] Performing note starting point detection on each non-silent frame audio segment and obtaining a note starting point sequence consisting of the note starting points in all non-silent frame audio segments, further comprising the steps of: post-processing all the note starting points in the note starting point sequence;

[0040] The post-processing method is: merging the starting points of consecutive notes with a time interval less than 0.1 seconds and the starting points of notes with zero frequencies before and after.

[0041] According to another aspect of the present application, the present application provides a note cutting system for implementing any one of the note cutting methods described above, the note cutting system comprising:

[0042] The first note cutting module is used to cut the audio file into multiple audio frames;

[0043] A silent frame detection module, used for performing silent frame detection on the multiple audio frames;

[0044] A pitch detection module, used to perform pitch detection and pitch conversion on each frame of audio clips except for silent frames, and obtain a converted pitch sequence;

[0045] A note starting point detection module is used to detect the note starting point of each non-silent frame audio segment and obtain a note starting point sequence composed of the note starting points in all non-silent frame audio segments;

[0046] A note second cutting module, used for dividing the converted pitch sequence according to the note starting points in the note starting point sequence to obtain a plurality of pitch groups;

[0047] The pitch recognition module is used to identify the largest number of pitches in each pitch group and obtain the sight-singing note sequence.

[0048] According to another aspect of the present application, the present application also provides a sight-singing evaluation method, comprising the following steps:

[0049] Cut the sight-singing audio according to any of the note cutting methods described above to obtain a sight-singing note sequence;

[0050] Aligning the sight-reading note sequence with the musical score note string;

[0051] According to the result of aligning the sight-singing note sequence with the musical score note string, the sight-singing accuracy of the sight-singing audio is calculated and a sight-singing evaluation is given.

[0052] Preferably,

[0053] Aligning the sight-singing note sequence with the musical score note string comprises the steps of:

[0054] The notes in the sight-singing note sequence are numbered and A=(a1, a2, ..., a m ), where m is the number of notes in the sight-singing note sequence;

[0055] The notes in the musical score note string are numbered and B=(b1, b2, ..., b n ), where n is the number of notes in the musical score note string;

[0056] Find the longest matching string C between the sight-singing note sequence and the musical score note string = (c1, c2, ..., c k ), where k is the number of notes of the sight-singing note sequence aligned with the note string of the musical score;

[0057] Find the number of missed notes in the sight-singing note sequence and record it as u;

[0058] Find the number of extra notes in the sight-singing note sequence and record it as v;

[0059] Find out the number of wrong notes in the sight-singing note sequence and record it as mvk=nuk;

[0060] The calculation formula of the sight-singing accuracy of the sight-singing audio is:

[0061] Sight-singing accuracy = k / (m+u)*100%; or

[0062] Sight-singing accuracy = k / (n+v)*100%.

[0063] The beneficial effects of the present invention are as follows:

[0064] (1) A note cutting method is provided, which can quickly complete the fine cutting of notes and perfectly preserve the acoustic characteristics of the original audio, which is conducive to the reprocessing of notes;

[0065] (2) By aligning the note strings, the singer's slight advances and delays during the singing process can be accurately adjusted, making the final evaluation more accurate.

[0066] (3) The sight-singing evaluation method is based on the refined segmentation of notes. The segmented note strings can avoid interference from adjacent note strings during evaluation. Therefore, the evaluation method can achieve high-precision scoring effects under various timbres. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] Figure 1 is a flow chart of the note cutting method of the present invention;

[0068] Figure 2 It is a flow chart of the sight singing evaluation method in the present invention. DETAILED DESCRIPTION

[0069] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0070] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by technicians in the technical field to which this application belongs; the terms used in the specification of the application are only for the purpose of describing specific embodiments and are not intended to limit this application; the terms "including" and "having" in the specification and claims of this application and the above-mentioned figure descriptions and any variations thereof are intended to cover non-exclusive inclusions.

[0071] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0072] The note cutting method, cutting system and sight-singing evaluation method based on the method according to the present application are described in detail below with reference to the accompanying drawings.

[0073] like Figure 1 A note cutting method is shown, comprising the following steps:

[0074] Step 1-1, dividing the audio file into multiple audio frames;

[0075] Step 1-2, performing silent frame detection on the multiple audio frames, marking each silent frame in the multiple audio frames, and defining each audio frame except the silent frame as a non-silent frame audio segment;

[0076] Step 1-3, performing pitch detection and pitch conversion on each non-silent frame audio segment, and obtaining a converted pitch sequence;

[0077] Step 1-4, performing note starting point detection on each non-silent frame audio segment, and obtaining a note starting point sequence consisting of the note starting points in all non-silent frame audio segments;

[0078] Step 1-5, dividing the converted pitch sequence according to the note starting points in the note starting point sequence to obtain a plurality of pitch groups;

[0079] Step 1-6, taking the pitch with the largest number in each pitch group as the note of the pitch group, and forming a sight-singing note sequence with the notes of the multiple pitch groups.

[0080] By cutting the audio files into frames with high sampling accuracy and small particles, the existing method of comparing the similarity after cutting according to the duration can make the comparison more precise in the subsequent comparison and scoring process; through high-pitched conversion and the use of the converted pitch, the accuracy of the evaluation is avoided from being affected by the sound source type (such as local accent, etc.), and the pitch division according to the starting point of the note is adopted, which makes the subsequent evaluation process more accurate and precise, and can make detailed divisions in a paragraph of lyrics, which is more precise than the comparison of the whole paragraph.

[0081] In step 1-1, dividing the audio file into multiple audio segments includes the following steps:

[0082] The audio parameters of each frame of the audio clip are set, wherein the audio parameters include a channel type and a sampling frequency; specifically, the channel type can be selected as mono or dual channel, etc., and can be determined according to the audio file type. When extracting, the extracted channel is mono, and the sampling frequency is optional and can be selected as needed, preferably 44100, which has good adaptability to audio files at this frequency.

[0083] The audio file is divided into multiple audio frame segments according to the audio parameters.

[0084] For example, you can use the following method to operate, input the audio file of sight singing, and use ffmpeg to set the audio parameters. The specific setting is that the audio parameters use mono, frequency 44.1K Hz, and 44100 samples per second. The specific conversion command is: ffmpeg-isrc.mp3-ac1-ar44100-ydest.mp3, where -ac1 represents mono, and -ar44100 represents the sampling rate of 44100 (select the sampling rate according to your needs, preferably higher than 44100, at this time, the accuracy is good and the adaptability is good.

[0085] The specific information after conversion is that the frame length of each frame is 512, 1024, 2048 or 4096 points, preferably 2048 points, the frame shift is 256, 512, 1024 sampling points, preferably 512 sampling points, about 86 frames per second.

[0086] Under the conditions of frame length 2048 and frame shift 512, the framing process has good adaptability to audio files. If the settings are too high, the output will be too slow and inefficient. If the settings are too low, the adaptability to low frequencies will be poor.

[0087] In step 1-2, performing silent frame detection on the multiple audio frames, marking each silent frame in the multiple audio frames, and defining each audio frame except the silent frame as a non-silent frame audio segment, comprises the steps of:

[0088] Set the energy threshold of silent frames; the threshold refers to intensity or loudness;

[0089] Calculate the energy value of each frame of audio clip;

[0090] Compare the energy value of each frame of the audio clip with the energy threshold, and mark the audio clip with an energy value less than or equal to the energy threshold as a silent frame;

[0091] The audio segments of each frame except the silent frame are defined as the non-silent frame audio segments.

[0092] Specifically, silent frame detection can use voice activity detection (VAD), which is also called voice endpoint detection or voice boundary detection. The silent frame is detected first, and the subsequent processing of the silent frame is skipped. VAD mainly has several feature extraction methods based on energy, frequency domain, cepstrum, harmonic, and long-term information. Energy extraction is used here, and the result of energy extraction is better and more accurate.

[0093] For specific methods, please refer to:

[0094] First, assume that there is no one singing at the beginning, and calculate the average energy of the initial sound of 8 frames as the baseline, which is equivalent to the background noise. If the root mean square of a frame is greater than this baseline, it is considered that someone is singing, otherwise it is considered that no one is singing. There will be no human singing at the beginning, end, and rest of the audio. Such frames are called silent frames. The energy value of each point in the frame is compared with a pre-set threshold. If it is less than the threshold, it is considered a silent frame, and the frequency is recorded as 0 Hz.

[0095] Preferably, the energy threshold of the silent frame is set to 0 to 0.1. The above range has the best adaptability. If it is too large, it will affect the accuracy of silent judgment and may mistake human voice for silent, resulting in result deviation.

[0096] In step 1-3, pitch detection and pitch conversion are performed on each non-silent frame audio segment, and a converted pitch sequence is obtained, including the steps of:

[0097] A constant Q transform is used to perform frequency analysis on each non-silent frame audio segment, and a frequency spectrum feature of each non-silent frame audio segment is obtained;

[0098] Constant Q transform (CQT): refers to a filter group whose center frequency is distributed according to an exponential law, and whose filter bandwidths are different, but whose center frequency to bandwidth ratio is a constant Q. The horizontal axis frequency of its spectrum is not linear, but is based on log2, and the filter window length can be changed according to the different spectral line frequencies, so the performance is better. Since the distribution of CQT is the same as that of the scale frequency, the amplitude value of the music signal at each note frequency can be directly obtained by calculating the CQT spectrum of the music signal, which is very suitable for music audio data signal processing. In the constant Q transform of the present invention, a recursive subsampling method is used for transformation, wherein the sampling frequency fS is selected from 512, 1024, 2048 and 4096, or other suitable frequencies can be selected as needed; preferably, 2048 is used. The advantage of using the above frequencies is good applicability.

[0099] A pitch extraction algorithm is used to extract the fundamental frequency of each non-silent frame audio segment, and the fundamental frequency of each non-silent frame audio segment is used as the pitch of the non-silent frame audio segment, thereby obtaining an initial pitch sequence composed of all the pitches of all non-silent frame audio segments.

[0100] CEP (Cepstrum Pitch Detection Method): The audio signal is placed in the frequency domain, and the harmonic information is obtained by calculating the fast Fourier transform, thereby extracting the fundamental frequency. Specifically, the frequency spectrum of the audio signal is logarithmized, and then the cepstrum is obtained by inverse discrete Fourier transform, and then the basic frequency position is determined by extracting the cepstrum peak, thereby obtaining the pitch frequency value f. The sampling frequency specifically used in the present invention is 512, 1024, 2048 or 4096, and other commonly used frequencies can also be selected, preferably 2048, and the advantage of using the above frequencies is good applicability.

[0101] The CQT and CEP methods are used together to output results with high accuracy; and the amplitude value of the audio signal at each note frequency can be directly obtained, which is convenient for audio processing.

[0102] Preprocessing the initial pitch sequence to obtain a preprocessed pitch sequence;

[0103] The pitches in the preprocessed pitch sequence are converted with standard pitches (which may be standard piano pitches, etc.), thereby obtaining a converted pitch sequence.

[0104] For example, the pitch of each sound frame is detected using a pitch extraction algorithm and converted into 88 standard notes of a piano. Sound is composed of a series of vibrations with different frequencies and amplitudes emitted by the sound-emitting body. Among these vibrations, there is a vibration with the lowest frequency. The sound emitted by it is the fundamental tone, and the others are overtones. The fundamental tone determines the pitch. The frequency spectrum characteristics are obtained by frequency analysis using the Constant Q transform (CQT). The CQT transform has a higher resolution for low frequencies; the pitch extraction algorithm CEP is used to select the fundamental frequency as the pitch of each frame.

[0105] The initial pitch sequence is preprocessed in step 1-4 as follows:

[0106] When the pitches of the preceding and following frames are both zero, the preceding and following frames are merged into one frame with the pitch being zero;

[0107] When the pitch deviation value of the two frames before and after exceeds the preset threshold, 50% can be selected. Such processing can avoid noise interference. When the result is more accurate, the pitch of the two frames before and after are set to the average pitch of the two frames before and after. For example, in adjacent 1 / 2 / 3 frames, if the pitch of the second frame is significantly higher than 1 and 3, 2 is regarded as a very high pitch, the pitch of the second frame is deleted, and then the average of the pitches of 1 and 3 is calculated as the pitch of the second frame. Such processing can effectively eliminate noise interference.

[0108] Specifically, the conversion formula for converting the pitches in the preprocessed pitch sequence to the standard pitch is: Note=69+12*log(f / 440) / log(2), wherein f is the frequency of each pitch in the preprocessed pitch sequence, and Note is the standard pitch, for example: MIDI pitch.

[0109] In step 1-5, the note starting point detection is performed on each non-silent frame audio segment, and a note starting point sequence composed of the note starting points in all non-silent frame audio segments is obtained, including the steps of:

[0110] Calculate the energy value and phase value of each non-silent frame audio segment;

[0111] Find all candidate starting points according to the energy value and phase value of each non-silent frame audio segment;

[0112] Number all candidate starting points and calculate the probability of each candidate starting point being the starting point of a note;

[0113] The probability value of each candidate starting point is compared with the set probability threshold. When the probability value of a candidate starting point is greater than or equal to the probability threshold, the candidate starting point is determined as a note starting point, thereby obtaining a note starting point sequence consisting of all note starting points. The specific threshold is selected to be 0.75-0.85, and the recognition accuracy is higher within this range.

[0114] Note starting point detection has always been a difficult problem in music information retrieval. The present invention adopts the QMUL algorithm, a signal-based algorithm that combines the complex domain composed of energy (observing the energy of the signal) and phase (observing the FFT state deviation). It includes an adaptive whitening component that smoothes the time and frequency changes of the signal, thereby "introducing the amplitude of each frequency band into a similar dynamic range, making the peak with larger amplitude more obvious. The algorithm follows the peak in the complex domain to calculate the possibility of sudden events in each frequency domain, and uses a peak selection algorithm to mark onsets. The algorithm for detecting the note starting point obtains an onset sequence, and an onset of 1 indicates a candidate starting point.

[0115] Onset sequence example: (serial number timestamp probability threshold) such as (1118956320.83), (2118956990.60),…

[0116] Preferably, performing note starting point detection on each non-silent frame audio segment and obtaining a note starting point sequence composed of the note starting points in all non-silent frame audio segments further includes the steps of: post-processing all the note starting points in the note starting point sequence;

[0117] The post-processing method is: merging the starting points of consecutive notes with a time interval less than 0.1 seconds and the starting points of notes with zero frequencies before and after.

[0118] The above processing method can eliminate interference such as non-human background sound and improve accuracy.

[0119] The present invention provides a note cutting system for implementing the above-mentioned note cutting method, and the note cutting system comprises:

[0120] The first note cutting module is used to cut the audio file into multiple audio frames;

[0121] A silent frame detection module, used for performing silent frame detection on the multiple audio frames;

[0122] A pitch detection module, used to perform pitch detection and pitch conversion on each frame of audio clips except for silent frames, and obtain a converted pitch sequence;

[0123] A note starting point detection module is used to detect the note starting point of each non-silent frame audio segment and obtain a note starting point sequence composed of the note starting points in all non-silent frame audio segments;

[0124] A note second cutting module, used for dividing the converted pitch sequence according to the note starting points in the note starting point sequence to obtain a plurality of pitch groups;

[0125] The pitch recognition module is used to identify the largest number of pitches in each pitch group and obtain the sight-singing note sequence.

[0126] like Figure 2 As shown, the present invention provides a sight-singing evaluation method, comprising the following steps:

[0127] Step 2-1, cutting the sight-singing audio according to the note cutting method described above, and obtaining a sight-singing note sequence;

[0128] Step 2-2, aligning the sight-singing note sequence with the musical score note string;

[0129] Step 2-3, based on the result of aligning the sight-singing note sequence with the musical score note string, calculate the sight-singing accuracy of the sight-singing audio and provide a sight-singing evaluation.

[0130] In step 2-2, the sight-singing note sequence is aligned with the score note string, including the steps of:

[0131] The notes in the sight-singing note sequence are numbered and A=(a1, a2, ..., a m ), where m is the number of notes in the sight-singing note sequence;

[0132] The notes in the musical score note string are numbered and B=(b1, b2, ..., b n ), where n is the number of notes in the musical score note string;

[0133] Find the longest matching string C between the sight-singing note sequence and the musical score note string = (c1, c2, ..., c k ), wherein k is the number of notes of the sight-singing note sequence aligned with the note string of the musical score, wherein the alignment also includes aligning the pitches.

[0134] The specific steps of steps 2-3 are:

[0135] Find the number of missed notes in the sight-singing note sequence and record it as u;

[0136] Find the number of extra notes in the sight-singing note sequence and record it as v;

[0137] Find out the number of wrong notes in the sight-singing note sequence and record it as mvk=nuk;

[0138] The calculation formula of the sight-singing accuracy of the sight-singing audio is:

[0139] Sight-singing accuracy = k / (m+u)*100%; or

[0140] Sight-singing accuracy = k / (n+v)*100%.

[0141] The following is a detailed description of the alignment and evaluation methods:

[0142] The input of note string alignment is the note sequence A = (a1, a2, ..., am) and the musical note string B = (b1, b2, ..., bn) after note segmentation. The Needleman-Wunsch (NW) algorithm, one of the dynamic programming algorithms, is used to find the longest matching string C = (c1, c2, ..., ck). Each note in the C sequence appears in both sequences A and B in the same order, but may not be continuous. After the two note strings A and B are aligned, each note will appear in four situations:

[0143] a) There are k notes aligned;

[0144] b) There is a gap in A, that is, a missing note, and a '-' is added to the corresponding position in the A sequence. Assume that u notes are missed (that is, u '-'s are added to the A sequence);

[0145] c) There is a gap in B, that is, there is an extra note, and a '-' is added to the corresponding position of the B sequence. Assume that v extra notes are sung (that is, v '-' are added to the B sequence);

[0146] d) The other notes are mismatched, i.e., sung incorrectly. The number of mismatches is mvk=nuk.

[0147] It is not difficult to see that: m+u=n+v, which means that after making up for the number of missed notes and the number of extra notes, the two strings of notes become the same length;

[0148] Number of wrong notes = number of missed notes (u notes) + number of extra notes (v notes) + number of mismatches (mvk) = m + uk; for example, insufficient pitch.

[0149] The calculation formula of sight-singing accuracy is as follows:

[0150] Sight-singing accuracy: k / (m+u)*100%.

Claims

1. A note cutting method, characterized in that: The following steps are involved: Split the audio file into multiple audio segments; Performing silent frame detection on the multiple audio frames, marking each silent frame in the multiple audio frames, and defining each audio frame except the silent frame as a non-silent frame audio segment; Performing pitch detection and pitch conversion on each non-silent frame audio segment, and obtaining a converted pitch sequence; Perform note starting point detection on each non-silent frame audio segment, and obtain a note starting point sequence consisting of note starting points in all non-silent frame audio segments; Segmenting the converted pitch sequence according to the note starting points in the note starting point sequence to obtain a plurality of pitch groups; The pitch with the largest number in each pitch group is used as the note of the pitch group, and the notes of the multiple pitch groups form a sight-singing note sequence.

2. The note cutting method according to claim 1, characterized in that: Splitting an audio file into multiple audio segments includes the following steps: Setting audio parameters of each frame of the audio clip, wherein the audio parameters include channel type and sampling frequency; The audio file is divided into multiple audio frame segments according to the audio parameters.

3. The note cutting method according to claim 1, characterized in that: Performing silent frame detection on the multiple audio frames, marking each silent frame in the multiple audio frames, and defining each audio frame except the silent frame as a non-silent frame audio segment, comprising the steps of: Set the energy threshold of silent frames; Calculate the energy value of each frame of audio clip; Compare the energy value of each frame of the audio clip with the energy threshold, and mark the audio clip with an energy value less than or equal to the energy threshold as a silent frame; The audio segments of each frame except the silent frame are defined as the non-silent frame audio segments.

4. The note cutting method according to claim 1, characterized in that: Performing pitch detection and pitch conversion on each non-silent frame audio segment and obtaining a converted pitch sequence includes the following steps: A constant Q transform is used to perform frequency analysis on each non-silent frame audio segment, and a frequency spectrum feature of each non-silent frame audio segment is obtained; A pitch extraction algorithm is used to extract the fundamental frequency of each non-silent frame audio segment, and the fundamental frequency of each non-silent frame audio segment is used as the pitch of the non-silent frame audio segment, thereby obtaining an initial pitch sequence composed of all pitches of all non-silent frame audio segments; Preprocessing the initial pitch sequence to obtain a preprocessed pitch sequence; The pitches in the preprocessed pitch sequence are converted with the standard pitches to obtain a converted pitch sequence.

5. The note cutting method according to claim 4, characterized in that: The initial pitch sequence is preprocessed as follows: When the pitches of the preceding and following frames are both zero, the preceding and following frames are merged into one frame with the pitch being zero; When the pitch deviation value of the two frames before and after exceeds a preset threshold, the pitches of the two frames before and after are set to the average pitch value of the two frames before and after.

6. The note cutting method according to claim 4, characterized in that: The conversion formula for converting the pitches in the preprocessed pitch sequence to the standard pitch is: Note=69+12*log(f / 440) / log(2), where f is the frequency of each pitch in the preprocessed pitch sequence, and Note is the standard pitch.

7. The note cutting method according to claim 1, characterized in that: Performing note starting point detection on each non-silent frame audio segment and obtaining a note starting point sequence consisting of note starting points in all non-silent frame audio segments includes the following steps: Calculate the energy value and phase value of each non-silent frame audio segment; Find all candidate starting points according to the energy value and phase value of each non-silent frame audio segment; Number all candidate starting points and calculate the probability of each candidate starting point being the starting point of a note; The probability value of each candidate starting point is compared with the set probability threshold. When the probability value of a candidate starting point is greater than the probability threshold, the candidate starting point is determined as a note starting point, thereby obtaining a note starting point sequence consisting of all note starting points.

8. The note cutting method according to claim 7, characterized in that: Performing note starting point detection on each non-silent frame audio segment and obtaining a note starting point sequence consisting of the note starting points in all non-silent frame audio segments, further comprising the steps of: post-processing all the note starting points in the note starting point sequence; The post-processing method is: merging the starting points of consecutive notes with a time interval less than 0.1 seconds and the starting points of notes with zero frequencies before and after.

9. A note cutting system, characterized in that: Used to implement the note cutting method according to any one of claims 1 to 8, the note cutting system comprises: The first note cutting module is used to cut the audio file into multiple audio frames; A silent frame detection module, used for performing silent frame detection on the multiple audio frames; A pitch detection module, used to perform pitch detection and pitch conversion on each frame of audio clips except for silent frames, and obtain a converted pitch sequence; A note starting point detection module is used to detect the note starting point of each non-silent frame audio segment and obtain a note starting point sequence composed of the note starting points in all non-silent frame audio segments; A note second cutting module, used for dividing the converted pitch sequence according to the note starting points in the note starting point sequence to obtain a plurality of pitch groups; The pitch recognition module is used to identify the largest number of pitches in each pitch group and obtain the sight-singing note sequence.

10. A sight-singing evaluation method, characterized in that: The following steps are involved: Cutting the sight-singing audio according to the note cutting method described in any one of claims 1 to 8 to obtain a sight-singing note sequence; Aligning the sight-reading note sequence with the musical score note string; According to the result of aligning the sight-singing note sequence with the musical score note string, the sight-singing accuracy of the sight-singing audio is calculated and a sight-singing evaluation is given.

Images