A tempo estimation method, device, computer equipment and storage medium

Abstract

The present invention discloses a method, device, computer equipment and storage medium for estimating the tempo of the current music. The method comprises: extracting audio features from the current music; performing autocorrelation processing on the audio features; listing multiple possible options of the number of beats per minute for the current music; generating a feature beat array based on each possible option of the number of beats per minute; performing cross-correlation processing on the audio features after the autocorrelation processing and each of the feature beat arrays; based on the cross-correlation processing result, selecting a cross-correlation function whose dynamic range meets a preset threshold as the tempo estimation result of the current music. The present invention combines audio features and the number of beats per minute, and processes them using the autocorrelation and cross-correlation methods, thereby improving the speed estimation accuracy of the music tempo.

Description

Technical Field

[0001] The present invention relates to the field of computer software technology, and in particular to a beat speed estimation method, device, computer equipment and storage medium. Background Art

[0002] Tempo is an important element of music. The most common unit is beats per minute (BPM), which indicates the number of times a specified note appears in one minute. The larger the BPM value, the faster the song. The main function of the music tempo estimation algorithm is to estimate the tempo of a piece of music. After obtaining the music tempo, it can be used for beat tracking or other applications such as estimating music structure. It is an important starting point in the field of music research.

[0003] The most mainstream approach at present is usually to first reduce the dimensionality of the music, and then extract the onset strength curve, energy envelope, or spectral flux, etc., which represent the intensity of the music, and then judge the periodicity of the reduced-dimensional curve. In theory, obtaining periodicity is like obtaining the tempo feature. Summary of the invention

[0004] The embodiment of the present invention provides a beat speed estimation method, aiming to improve the speed estimation accuracy of music beat.

[0005] In a first aspect, an embodiment of the present invention provides a method for estimating a tempo, comprising:

[0006] Extract audio features from the current music;

[0007] Performing autocorrelation processing on the audio features;

[0008] List multiple possible options for the number of beats per minute for the current music;

[0009] Generate a feature beat array based on each possible option of the number of beats per minute;

[0010] Performing cross-correlation processing on the audio features after the autocorrelation processing and each of the feature beat arrays;

[0011] Based on the cross-correlation processing result, a cross-correlation function whose dynamic range meets a preset threshold is selected as the beat speed estimation result of the current music.

[0012] In a second aspect, an embodiment of the present invention provides a beat speed estimation device, comprising:

[0013] A feature extraction unit, used to extract audio features from the current music;

[0014] An autocorrelation processing unit, used for performing autocorrelation processing on the audio features;

[0015] An option listing unit, used to list multiple possible options of the number of beats per minute for the current music;

[0016] An array generating unit, for generating a characteristic beat array based on each possible option of the number of beats per minute;

[0017] A cross-correlation processing unit, used for performing cross-correlation processing on the audio features after the autocorrelation processing and each of the feature beat arrays;

[0018] The range selection unit is used to select the cross-correlation function whose dynamic range meets the preset threshold as the beat speed estimation result of the current music based on the cross-correlation processing result.

[0019] In a third aspect, an embodiment of the present invention provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the beat tempo estimation method as described in the first aspect when executing the computer program.

[0020] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the beat speed estimation method as described in the first aspect is implemented.

[0021] The embodiments of the present invention provide a method, device, computer equipment and storage medium for estimating the tempo of the current music. The method includes: extracting audio features from the current music; performing autocorrelation processing on the audio features; listing multiple possible options of the number of beats per minute for the current music; generating a feature beat array based on each possible option of the number of beats per minute; performing cross-correlation processing on the audio features after the autocorrelation processing and each of the feature beat arrays; based on the cross-correlation processing result, selecting a cross-correlation function whose dynamic range meets a preset threshold as the tempo estimation result of the current music. The embodiments of the present invention improve the speed estimation accuracy of the music tempo by combining audio features and the number of beats per minute and processing them using autocorrelation and cross-correlation methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying any creative work.

[0023] Figure 1 A schematic diagram of a process flow of a beat speed estimation method provided by an embodiment of the present invention;

[0024] Figure 2 A schematic block diagram of a beat speed estimation device provided by an embodiment of the present invention;

[0025] Figure 3 A time domain waveform diagram of music in a beat speed estimation method provided by an embodiment of the present invention;

[0026] Figure 4 A novelty curve diagram of a beat speed estimation method provided by an embodiment of the present invention;

[0027] Figure 5 A schematic diagram of an autocorrelation function of a novelty curve in a beat speed estimation method provided by an embodiment of the present invention;

[0028] Figure 6 A schematic diagram of amplifying the peak value within a target period of a novelty curve in a beat speed estimation method provided by an embodiment of the present invention;

[0029] Figure 7 A schematic diagram of a characteristic beat array at 152 BPM in a beat speed estimation method provided by an embodiment of the present invention;

[0030] Figure 8 A schematic diagram of a characteristic beat array at 116 BPM in a beat speed estimation method provided by an embodiment of the present invention;

[0031] Fig. 9 A schematic diagram of a characteristic beat array at 92 BPM in a beat speed estimation method provided by an embodiment of the present invention;

[0032] Fig.10 A schematic diagram of a characteristic beat array at 77 BPM in a beat speed estimation method provided by an embodiment of the present invention;

[0033] Fig.11 A schematic diagram of the cross-correlation between a novelty curve and a characteristic beat array at 152 BPM in a beat speed estimation method provided by an embodiment of the present invention;

[0034] Fig.12A schematic diagram of the cross-correlation between a novelty curve and a characteristic beat array at 116 BPM in a beat speed estimation method provided by an embodiment of the present invention;

[0035] Fig.13 A schematic diagram of the cross-correlation between a novelty curve and a characteristic beat array at 92 BPM in a beat speed estimation method provided by an embodiment of the present invention;

[0036] Fig.14 A schematic diagram of the cross-correlation between a novelty curve and a characteristic beat array at 77 BPM in a beat speed estimation method provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0037] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0038] It should be understood that when used in this specification and the appended claims, the terms "include" and "comprises" indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0039] It should also be understood that the terms used in this specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the specification of the present invention and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0040] It should be further understood that the term "and / or" used in the present description and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0041] See below Figure 1 , Figure 1 A schematic flow chart of a beat speed estimation method provided by an embodiment of the present invention, the method specifically comprises: steps S101 to S106.

[0042] S101, extracting audio features from current music;

[0043] S102, performing autocorrelation processing on the audio features;

[0044] S103, listing multiple possible options of the number of beats per minute for the current music;

[0045] S104, generating a characteristic beat array based on each possible option of the number of beats per minute;

[0046] S105, performing cross-correlation processing on the audio features after the autocorrelation processing and each of the feature beat arrays;

[0047] S106. Based on the cross-correlation processing result, a cross-correlation function whose dynamic range meets a preset threshold is selected as a tempo estimation result of the current music.

[0048] In this embodiment, in order to estimate the tempo, audio features are first extracted from the currently playing music, and then the audio features are autocorrelated to determine their periodicity. At the same time, multiple possible options of the number of beats per minute of the currently playing music are listed, and a corresponding feature beat array is generated for each possible option. Then, the audio features and the feature beat array are cross-correlated to obtain the correlation between the two, and further, from different cross-correlation processing results, a cross-correlation function with a dynamic range that meets the requirements is selected as the final tempo prediction result. Here, it should be noted that the cross-correlation function is the cross-correlation processing result.

[0049] The tempo estimation method of this embodiment can effectively avoid the probability of finding wrong options when estimating the tempo of general music, and improve the speed estimation accuracy of the music tempo. In addition, the tempo estimation method provided by this embodiment can also provide support for other algorithms involving tempo estimation. Figure 3 As shown, before extracting the audio features, the current music will naturally be converted into a corresponding time domain waveform.

[0050] In one embodiment, the step S101 includes:

[0051] Constructing novelty curves for recurring or quasi-periodic patterns in current music;

[0052] A spectrogram of the novelty curve is calculated, and a frequency axis in the spectrogram is converted into a rhythm axis, thereby obtaining the audio feature.

[0053] In this embodiment, the audio features are obtained by extracting and constructing a novelty curve of the current music. The novelty curve refers to the amplitude by which the sound of the music becomes louder from the previous moment to the next moment. For example, 10 to 100 is an increase rate of 10, 100 to 200 is an increase rate of 2, 200 to 0 is an increase of 0 times (no negative numbers), and so on. Figure 4That is, the novelty curve is constructed. Of course, in other embodiments, onset strength curve (onset strength curve), energy envelope (energy envelope) or spectral flux (spectral change) etc. can also be set as the audio feature.

[0054] In one embodiment, the step S102 includes:

[0055] The autocorrelation of the audio features is calculated according to the following formula:

[0056]

[0057] Where R(k) is the audio feature, E is the expected value, and X i is the value of the random variable at time t(i), μ i is the expected value at time t(i), X i+k is the value of the random variable at t(i+k), μ i+k is the expected value at t(i+k), σ 2 is the variance.

[0058] In this embodiment, the extracted audio features are subjected to autocorrelation calculation to obtain the periodicity judgment of the audio features. Here, in this embodiment, the novelty curve is used as the audio feature, so the novelty curve is subjected to autocorrelation calculation to obtain the following: Figure 5 The autocorrelation function is shown.

[0059] Comparing an ordered sequence of random variables with itself is the definition of the autocorrelation function in statistics. Every sequence without phase difference is similar to itself, that is, in this case, the autocorrelation function value is the largest. If the components of the sequence are correlated with each other (no longer random), the value calculated by the following correlation value equation is no longer zero, and such a component is autocorrelated.

[0060] In one embodiment, the step S103 includes:

[0061] Determine a target period corresponding to a beat according to the search range of the music speed;

[0062] Obtain all peaks in the target period and the delay times and beats per minute corresponding to the peaks;

[0063] All peaks in the target cycle are taken as possible options for the BPM.

[0064] In this embodiment, in the process of listing possible BPM options, a search range for music speed can be first estimated or set, and then a beat cycle (i.e., the target cycle) can be determined based on the search range. Within this target cycle, at least one peak value can be obtained, and then the obtained peak value can be set as a possible option for beats per minute.

[0065] For example, combining Figure 5 and Figure 6 First, set the search range of music speed to 70BPM~170BPM, then the target period corresponding to one beat is 0.353~0.847(second), and then within the target period range of 0.353~0.847(second), a total of 4 peaks (peak1, peak2, peak3 and peak4) can be obtained, and the BPMs corresponding to the 4 peaks are all listed as options. The delays (delay) and corresponding BPMs of the 4 peaks are shown in Table 1:

[0066] Delay(S) Related BPM peak1 0.394736842 152 Peak2 0.517241379 116 Peak3 0.652173813 92 Peak4 0.779220779 77

[0067] Table 1

[0068] In one embodiment, the step S104 includes:

[0069] Determining a strong musical beat per second based on the number of beats per minute;

[0070] Construct an array of characteristic beats per minute based on strong musical beats per second.

[0071] In this embodiment, according to the number of beats per minute, a pulse within a period of 1 second (i.e., the strong music beat per second) is determined, and then the feature beat array (featurebeatarray) is constructed according to the strong music beat per second within each minute. For example, if the tempo of a piece of music is 60BPM, then the featurebeatarray corresponding to 60BPM is one pulse in a period of one second; if the tempo of a piece of music is 120BPM, then the featurebeatarray corresponding to 120BPM is two pulses in a period of one second. In a specific embodiment, Figures 7-10 As shown, the featurebeatarrays corresponding to the generated music of 152BPM, 116BPM, 92BPM and 77BPM were constructed respectively.

[0072] In one embodiment, the step S105 includes:

[0073] The cross-correlation calculation is performed on the audio features after self-phase processing and the feature beat array according to the following formula:

[0074] (R*g,x)=∫R * (t)g(x+t)dt

[0075] Where (R*g,x) is the calculated cross-correlation function, R is the audio feature, g is the feature beat array, and t is the period.

[0076] In this embodiment, the correlation between the audio features and the feature beat array is obtained by performing cross-correlation calculation on the audio features and the feature beat array. Figures 11 to 14 As shown, at different BPMs, different results, namely, cross-correlation functions, can be obtained by performing cross-correlation calculations on the novelty curve and the characteristic beat array.

[0077] In statistics, cross-correlation is sometimes used to refer to the covariance cov(X, Y) between two random vectors X and Y, to distinguish it from the concept of "covariance" of vector X, which is the covariance matrix between the scalar components of X. In signal processing, cross-correlation (sometimes also called "cross-covariance") is a measure used to represent the similarity between two signals, usually used to find features in an unknown signal by comparing it to a known signal. It is a function of the difference between the two signals relative to time, sometimes called the "sliding dot product", and has applications in pattern recognition and cryptanalysis.

[0078] In one embodiment, the step S106 includes:

[0079] Calculate the difference between the maximum and minimum values ​​in each cross-correlation function;

[0080] The cross-correlation function with the largest difference is selected as the tempo estimation result of the current music.

[0081] In this embodiment, the dynamic range refers to the ratio of the maximum value to the minimum value of a variable signal (such as sound or light). When selecting the dynamic range of the cross-correlation function, in order to ensure the estimation accuracy, the difference between the maximum value and the minimum value of the dynamic range is used as the selection criterion, that is, the cross-correlation function corresponding to the dynamic range with the largest difference is selected, and this is used as the estimation result of the beat speed of the current music.

[0082] Figure 2 A schematic block diagram of a beat speed estimation device 200 provided in an embodiment of the present invention, the device 200 includes:

[0083] A feature extraction unit 201 is used to extract audio features from the current music;

[0084] An autocorrelation processing unit 202, configured to perform autocorrelation processing on the audio features;

[0085] An option listing unit 203, used to list multiple possible options of the number of beats per minute for the current music;

[0086] An array generating unit 204, configured to generate a characteristic beat array based on each possible option of the number of beats per minute;

[0087] A cross-correlation processing unit 205, configured to perform cross-correlation processing on the audio features after the autocorrelation processing and each of the feature beat arrays;

[0088] The range selection unit 206 is used to select a cross-correlation function whose dynamic range meets a preset threshold as a tempo estimation result of the current music based on the cross-correlation processing result.

[0089] In one embodiment, the feature extraction unit 201 includes:

[0090] A curve construction unit for constructing a novelty curve about recurring or quasi-periodic patterns in current music;

[0091] The axis conversion unit is used to calculate the spectrogram of the novelty curve and convert the frequency axis in the spectrogram into a rhythm axis to obtain the audio feature.

[0092] In one embodiment, the autocorrelation processing unit 202 includes:

[0093] The autocorrelation calculation unit is used to perform autocorrelation calculation on the audio feature according to the following formula:

[0094]

[0095] Where R(k) is the audio feature, E is the expected value, and X i is the value of the random variable at time t(i), μ i is the expected value at time t(i), X i+k is the value of the random variable at t(i+k), μ i+k is the expected value at t(i+k), σ 2 is the variance.

[0096] In one embodiment, the option listing unit 203 includes:

[0097] A period determination unit, used to determine a target period corresponding to a beat according to a search range of music speed;

[0098] A peak value acquisition unit, used for acquiring all peak values ​​in the target period and the delay time and beats per minute corresponding to the peak values;

[0099] The peak value setting unit is used to take all peak values ​​in the target cycle as possible options for the beats per minute.

[0100] In one embodiment, the array generation unit 204 includes:

[0101] A beat determination unit, used for determining a strong music beat per second according to the beats per minute;

[0102] The array building unit is used to build a characteristic beat array per minute based on the strong music beat per second.

[0103] In one embodiment, the cross-correlation processing unit 205 includes:

[0104] The cross-correlation calculation unit is used to perform cross-correlation calculation on the self-phase processed audio features and the feature beat array according to the following formula:

[0105] (R*g,x)=∫R * (t)g(x+t)dt

[0106] Where (R*g,x) is the calculated cross-correlation function, R is the audio feature, g is the feature beat array, and t is the period.

[0107] In one embodiment, the range selection unit 206 includes:

[0108] A difference calculation unit, used for calculating the difference between the maximum value and the minimum value in each cross-correlation function;

[0109] The difference selection unit is used to select the cross-correlation function with the largest difference as the beat speed estimation result of the current music.

[0110] Since the embodiments of the apparatus part correspond to the embodiments of the method part, please refer to the description of the embodiments of the method part for the embodiments of the apparatus part, which will not be repeated here.

[0111] The embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed, the steps provided in the above embodiment can be implemented. The storage medium may include: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

[0112] The embodiment of the present invention also provides a computer device, which may include a memory and a processor, wherein a computer program is stored in the memory, and when the processor calls the computer program in the memory, the steps provided in the above embodiment may be implemented. Of course, the computer device may also include various network interfaces, power supplies and other components.

[0113] The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same and similar parts between the various embodiments can be referred to each other. For the system disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of this application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the claims of this application.

[0114] It should also be noted that, in this specification, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprises", "comprising" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the statement "comprising a ..." does not exclude the presence of other identical elements in the process, method, article or device including the element.

Claims

1. A beat speed estimation method, characterized in that: include: Extract audio features from the current music; Performing autocorrelation processing on the audio features; List multiple possible options for the number of beats per minute for the current music; Generate a feature beat array based on each possible option of the number of beats per minute; Performing cross-correlation processing on the audio features after the autocorrelation processing and each of the feature beat arrays; Based on the cross-correlation processing result, a cross-correlation function whose dynamic range meets a preset threshold is selected as a beat speed estimation result of the current music; The multiple possible options for listing the number of beats per minute for the current music include: Determine a target period corresponding to a beat according to the search range of the music speed; Obtain all peaks in the target period and the delay times and beats per minute corresponding to the peaks; All peaks in the target cycle are taken as possible options for the BPM.

2. The method for estimating the beat speed according to claim 1, characterized in that: The step of extracting audio features from the current music includes: Constructing novelty curves for recurring or quasi-periodic patterns in current music; A spectrogram of the novelty curve is calculated, and a frequency axis in the spectrogram is converted into a rhythm axis, thereby obtaining the audio feature.

3. The method for estimating the beat speed according to claim 1, characterized in that: The performing autocorrelation processing on the audio feature comprises: The autocorrelation of the audio features is calculated according to the following formula: Where R(k) is the audio feature, E is the expected value, and X i is the value of the random variable at time t(i), μ i is the expected value at time t(i), X i+k is the value of the random variable at t(i+k), μ i+k is the expected value at t(i+k), σ 2 is the variance.

4. The method for estimating the beat speed according to claim 1, characterized in that: The method of generating a characteristic beat array for each possible option based on the number of beats per minute comprises: Determining a strong musical beat per second based on the number of beats per minute; Construct an array of characteristic beats per minute based on strong musical beats per second.

5. The method for estimating the beat speed according to claim 1, characterized in that: The cross-correlation processing of the audio features after the autocorrelation processing and each of the feature beat arrays comprises: The cross-correlation calculation is performed on the audio features after self-phase processing and the feature beat array according to the following formula: (R*g,x)=∫R * (t)g(x+t)dt Where (R*g,x) is the calculated cross-correlation function, R is the audio feature, g is the feature beat array, and t is the period.

6. The method for estimating the beat speed according to claim 5, characterized in that: The method of selecting a cross-correlation function whose dynamic range meets a preset threshold as a tempo estimation result of the current music based on the cross-correlation processing result includes: Calculate the difference between the maximum and minimum values ​​in each cross-correlation function; The cross-correlation function with the largest difference is selected as the tempo estimation result of the current music.

7. A beat speed estimation device, characterized in that: include: A feature extraction unit, used to extract audio features from the current music; An autocorrelation processing unit, used for performing autocorrelation processing on the audio features; An option listing unit, used to list multiple possible options of the number of beats per minute for the current music; An array generating unit, for generating a characteristic beat array based on each possible option of the number of beats per minute; A cross-correlation processing unit, used for performing cross-correlation processing on the audio features after the autocorrelation processing and each of the feature beat arrays; A range selection unit, for selecting a cross-correlation function whose dynamic range meets a preset threshold as a tempo estimation result of the current music based on the cross-correlation processing result; The option listing unit includes: A period determination unit, used to determine a target period corresponding to a beat according to a search range of music speed; A peak value acquisition unit, used for acquiring all peak values ​​in the target period and the delay time and beats per minute corresponding to the peak values; The peak value setting unit is used to take all peak values ​​in the target cycle as possible options for the beats per minute.

8. A computer device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the beat speed estimation method according to any one of claims 1 to 6 when executing the computer program.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for estimating the beat speed according to any one of claims 1 to 6 is implemented.

Images