Music beat detection method, detection device and electronic equipment

Abstract

The present application provides a method, a detection device and an electronic device for detecting music beats. The method includes: acquiring target audio data; establishing a state space based on the target audio data, the state space including multiple BPMs and multiple audio points with hidden states, the audio points are obtained by splitting the BPM according to the duration of the target audio data, and the hidden states include the state of being a beat point or the state of not being a beat point; determining whether the target audio data is a beat point at least based on the spatial state. This method takes into account the fact that the music rhythm itself has the characteristic of repetitive rules, by establishing a discrete state space including BPM and multiple audio points under BPM, and determining whether the target audio data is a beat point based on the state space, so that the state of being a beat point and the state of not being a beat point can be measured at the same time, thereby ensuring the high accuracy of the result.

Description

Technical Field

[0001] The present application relates to the field of music information analysis, and more specifically, to a music beat detection method, a detection device, a computer-readable storage medium, a processor, and an electronic device. Background Art

[0002] Rhythm and pitch are the two fundamental components of music. Automatically tracking music is an important technical approach in music information analysis, and can be applied to tasks such as music plagiarism. Current mainstream music rhythm detection methods are limited to energy variations within a small range. For example, patent CN107103917A uses the energy peak between three adjacent frames to determine whether a beat is present. This method only considers the energy distribution of beat points. While beat points do have higher energy at adjacent time points, points with higher energy are not necessarily beat points, resulting in low rhythm detection accuracy.

[0003] The above information disclosed in the background technology section is only used to enhance the understanding of the background technology of the technology described in this article. Therefore, the background technology may contain certain information that does not form the prior art known in this country to those skilled in the art. Summary of the Invention

[0004] The main purpose of this application is to provide a music beat detection method, detection device, computer-readable storage medium, processor and electronic device to solve the problem in the prior art of determining whether it is a beat point based on energy distribution, resulting in low detection accuracy.

[0005] According to one aspect of an embodiment of the present invention, a method for detecting music beats is provided, comprising: acquiring target audio data; establishing a state space based on the target audio data, the state space comprising multiple BPMs and multiple audio points of each BPM, the audio point having a hidden state, each audio point being obtained by splitting the BPM according to the duration of the target audio data, the hidden state comprising a state of being a beat point or a state of not being a beat point; and determining whether the target audio data is the beat point based at least on the state space.

[0006] Optionally, establishing a state space according to the target audio data includes: obtaining a plurality of the BPMs according to Determine a first number fps, where T1=1 second and T2 is the duration of the target audio data; Determine the number M of audio points of each BPM; and split the BPM according to the number of each audio point to obtain multiple audio points.

[0007] Optionally, determining whether the target audio data is the beat point at least based on the state space includes: determining a plurality of first probabilities based on the state space, the first probability being the probability that any two of the audio points are adjacent in time; determining an amplitude sum, the amplitude sum being the sum of the amplitudes of the target audio data within a predetermined frequency band, and the predetermined frequency band being a preset frequency band range; determining a plurality of second probabilities based on the state space and the amplitude sum, the second probability being the probability that each of the audio points corresponding to the target audio data has the corresponding hidden state; determining whether the target audio data is the beat point and determining the BPM of the target audio data based on each of the first probabilities and each of the second probabilities.

[0008] Optionally, determining a plurality of first probabilities according to the state space includes: Determine each of the first probabilities, wherein, ω t is the BPM of the audio point at time t, ω t-1 is the BPM of the audio point at time t-1, represents the audio point at the time t.

[0009] Optionally, determining a plurality of second probabilities according to the state space and the amplitude sum includes: according to Determine each second probability, where i is the hidden state, i=0 or 1, when i=0, y i Indicates that the hidden state of the audio point is not the state of the beat point. When i=1, y i The hidden state representing the audio point is the state of the beat point, and S(t) is the amplitude sum.

[0010] Optionally, determining whether the target audio data is the beat point and determining the BPM of the target audio data according to each of the first probabilities and each of the second probabilities includes: Determine Y t The hidden state of the corresponding audio point is the hidden state of the target audio data, Y t The corresponding BPM is the BPM of the target audio data, P(S0) is the preset probability value corresponding to t=0, x1-T is the time domain value of the target audio data.

[0011] Optionally, determining the amplitude sum based on the target audio data includes: determining a time domain signal of the target audio data; performing Fourier transform on the time domain signal to obtain a frequency domain signal; and determining the amplitude sum based on the frequency domain signal and the predetermined frequency band.

[0012] Optionally, there are multiple target audio data, and among the multiple target audio data, the first target audio data, the second target audio data, and the third target audio data are three beat points that are adjacent in time sequence. After determining whether the target audio data is the beat point at least based on the state space, the method further includes: obtaining a first time interval between the first target audio data and the second target audio data; obtaining a second time interval between the second target audio data and the third target audio data; and obtaining a second time interval between the second target audio data and the third target audio data according to the state space. Determine the accuracy of the third target audio data as the beat point, where μ t =0.9μ t-1 +0.1X t , μ t-1 =0.9μ t-2 +0.1X t-1 , X t-1 is the first time interval, X t is the second time interval; and when the accuracy is less than or equal to a predetermined value, determining that the third target audio data is not the beat point.

[0013] According to another aspect of an embodiment of the present invention, a device for detecting a music beat is further provided, comprising: a first acquisition unit for acquiring target audio data; an establishment unit for establishing a state space based on the target audio data, wherein the state space comprises multiple BPMs and multiple audio points of each of the BPMs, the audio points having a hidden state, each of the audio points being obtained by splitting the BPM according to the duration of the target audio data, the hidden state comprising a state of being a beat point or a state of not being a beat point; and a first determination unit for determining whether the target audio data is the beat point based at least on the state space.

[0014] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is further provided, wherein the computer-readable storage medium includes a stored program, wherein the program executes any one of the methods described above.

[0015] According to another aspect of an embodiment of the present invention, a processor is provided, wherein the processor is configured to run a program, wherein any one of the methods is executed when the program is run.

[0016] According to another aspect of an embodiment of the present invention, an electronic device is also provided, comprising: one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include methods for executing any one of the methods described.

[0017] In an embodiment of the present invention, first, target audio data is acquired; then, a state space is established based on the target audio data, the state space including multiple BPMs and multiple audio points with hidden states, the audio points being obtained by splitting the BPM according to the duration of the target audio data, the hidden states including states of being a beat point or not being a beat point; finally, whether the target audio data is a beat point is determined based at least on the spatial state. Compared to the prior art, which only considers whether the target audio data is a beat point from an energy perspective, resulting in a low accuracy rate for rhythm detection, this method takes into account the fact that musical rhythm itself has repetitive regular characteristics, establishes a discrete state space including BPM and multiple audio points under the BPM, and determines whether the target audio data is a beat point based on the state space. In this way, the states of being a beat point and not being a beat point can be measured simultaneously, thereby ensuring a high accuracy of the result, thereby solving the problem in the prior art of determining whether a beat point is determined based on energy distribution, resulting in a low detection accuracy rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings that constitute part of this application are used to provide a further understanding of this application. The illustrative embodiments of this application and their descriptions are used to explain this application and do not constitute an improper limitation on this application. In the drawings:

[0019] Figure 1 A schematic diagram showing a flow chart of a method for detecting music beat according to an embodiment of the present application is shown;

[0020] Figure 2 A schematic structural diagram of a music beat detection device according to an embodiment of the present application is shown;

[0021] Figure 3 A schematic diagram of a state space according to an embodiment of the present application is shown;

[0022] Figure 4 The flowchart of the method for detecting music beat according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0023] It should be noted that the following detailed descriptions are illustrative and intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which the present application belongs.

[0024] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, they indicate the presence of features, steps, operations, devices, components and / or combinations thereof.

[0025] It should be understood that when an element (such as a layer, film, region, or substrate) is described as being "on" another element, the element may be directly on the other element or intervening elements may be present. Moreover, in the specification and claims, when it is described that an element is "connected to" another element, the element may be "directly connected to" the other element or "connected to" the other element through a third element.

[0026] For ease of description, some nouns or terms involved in the embodiments of the present application are explained below:

[0027] Beats per minute (BPM): The number of sound beats produced in a one-minute period.

[0028] As mentioned in the background technology, the existing technology determines whether it is a beat point based on energy distribution, resulting in low detection accuracy. In order to solve the above problem, a typical embodiment of the present application provides a music beat detection method, detection device, computer-readable storage medium, processor and electronic device.

[0029] According to an embodiment of the present application, a method for detecting music beat is provided.

[0030] Figure 1 FIG. 1 is a flow chart of a method for detecting a music beat according to an embodiment of the present application. Figure 1 As shown, the method includes the following steps:

[0031] Step S101, obtaining target audio data;

[0032] Step S102: Establishing a state space based on the target audio data, wherein the state space includes multiple BPMs and multiple audio points at each BPM, each audio point having a hidden state. Each audio point is obtained by splitting the BPM according to the duration of the target audio data, and the hidden state includes a state of being a beat point or a state of not being a beat point.

[0033] Step S103: Determine whether the target audio data is the beat point at least based on the state space.

[0034] In the above method, first, the target audio data is obtained; then, a state space is established based on the target audio data, the state space including multiple BPMs and multiple audio points with hidden states, the audio points being obtained by splitting the BPM according to the duration of the target audio data, and the hidden states including the state of being a beat point or the state of not being a beat point; finally, whether the target audio data is a beat point is determined at least based on the spatial state. Compared with the prior art, which only considers whether the target audio data is a beat point from the energy perspective, resulting in a low accuracy rate of rhythm detection, this method takes into account the repetitive regular characteristics of the musical rhythm itself, establishes a discrete state space including BPM and multiple audio points under the BPM, and determines whether the target audio data is a beat point based on the state space. In this way, the state of being a beat point and the state of not being a beat point can be measured simultaneously, thereby ensuring a high accuracy of the result, thereby solving the problem of the prior art that determining whether it is a beat point based on energy distribution results in a low detection accuracy rate.

[0035] It should be noted that the above method can be applied to real-time audio data or a complete song. When applied to real-time audio data, the target audio data is the real-time audio data; when applied to a complete song, the target audio data is each segment of the song.

[0036] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0037] In one embodiment of the present application, a state space is established based on the target audio data, including: obtaining a plurality of the above BPMs; Determine the first number fps, where T1 = 1 second and T2 is the duration of the target audio data; Determine the number M of audio points of each BPM; split the BPM according to the number of each audio point to obtain multiple audio points. Each audio point has a hidden state, which refers to whether it is a beat point under the corresponding BPM. Because there is a time interval between beat points, this time interval can be calculated by BPM, that is, there are multiple non-beat points between beat points. In order to ensure the uniformity of time resolution under different spatial states (i.e., different BPMs), therefore, in this embodiment, it is necessary to use To calculate the number of audio points (ie the number of hidden states).

[0038] Specifically, according to the BPM statistics of common songs, the number of BPMs can be 55 to 215, so the total number of BPMs is 161. When the duration T2 of the target audio data is 10ms, the first number fps is 100. Those skilled in the art can set the duration of the target audio data according to actual needs. According to the different time intervals corresponding to different BPMs, the number of BPMs with different resolutions under the corresponding BPM is set to better track the state transitions under different BPMs.

[0039] For example, when the BPM is 55, the number of audio points in this state is calculated by the above formula to be 109 (that is, the number of audio points is 109). Assuming that audio point 1 is a beat point and the rest are non-beat points, then if the BPM remains unchanged, the number of non-beat points between beat points is 108. The state space is as follows Figure 3 As shown, the black dots in the figure represent each element in the state space.

[0040] In another embodiment of the present application, determining whether the target audio data is the beat point is performed at least based on the state space, including: determining a plurality of first probabilities based on the state space, the first probability being the probability that any two of the audio points are temporally adjacent; determining an amplitude sum, the amplitude sum being the sum of the amplitudes of the target audio data within a predetermined frequency band, the predetermined frequency band being a preset frequency band range; determining a plurality of second probabilities based on the state space and the amplitude sum, the second probability being the probability that each of the audio points corresponding to the target audio data has the corresponding hidden state; determining whether the target audio data is the beat point and determining the BPM of the target audio data based on each of the first probabilities and the second probabilities. In actual situations, there is usually only one possibility for the state between beat points, that is, gradually transferring to the next beat point under the BPM; the BPM transition is possible only when it is at a beat point, that is, the next beat point is still a beat point, but it is not the beat point under the previous BPM. In this embodiment, the probability that any two of the above audio points are adjacent in time is first determined, and then the probability that each of the above audio points has a specified hidden state is determined based on the amplitude and, finally, based on the first probability and the second probability, whether the target audio data is a beat point and its corresponding BPM are determined, further realizing the combination of the periodic characteristics of the rhythm and the change rules between the beat points to determine whether the target audio data is a beat point, further ensuring the accuracy of rhythm detection.

[0041] In another embodiment of the present application, according to the above state space, determining multiple first probabilities includes: according to Determine each of the above first probabilities, where ω t is the BPM of the audio point at time t, ωt-1 is the BPM of the audio point at time t-1, In this embodiment, the first probabilities are obtained by the above formula, which facilitates the subsequent elimination of unreasonable audio points based on the above first probabilities, making the detection result more accurate, thereby further improving the accuracy of rhythm detection.

[0042] In a specific embodiment, the above-mentioned first probability can also be obtained by training music data, marking the beat points of the music data as 1 according to BPM, and taking the middle position of the beat point as a value between 0 and 1 according to the time progress, and training is performed based on the marked value.

[0043] In order to further improve the accuracy of rhythm detection, in another embodiment of the present application, multiple second probabilities are determined based on the above state space and the above amplitude and, including: Determine each of the above second probabilities, where i is the above hidden state, i = 0 or 1, when i = 0, y i Indicates that the hidden state of the audio point is not the state of the beat point. When i=1, y i The hidden state of the audio point is the state of the beat point, and S(t) is the amplitude and y. i It represents the state that people actually feel, that is, whether it is a beat point. Unlike the hidden state in the state space, people only feel two states, namely beat points and non-beat points, while the hidden state actually changes with the change of the state. For example, when the BPM value is 55 as mentioned above, there are 109 hidden state results, but we can define hidden state 1 as a beat point and the other hidden states as non-beat points.

[0044] In a specific embodiment, the second probability can also be obtained by training music data, labeling the status of each frame of audio in the training set according to fps, and performing training based on the labeling.

[0045] In another embodiment of the present application, determining whether the target audio data is the beat point and determining the BPM of the target audio data according to each of the first probabilities and each of the second probabilities includes: Determine Y t The hidden state of the corresponding audio point is the hidden state of the target audio data, Y t The corresponding BPM is the BPM of the target audio data, P(S0) is the probability value corresponding to the preset t=0, x1-Tis the time domain value of the target audio data. Because unreasonable audio points are eliminated during the process of determining the first and second probabilities, in this embodiment, a more accurate detection result can be obtained based on the more accurate first and second probabilities, thereby further improving the accuracy of rhythm detection.

[0046] In another embodiment of the present application, determining the amplitude sum based on the above-mentioned target audio data includes: determining the time domain signal of the above-mentioned target audio data; performing Fourier transform on the above-mentioned time domain signal to obtain a frequency domain signal; and determining the above-mentioned amplitude sum based on the above-mentioned frequency domain signal and the above-mentioned predetermined frequency band.

[0047] Specifically, the time domain signal x(t) of the target audio data is determined, where t = 1, 2, ..., T represents a time. Then, the time domain signal is Fourier transformed to obtain a frequency domain signal X(f), where f represents the audio point corresponding to the Fourier transform. Then, the predetermined frequency range is set to 55 Hz-4000 Hz, and the amplitude and sum are calculated according to the following formula:

[0048]

[0049] The reason for selecting the 55-4000Hz frequency band is that the pitch of a musical note corresponds to a fixed standard frequency value, and the commonly used 5-octave pitch is between 55-1760Hz. At the same time, the experimental value is obtained by considering the influence of the first few harmonics and reducing other noise interference.

[0050] In another embodiment of the present application, there are multiple target audio data, and among the multiple target audio data, the first target audio data, the second target audio data, and the third target audio data are the three beat points that are adjacent in chronological order. After determining whether the target audio data is the beat point at least based on the state space, the method further includes: obtaining a first time interval between the first target audio data and the second target audio data; obtaining a second time interval between the second target audio data and the third target audio data; and obtaining a second time interval between the second target audio data and the third target audio data according to the state space. Determine the accuracy of the third target audio data as the beat point, where μ t =0.9μ t-1 +0.1X t , μ t-1 =0.9μ t-2 +0.1X t-1 , X t-1 is the first time interval mentioned above, X tis the second time interval; if the accuracy is less than or equal to the predetermined value, the third target audio data is determined not to be the beat point. In this embodiment, a correlation analysis is performed based on the first time interval and the second time interval to determine whether the determination result is accurate, thereby further improving the accuracy of the detection.

[0051] It should be noted that, in the above embodiment, μ t is the average value of all time intervals between the first beat point detected in a piece of music and the third target audio point, μ t-1 is the average value of all time intervals between the first detected beat point and the second target audio point, μ t-2 Similarly, the time interval refers to the time interval between two adjacent beat points. When the first target audio point is the first beat point detected, then μ t-1 is equal to the first time interval. When the first target audio point is not the first beat point detected, μ t-1 is equal to 0.9μ t-2 +0.1X t-1 .

[0052] In a specific embodiment of the present application, the above-mentioned predetermined value is 0.95. Of course, in actual applications, the above-mentioned predetermined value can also be other values, and those skilled in the art can set it according to actual conditions.

[0053] The present application also provides a device for detecting a music beat. It should be noted that the device for detecting a music beat in the present application can be used to execute the method for detecting a music beat provided in the present application. The following describes the device for detecting a music beat provided in the present application.

[0054] Figure 2 Schematic diagram of a device for detecting music beat according to an embodiment of the present application. Figure 2 As shown, the device includes:

[0055] A first acquiring unit 10 is configured to acquire target audio data;

[0056] An establishing unit 20 is configured to establish a state space based on the target audio data, wherein the state space includes a plurality of BPMs and a plurality of audio points at each of the BPMs, wherein the audio points have a hidden state, each of the audio points being obtained by splitting the BPM according to the duration of the target audio data, and the hidden state includes a state of being a beat point or a state of not being a beat point;

[0057] The first determining unit 30 is configured to determine whether the target audio data is the beat point at least according to the state space.

[0058] The above-mentioned device includes an acquisition unit, an establishment unit, and a determination unit, wherein the acquisition unit is configured to acquire target audio data; the establishment unit is configured to establish a state space based on the target audio data, the state space comprising multiple BPMs and multiple audio points having hidden states, the audio points being obtained by splitting the BPM according to the duration of the target audio data, the hidden states comprising a state of being a beat point or a state of not being a beat point; and the determination unit is configured to determine whether the target audio data is a beat point based on at least the spatial state. Compared to the prior art, which only considers whether the target audio data is a beat point from an energy perspective, resulting in low rhythm detection accuracy, the present device takes into account the repetitive regularity of musical rhythm and establishes a discrete state space comprising the BPM and multiple audio points under the BPM. Based on the state space, it determines whether the target audio data is a beat point. This allows for simultaneous measurement of both beat point states and non-beat point states, thereby ensuring high accuracy of the result and thus resolving the prior art problem of determining whether a beat point is a beat point based on energy distribution, resulting in low detection accuracy.

[0059] In one embodiment of the present application, the establishment unit includes an acquisition module, a first determination module, a second determination module and a processing module, wherein the acquisition module is used to acquire a plurality of the above-mentioned BPMs; the first determination module is used to determine the BPMs according to the BPMs. Determine the first number fps, where T1=1 second, T2 is the duration of the target audio data; the second determination module is used to determine the first number fps according to Determine the number M of audio points of each BPM; the processing module is used to split the BPM according to the number of each audio point to obtain multiple audio points. Each audio point has a hidden state, which refers to whether it is a beat point under the corresponding BPM. Because there is a time interval between beat points, this time interval can be calculated by BPM, that is, there are multiple non-beat points between beat points. In order to ensure the uniformity of time resolution under different spatial states (i.e. different BPMs), therefore, in this embodiment, it is necessary to To calculate the number of audio points (ie the number of hidden states).

[0060] Specifically, based on BPM statistics for common songs, the number of BPMs can range from 55 to 215, resulting in a total of 161 BPMs. The fps setting needs to be tailored to actual needs; in this application, the fps setting is set to 100. Depending on the time intervals corresponding to different BPMs, the BPM number at different resolutions is set to better track state transitions at different BPMs.

[0061] For example, when the BPM is 55, the number of audio points in this state is calculated by the above formula to be 109 (that is, the number of hidden states is 109). Assuming that hidden state 1 is a beat point and the rest are non-beat points, then if the BPM remains unchanged, the number of non-beat points between beat points is 108. The state space is as follows Figure 3 As shown, the black dots in the figure represent each element in the state space.

[0062] In another embodiment of the present application, the above-mentioned determination unit includes a third determination module, a fourth determination module, a fifth determination module and a sixth determination module, wherein the above-mentioned third determination module is used to determine multiple first probabilities based on the above-mentioned state space, and the above-mentioned first probability is the probability that any two of the above-mentioned audio points are adjacent in time; the above-mentioned fourth determination module is used to determine the amplitude sum, and the above-mentioned amplitude sum is the sum of the amplitudes of the above-mentioned target audio data within a predetermined frequency band, and the above-mentioned predetermined frequency band is a preset frequency band range; the above-mentioned fifth determination module is used to determine multiple second probabilities based on the above-mentioned state space and the above-mentioned amplitude sum, and the above-mentioned second probability is the probability that each of the above-mentioned audio points corresponding to the above-mentioned target audio data has the corresponding above-mentioned hidden state; the above-mentioned sixth determination module is used to determine whether the above-mentioned target audio data is the above-mentioned beat point, and determine the above-mentioned BPM of the above-mentioned target audio data based on each of the above-mentioned first probabilities and each of the above-mentioned second probabilities. In actual situations, there is usually only one possibility for the state between beat points, that is, gradually transferring to the next beat point under the BPM; the BPM transition is possible only when it is at a beat point, that is, the next beat point is still a beat point, but it is not the beat point under the previous BPM. In this embodiment, the probability that any two of the above audio points are adjacent in time is first determined, and then the probability that each of the above audio points has a specified hidden state is determined based on the amplitude and, finally, based on the first probability and the second probability, whether the target audio data is a beat point and its corresponding BPM are determined, further realizing the combination of the periodic characteristics of the rhythm and the change rules between the beat points to determine whether the target audio data is a beat point, further ensuring the accuracy of rhythm detection.

[0063] In another embodiment of the present application, the third determination module includes a first determination submodule, wherein the first determination submodule is configured to determine the Determine each of the above first probabilities, where ω t is the BPM of the audio point at time t, ω t-1 is the BPM of the audio point at time t-1, In this embodiment, by eliminating unreasonable audio points, the detection result is made more accurate, thereby further improving the accuracy of rhythm detection.

[0064] In a specific embodiment, the above-mentioned first probability can also be obtained by training music data, marking the beat points of the music as 1 according to BPM, and taking the middle position of the beat point as a value between 0 and 1 according to the time progress, and training is performed based on the marked value.

[0065] In order to further improve the accuracy of rhythm detection, in another embodiment of the present application, the fifth determination module includes a second determination submodule, wherein the second determination submodule is configured to determine the rhythm according to the Determine each of the above second probabilities, where i is the above hidden state, i = 0 or 1, when i = 0, y i Indicates that the hidden state of the audio point is not the state of the beat point. When i=1, y i The hidden state of the audio point is the state of the beat point, and S(t) is the amplitude and y. i It represents the state that people actually feel, that is, whether it is a beat point. Unlike the hidden state in the state space, people only feel two states, namely beat points and non-beat points, while the hidden state actually changes with the change of the state. For example, when the BPM value is 55 as mentioned above, there are 109 hidden state results, but we can define hidden state 1 as a beat point and the other hidden states as non-beat points.

[0066] In a specific embodiment, the second probability can also be obtained by training music data, labeling the status of each frame of audio in the training set according to fps, and performing training based on the labeling.

[0067] In another embodiment of the present application, the sixth determination module includes a third determination submodule, wherein the third determination submodule is configured to determine the Determine Y t The hidden state of the corresponding audio point is the hidden state of the target audio data, Y t The corresponding BPM is the BPM of the target audio data, P(S0) is the probability value corresponding to the preset t=0, and x 1-T is the time domain value of the target audio data. Because unreasonable audio points are eliminated during the process of determining the first and second probabilities, in this embodiment, a more accurate detection result can be obtained based on the more accurate first and second probabilities, thereby further improving the accuracy of rhythm detection.

[0068] In another embodiment of the present application, the above-mentioned fourth determination module includes a fourth determination sub-module, a processing sub-module and a determination sub-module, wherein the above-mentioned fourth determination sub-module is used to determine the time domain signal of the above-mentioned target audio data; the above-mentioned processing sub-module is used to perform Fourier transform on the above-mentioned time domain signal to obtain a frequency domain signal; the above-mentioned determination sub-module is used to determine the above-mentioned amplitude and based on the above-mentioned frequency domain signal and the above-mentioned predetermined frequency band.

[0069] Specifically, the time domain signal x(t) of the target audio data is determined, where t = 1, 2, ..., T represents a time. Then, the time domain signal is Fourier transformed to obtain a frequency domain signal X(f), where f represents the audio point corresponding to the Fourier transform. Then, the predetermined frequency range is set to 55 Hz-4000 Hz, and the amplitude and sum are calculated according to the following formula:

[0070]

[0071] The reason for selecting the 55-4000Hz frequency band is that the pitch of a musical note corresponds to a fixed standard frequency value, and the commonly used 5-octave pitch is between 55-1760Hz. At the same time, the experimental value is obtained by considering the influence of the first few harmonics and reducing other noise interference.

[0072] In another embodiment of the present application, there are multiple target audio data, and among the multiple target audio data, the first target audio data, the second target audio data and the third target audio data are the three beat points that are adjacent in chronological order. After determining whether the target audio data is the beat point at least based on the state space, the device further includes a second acquisition unit, a third acquisition unit, a second determination unit and a third determination unit, wherein the second acquisition unit is used to obtain a first time interval between the first target audio data and the second target audio data; the third acquisition unit is used to obtain a second time interval between the second target audio data and the third target audio data; and the second determination unit is used to determine whether the target audio data is the beat point based on the state space. Determine the accuracy of the third target audio data as the beat point, where μ t =0.9μ t-1 +0.1X t , μ t-1 =0.9μ t-2 +0.1X t-1 , X t-1 is the first time interval mentioned above, X tis the second time interval; and the third determination unit is configured to determine, if the accuracy is less than or equal to a predetermined value, that the third target audio data is not the beat point. In this embodiment, a correlation analysis is performed based on the first time interval and the second time interval to determine whether the determination result is accurate, thereby further improving the accuracy of the detection.

[0073] It should be noted that, in the above embodiment, μ t is the average value of all time intervals between the first beat point detected in a piece of music and the third target audio point, μ t-1 is the average value of all time intervals between the first detected beat point and the second target audio point, μ t-2 Similarly, the time interval refers to the time interval between two adjacent beat points. When the first target audio point is the first beat point detected, then μ t-1 is equal to the first time interval. When the first target audio point is not the first beat point detected, μ t-1 is equal to 0.9μ t-2 +0.1X t-1 .

[0074] In a specific embodiment of the present application, the above-mentioned predetermined value is 0.95. Of course, in actual applications, the above-mentioned predetermined value can also be other values, and those skilled in the art can set it according to actual conditions.

[0075] The above-mentioned music beat detection device includes a processor and a memory. The above-mentioned first acquisition unit, establishment unit and first determination unit are all stored in the memory as program units, and the processor executes the above-mentioned program units stored in the memory to realize corresponding functions.

[0076] The processor includes a kernel, which retrieves the corresponding program unit from the memory. One or more kernels can be configured, and kernel parameters can be adjusted to address the existing problem of determining whether a beat point is based on energy distribution, resulting in low detection accuracy.

[0077] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0078] An embodiment of the present invention provides a computer-readable storage medium having a program stored thereon. When the program is executed by a processor, the method for detecting the music beat is implemented.

[0079] An embodiment of the present invention provides a processor, which is used to run a program, wherein the program executes the music beat detection method when running.

[0080] An embodiment of the present invention provides an electronic device, comprising: one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include functions for executing any one of the above methods.

[0081] The above-mentioned electronic device includes one or more processors, a memory and one or more programs, wherein the one or more programs are stored in the above-mentioned memory and are configured to be executed by the one or more processors, and the one or more programs include methods for executing any of the above-mentioned methods. Compared with the prior art, which only considers whether the target audio data is a beat point from the energy perspective, resulting in a low accuracy rate of rhythm detection, this method takes into account the characteristics of the music rhythm itself with repetitive rules, and establishes a discrete state space including BPM and multiple audio points under BPM, and determines whether the target audio data is a beat point based on the state space. In this way, the state of being a beat point and the state of not being a beat point can be measured at the same time, thereby ensuring a high accuracy of the result, thereby solving the problem of the prior art that determining whether it is a beat point based on energy distribution results in a low detection accuracy rate.

[0082] An embodiment of the present invention provides a device, comprising a processor, a memory, and a program stored in the memory and executable on the processor. When the processor executes the program, at least the following steps are performed:

[0083] Step S101, obtaining target audio data;

[0084] Step S102: Establishing a state space based on the target audio data, wherein the state space includes multiple BPMs and multiple audio points at each BPM, each audio point having a hidden state. Each audio point is obtained by splitting the BPM according to the duration of the target audio data, and the hidden state includes a state of being a beat point or a state of not being a beat point.

[0085] Step S103: Determine whether the target audio data is the beat point at least based on the state space.

[0086] The devices in this article can be servers, PCs, PADs, mobile phones, etc.

[0087] The present application also provides a computer program product, which, when executed on a data processing device, is adapted to execute a program for initializing at least the following method steps:

[0088] Step S101, obtaining target audio data;

[0089] Step S102: Establishing a state space based on the target audio data, wherein the state space includes multiple BPMs and multiple audio points at each BPM, each audio point having a hidden state. Each audio point is obtained by splitting the BPM according to the duration of the target audio data, and the hidden state includes a state of being a beat point or a state of not being a beat point.

[0090] Step S103: Determine whether the target audio data is the beat point at least based on the state space.

[0091] In order to enable those skilled in the art to more clearly understand the technical solution of the present disclosure, the technical solution of the present disclosure will be described in detail below with reference to specific embodiments and comparative examples.

[0092] Example

[0093] The flowchart of the music beat detection method is as follows Figure 4 As shown, the method includes the following steps:

[0094] 1. Input: Determine the time domain signal x(t) of the target audio data, where t = 1, 2, ..., T represents the time.

[0095] 2. Determine the sum of the amplitudes: Perform a Fourier transform on the time domain signal to obtain a frequency domain signal X(f), where f represents the audio point corresponding to the Fourier transform. Then, set the predetermined frequency range to 55 Hz-4000 Hz and calculate the sum of the amplitudes according to the following formula:

[0096]

[0097] 3. Determine the first probability: First obtain multiple BPMs, and then Determine the first number fps, where T1 = 1 second, T2 is the duration of the target audio data, according to Determine the number M of audio points of each of the BPMs, and finally split the BPM according to the number of each of the audio points to obtain multiple audio points;

[0098] according to Determine a first probability, where ω t is the BPM of the audio point at time t, ω t-1 is the BPM of the audio point at time t-1, Represents the above audio point at the above time t.

[0099] 4. Determine the second probability: according to Determine each second probability, where i is the hidden state, i = 0 or 1, when i = 0, y i Indicates that the hidden state of the audio point is not the state of the beat point. When i=1, y i The hidden state of the audio point is the state of the beat point, and S(t) is the amplitude sum.

[0100] 5. Determine the beat and BPM: Determine Y t The hidden state of the corresponding audio point is the hidden state of the target audio data, Y t The corresponding BPM is the BPM of the target audio data, P(S0) is the probability value corresponding to the preset t=0, and x 1-T is the time domain value of the above target audio data.

[0101] 6. Determine the accuracy: Based on the time intervals of multiple beat points and Determine the accuracy of the beat point, where μ t =0.9μ t-1 +0.1X t , μ t-1 =0.9μ t-2 +0.1X t-1 , X t-1 is the first time interval mentioned above, X t is the second time interval mentioned above.

[0102] In the above embodiments of the present invention, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0103] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only exemplary. For example, the division of the above-mentioned units can be a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0104] The units described above as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0105] In addition, the functional units in the various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0106] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a computer-readable storage medium and includes a number of instructions for enabling a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the above-mentioned methods of each embodiment of the present invention. The aforementioned computer-readable storage medium includes: various media that can store program codes, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0107] From the above description, it can be seen that the above embodiments of the present application achieve the following technical effects:

[0108] 1) The music beat detection method of the present application, first, obtains the target audio data; then, establishes a state space based on the target audio data, the state space includes multiple BPMs and multiple audio points with hidden states, the audio points are obtained by splitting the above BPM according to the duration of the above target audio data, and the hidden states include the state of being a beat point or the state of not being a beat point; finally, determines whether the target audio data is a beat point at least based on the spatial state. Compared with the prior art, which only considers whether the target audio data is a beat point from the energy perspective, resulting in a low accuracy rate of rhythm detection, this method takes into account the characteristics of the repetitive rules of the music rhythm itself, by establishing a discrete state space including BPM and multiple audio points under BPM, and determining whether the target audio data is a beat point based on the state space. In this way, the state of being a beat point and the state of not being a beat point can be measured at the same time, thereby ensuring a high accuracy of the result, thereby solving the problem of the prior art of determining whether it is a beat point based on energy distribution, resulting in a low detection accuracy rate.

[0109] 2) The music beat detection device of the present application includes an acquisition unit, an establishment unit, and a determination unit, wherein the acquisition unit is used to acquire target audio data; the establishment unit is used to establish a state space based on the target audio data, the state space including multiple BPMs and multiple audio points with hidden states, the audio points being obtained by splitting the BPM according to the duration of the target audio data, the hidden states including being a beat point or not being a beat point; and the determination unit is used to determine whether the target audio data is a beat point based on at least the spatial state. Compared with the prior art, which only considers whether the target audio data is a beat point from the perspective of energy, resulting in low rhythm detection accuracy, the device takes into account the repetitive regular characteristics of musical rhythm itself and establishes a discrete state space including the BPM and multiple audio points under the BPM, and determines whether the target audio data is a beat point based on the state space. In this way, the state of being a beat point and the state of not being a beat point can be measured simultaneously, thereby ensuring high accuracy of the result, thereby solving the problem of low detection accuracy caused by determining whether a beat point is based on energy distribution in the prior art.

[0110] 3) The electronic device of the present application includes one or more processors, a memory and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include methods for executing any one of the above methods. Compared with the prior art, which only considers whether the target audio data is a beat point from the energy perspective, resulting in a low accuracy rate of rhythm detection, this method takes into account the characteristics of the music rhythm itself with repetitive rules, and establishes a discrete state space including BPM and multiple audio points under BPM, and determines whether the target audio data is a beat point based on the state space. In this way, the state of being a beat point and the state of not being a beat point can be measured at the same time, thereby ensuring a high accuracy of the result, and further solving the problem in the prior art of determining whether it is a beat point based on energy distribution, resulting in a low detection accuracy rate.

[0111] The above description is merely a preferred embodiment of the present application and is not intended to limit the present application. Various modifications and variations are possible for those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present application shall be included within the scope of protection of the present application.

Claims

1. A method for detecting music beat, characterized in that: include: Get target audio data; Establishing a state space based on the target audio data, the state space including a plurality of BPMs and a plurality of audio points of each BPM, the audio point having a hidden state, each audio point being obtained by splitting the BPM according to the duration of the target audio data, the hidden state including a state of being a beat point or a state of not being a beat point; determining, at least according to the state space, whether the target audio data is the beat point; Determining whether the target audio data is the beat point at least according to the state space includes: determining, according to the state space, a plurality of first probabilities, where the first probabilities are probabilities that any two of the audio points are temporally adjacent; Determining a sum of amplitudes, where the sum of amplitudes is the sum of amplitudes of the target audio data within a predetermined frequency band, where the predetermined frequency band is a preset frequency band range; determining, based on the state space and the amplitude sum, a plurality of second probabilities, where the second probabilities are probabilities that each of the audio points corresponding to the target audio data has the corresponding hidden state; According to each of the first probabilities and each of the second probabilities, it is determined whether the target audio data is the beat point, and the BPM of the target audio data is determined.

2. The method according to claim 1, characterized in that Establishing a state space according to the target audio data, including: Obtaining a plurality of the BPMs; according to , determine a first number fps, where Second, is the duration of the target audio data; according to , determine the number M of audio points of each BPM; The BPM is split according to the number of each audio point to obtain a plurality of audio points.

3. The method according to claim 1, characterized in that Determining a plurality of first probabilities according to the state space includes: according to , determine each of the first probabilities, where , is the BPM of the audio point at time t, is the BPM of the audio point at time t-1, represents the audio point at the time t.

4. The method according to claim 3, characterized in that Determining a plurality of second probabilities according to the state space and the amplitude sum, including: according to , determine each second probability, where i is the hidden state, i=0 or 1, when i=0, Indicates that the hidden state of the audio point is not the state of the beat point. When i=1, The hidden state representing the audio point is the state of the beat point, and S(t) is the amplitude sum.

5. The method according to claim 4, characterized in that Determining whether the target audio data is the beat point and the BPM of the target audio data according to each of the first probabilities and each of the second probabilities includes: according to ,Sure The hidden state of the corresponding audio point is the hidden state of the target audio data, The corresponding BPM is the BPM of the target audio data, is the probability value corresponding to the preset t=0, is the time domain value of the target audio data.

6. The method according to claim 1, characterized in that Determining the amplitude and the value according to the target audio data includes: Determining a time domain signal of the target audio data; Performing Fourier transform on the time domain signal to obtain a frequency domain signal; The amplitude sum is determined according to the frequency domain signal and the predetermined frequency band.

7. The method according to any one of claims 1 to 6, characterized in that There are a plurality of target audio data, and among the plurality of target audio data, first target audio data, second target audio data, and third target audio data are three beat points that are sequentially adjacent in chronological order. After determining whether the target audio data is the beat point based on at least the state space, the method further includes: Acquire a first time interval between the first target audio data and the second target audio data; Acquire a second time interval between the second target audio data and the third target audio data; according to , determining the accuracy of the third target audio data as the beat point, wherein, , , is the first time interval, is the second time interval; If the accuracy is less than or equal to a predetermined value, it is determined that the third target audio data is not the beat point.

8. A device for detecting music beat, characterized in that: include: A first acquiring unit, configured to acquire target audio data; an establishing unit, configured to establish a state space based on the target audio data, the state space including a plurality of BPMs and a plurality of audio points of each of the BPMs, the audio points having a hidden state, each of the audio points being obtained by splitting the BPM according to the duration of the target audio data, the hidden state including a state of being a beat point or a state of not being a beat point; a first determining unit, configured to determine whether the target audio data is the beat point based at least on the state space; The first determination unit includes a third determination module, a fourth determination module, a fifth determination module, and a sixth determination module, wherein the third determination module is configured to determine a plurality of first probabilities based on the state space, wherein the first probability is a probability that any two audio points are temporally adjacent; the fourth determination module is configured to determine an amplitude sum, wherein the amplitude sum is the sum of amplitudes of the target audio data within a predetermined frequency band, wherein the predetermined frequency band is a preset frequency band range; The fifth determination module is used to determine multiple second probabilities based on the state space and the amplitude sum, where the second probability is the probability that each audio point corresponding to the target audio data has the corresponding hidden state; the sixth determination module is used to determine whether the target audio data is the beat point and determine the BPM of the target audio data based on each first probability and each second probability.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium includes a stored program, wherein the program executes the method according to any one of claims 1 to 7.

10. A processor, characterized in that: The processor is configured to run a program, wherein the program executes the method according to any one of claims 1 to 7 when running.

11. An electronic device, characterized in that: include: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include instructions for executing the method according to any one of claims 1 to 7.

Images