Post-processing method, device, equipment and storage medium for music beat detection

Abstract

This application discloses a post-processing method, apparatus, device, and storage medium for music beat detection, relating to the field of artificial intelligence technology. The method comprises: obtaining a beat detection result for a song; determining the rhythm range of the song based on the song's tempo and rhythm type; determining a musical measure pointer sequence using a dynamic Bayesian network based on the beat detection result and rhythm range; and obtaining the position of the beats in at least one musical measure within the musical measure based on the musical measure pointer sequence and rhythm type. This method analyzes the beat detection results of a song using a dynamic Bayesian network, deriving the position of the beats in the musical measure from the beat detection results, effectively reducing the probability of missed beats and extra beats, and improving the accuracy of beat detection.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of artificial intelligence, and in particular to a post-processing method, apparatus, device, and storage medium for music beat detection. Background Art

[0002] Currently, software related to the music field is becoming increasingly powerful. These types of software can be used for music creation, editing, performance, mixing and other music processing. With the development of technology, there is a need to determine the location of music beats when developing related software.

[0003] In related technologies, after obtaining the beat detection result of a song, a peak detection algorithm is directly used to find the peak in the beat detection result, and the found peak is determined as the position of the beat.

[0004] In the related art, there are songs without obvious beat features and with dense drum beats. The peak detection of the beat prediction results of such songs is inaccurate, and it is easy to miss beats or have extra beats when performing post-processing of the music beat detection. Summary of the Invention

[0005] The embodiments of the present application provide a method, apparatus, device, and storage medium. The technical solutions provided by the embodiments of the present application are as follows:

[0006] According to one aspect of an embodiment of the present application, a post-processing method for music beat detection is provided, the method comprising:

[0007] Obtaining a beat detection result of a song, wherein the song includes a plurality of frame data, and the beat detection result includes a beat probability value of each frame data in the plurality of frame data, wherein the beat probability value of the frame data refers to a probability value that the frame data is a beat;

[0008] Determining a rhythm range of the song based on the tempo and rhythm type of the song, wherein the tempo of the song is used to indicate the number of beats in the song per unit time, the rhythm type is used to indicate the number of beats included in a musical measure and the regularity of the beats, the rhythm range includes a plurality of candidate frame rhythms, and the frame rhythm is used to indicate the speed between adjacent frame data in the song;

[0009] Determining a music measure pointer sequence using a dynamic Bayesian network based on the beat detection result and the rhythm range, wherein the music measure pointer sequence is used to indicate a position of at least one music measure in the song;

[0010] Based on the music measure pointer sequence and the rhythm type, the position of the beat in the at least one music measure in the music measure is obtained.

[0011] According to one aspect of an embodiment of the present application, a post-processing device for music beat detection is provided, the device comprising:

[0012] a detection result acquisition module, configured to acquire a beat detection result of a song, wherein the song includes a plurality of frame data, the beat detection result includes a beat probability value of each frame data in the plurality of frame data, and the beat probability value of the frame data refers to a probability value that the frame data is a beat;

[0013] a rhythm range determination module, configured to determine a rhythm range of the song based on the tempo and rhythm type of the song, wherein the tempo of the song indicates the number of beats in the song per unit time, the rhythm type indicates the number of beats in a musical measure and the regularity of the beats, the rhythm range includes a plurality of candidate frame rhythms, and the frame rhythm indicates the speed between adjacent frame data in the song;

[0014] a pointer sequence determination module, configured to determine a music measure pointer sequence using a dynamic Bayesian network based on the beat detection result and the rhythm range, wherein the music measure pointer sequence is used to indicate a position of at least one music measure in the song;

[0015] The beat position determination module is used to obtain the position of the beat in the at least one music measure in the music measure based on the music measure pointer sequence and the rhythm type.

[0016] According to one aspect of an embodiment of the present application, a computer device is provided, comprising a processor and a memory, wherein a computer program is stored in the memory, and the processor executes the computer program to implement the above method.

[0017] According to one aspect of an embodiment of the present application, a computer-readable storage medium is provided, in which a computer program is stored. The computer program is configured to be executed by a processor to implement the above method.

[0018] According to one aspect of an embodiment of the present application, a computer program product is provided. The computer program product includes a computer program. The computer program is loaded and executed by a processor to implement the above method.

[0019] The technical solutions provided by the embodiments of the present application include at least the following beneficial effects:

[0020] By obtaining the beat detection result of the song, which includes the beat probability value of each frame data in multiple frame data of the song, the rhythm range of the song is determined according to the speed and rhythm type of the song. Based on the above beat detection result and rhythm range, a dynamic Bayesian network is used to determine the music measure pointer sequence corresponding to the song. Finally, based on the music measure pointer sequence and the rhythm type of the song, the position of the beat of at least one music measure in the corresponding music measure can be obtained. This realizes the analysis of the beat detection result of the song by using a dynamic Bayesian network, and the deduction of the beat position of the music measure in the music measure from the beat detection result of the song, which effectively reduces the probability of missed beats and multiple beats and improves the accuracy of beat detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 This is a schematic diagram of an implementation environment for a solution provided by an embodiment of the present application;

[0022] Figure 2 This is a flowchart of a post-processing method for music beat detection provided by one embodiment of the present application;

[0023] Figure 3 This is a flowchart of a simple example of a post-processing method for music beat detection provided by one embodiment of the present application;

[0024] Figure 4 This is a block diagram of a post-processing device for music beat detection provided by one embodiment of the present application;

[0025] Figure 5 This is a structural block diagram of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0026] In order to make the objectives, technical solutions and advantages of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0027] Please refer to Figure 1 , which shows a schematic diagram of an implementation environment of a solution provided by an embodiment of the present application. The implementation environment of the solution may include: a computer device 10.

[0028] In the method provided in the embodiment of the present application, the execution subject of each step may be a computer device 10, which is an electronic device with data calculation, processing and storage functions. The computer device 10 may be either a terminal device or a server.

[0029] Exemplarily, terminal devices include but are not limited to edge devices, mobile phones, computers, intelligent voice interaction devices, smart home appliances, vehicle-mounted terminals, aircraft, game consoles, wearable devices, multimedia playback devices, augmented reality (AR) devices, virtual reality (VR) devices and other electronic devices.

[0030] Exemplarily, the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as edge computing services, cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms, but is not limited to these.

[0031] like Figure 1 As shown, the computer device 10 can, by executing the method provided in the embodiment of the present application, decode the music measure pointer sequence using the music measure pointer model based on the beat detection result and the speed of the song, and then determine the position of each music measure in the song according to the above music measure pointer sequence. Finally, according to the rhythm type of the song, the position of the beat in each music measure in the corresponding music measure of the song can be obtained.

[0032] The embodiments of the present application can be applied to various scenarios, including but not limited to music creation, music editing, virtual instrument performance, online singing platforms, etc.

[0033] In the related art, after obtaining the beat detection result of a song, a peak detection algorithm is directly used to find the peak value from the beat detection result, and the found peak value is determined as the location of the beat in the song. The beat detection accuracy of the related art is not high. When using a neural network model for song analysis to predict the beat of a song without obvious beat features, the beat probability value of each frame data in the beat detection result is low. At this time, when using the peak detection algorithm to determine the location of the beat from the beat detection result, it is easy to miss a beat. When using a neural network model for song analysis to predict the beat of a song with dense drum beats, the beat detection result is easily interfered with by the drum beats, and the frame data at the location of the drum beat is mistakenly detected as a beat, resulting in multiple beats.

[0034] Please refer to Figure 2 , which shows a flowchart of a post-processing method for music beat detection provided by an embodiment of the present application. The method may include at least one of the following steps (210-240):

[0035] Step 210, obtaining the beat detection result of the song, wherein the song includes multiple frame data, and the beat detection result includes the beat probability value of each frame data in the multiple frame data, and the beat probability value of the frame data refers to the probability value of the frame data being a beat.

[0036] Beat, also known as musical tempo, beat, or beat, is an important musical feature that plays a crucial role in music rhythm analysis, structure analysis, automatic mixing, note extraction, and note quantization. The beats of the same song all have the same duration.

[0037] In some embodiments, the beat may include at least one of a strong beat and a weak beat. The strong beat, also known as the heavy beat, refers to the most emphasized beat in the entire musical measure, and is the first beat in the musical measure; the weak beat refers to other beats in the entire musical measure that are less important than the strong beat, and is the beat other than the first beat in the musical measure.

[0038] Optionally, the beat may further include a sub-strong beat, which is a beat that is weaker than the strong beat but stronger than the weak beat.

[0039] The beat detection result includes the probability value of each frame data in the song being a beat. The above beat probability value can refer to the probability value of the frame data being a strong beat, or the probability value of the frame data being a strong beat, or the probability value of the frame data being neither a strong beat nor a weak beat.

[0040] In some embodiments, a song can be predicted using a beat detection model to obtain a beat detection result for the song. The beat detection model is used to predict the probability value of each frame data in the song being a beat. Optionally, the beat detection result of the song can be a one-dimensional vector. For example, assuming that the number of frame data of the song is N, where N is a positive integer, the beat detection result is a one-dimensional vector with a length equal to the number of frame data of the song. Each element in the one-dimensional vector corresponds to a probability value of a frame data being a strong beat, and the size of the one-dimensional vector can be expressed as 1×N. Optionally, the beat detection result of the song can be a matrix. For example, assuming that the number of frame data of the song is N, where N is a positive integer, the beat detection result is a matrix with the number of columns equal to the number of frame data of the song and the number of rows being 2. The two rows of the matrix represent the probability values ​​of the frame data being a strong beat and a weak beat, respectively. That is, the beat detection result can be viewed as a matrix consisting of a one-dimensional vector representing the probability value of each element corresponding to a frame data being a strong beat and a one-dimensional vector representing the probability value of each element corresponding to a frame data being a weak beat.

[0041] In some embodiments, the beat detection model can be one of the following neural network models: Recurrent Neural Network (RNN), Convolutional Neural Network (CNN), Region-based Convolutional Recurrent Neural Network (R-CRNN), Long Short Term Memory (LSTM), etc., or it can be other models that can predict the probability value of each frame data in the song being a beat. This application does not limit this.

[0042] In some embodiments, the above song can be a complete song or a part of a complete song, and this application does not limit this.

[0043] In some embodiments, the song is divided into uniform frames of data, and the size of each frame of data in the song is consistent. For example, a song with a length of 60 seconds is evenly divided into 360 frames of data, and the size of each frame of data is 10 milliseconds.

[0044] Through the above method, the beat probability value corresponding to each frame in the song is obtained.

[0045] Step 220, based on the speed and rhythm type of the song, determine the rhythm range of the song, wherein the speed of the song is used to indicate the number of beats of the song in unit time, the rhythm type is used to indicate the number of beats included in a music measure and the regularity of the beats, the rhythm range includes multiple candidate frame rhythms, and the frame rhythm is used to indicate the speed between adjacent frame data in the song.

[0046] The speed of a song refers to the tempo and beat of the song. In some embodiments, the speed of a song can be measured by the number of beats per minute (BPM). It is understandable that the higher the BPM, the faster the song, while the lower the BPM, the slower the song.

[0047] In some embodiments, the tempo of a song can be obtained by using the tempo prediction model described above, or by analyzing the song using other models. For example, the song is input into the tempo prediction model described above, and the tempo and tempo detection results of the song are obtained simultaneously.

[0048] In some embodiments, the speed of the song can also be set by the user or relevant technical personnel, and this application does not limit this.

[0049] The rhythm type refers to the organization of strong and weak beats in each measure of a song. The number of beats contained in a measure of the song can be obtained based on the rhythm type of the song. There are many types of rhythm types for songs, including but not limited to the following types: 4 / 4, 3 / 4, 2 / 4, 6 / 8, 9 / 8, 12 / 8, etc. For example, assuming that the rhythm type of a song is 3 / 4, it means that each measure of the song includes 3 beats, namely the strong beat, the weak beat, and the weak beat, and each beat is a quarter note, which represents the duration of a beat.

[0050] A measure is the basic unit of time in music, used to organize and divide the rhythm and structure of music. Each measure is typically composed of a certain number of beats. The first beat of a measure is usually called the strong beat, while the remaining beats are called the weak beats. Each measure of a song remains the same length.

[0051] In some embodiments, the frame cadence is used to indicate the offset from the current frame data to the next frame data. For example, assuming a song has N frames of data, the current frame data is the kth frame data in the song, the next frame data is the k+1th frame data in the song, N is a positive integer, k is a positive integer less than N, and the frame cadence of the current frame data is m, then the position of the next frame data can be represented by the position of the current frame data plus m, where m is greater than 0.

[0052] In some embodiments, the number of beats included in each music measure is determined based on the rhythm type; the minimum rhythm and maximum rhythm are determined based on the number of beats, the speed adjustment parameter and the speed of the song, and the speed adjustment parameter is used to indicate the minimum speed and maximum speed accepted by the song; based on the minimum rhythm and maximum rhythm, the rhythm range is determined.

[0053] In some embodiments, the speed adjustment parameter is used to indicate the maximum and minimum speeds that a song can accept. The rhythm range is determined by the speed of the song. The speed adjustment parameter can be obtained based on experiments or directly set by relevant technical personnel. This application does not limit this.

[0054] In some embodiments, based on the above speed adjustment parameters, the maximum speed and minimum speed acceptable to the song can be adjusted using at least one of the following adjustment methods: linear adjustment, exponential adjustment, logarithmic adjustment, etc.

[0055] In some embodiments, the above rhythm range can be a series of discrete values ​​with the same difference. For example, assuming the minimum rhythm is The maximum rhythm is If the above difference is 1, the rhythm range can be expressed as

[0056] In some embodiments, the above rhythm range can also be a continuous value. For example, assuming the minimum rhythm is The maximum rhythm is The rhythm range can be expressed as

[0057] For example, assuming that a song is divided into N frames of data, represents the frame rhythm of the k-th frame data in the song, T is the speed of the song, and the frame rhythm of the k-th frame data is calculated as follows:

[0058]

[0059] Where n represents the total number of beats in a music measure, Δ represents the duration of each frame of data in milliseconds, round represents the rounding operator, and k is a positive integer.

[0060] Based on the previous example, assuming that the speed adjustment parameters and the speed of the song are adjusted, the minimum speed that the song can accept is T min , the maximum speed is T max , then the acceptable speed range of the song is T min ~T max , the minimum tempo of the above songs It can be expressed as:

[0061]

[0062] The maximum tempo of the above song It can be expressed as:

[0063]

[0064] The rhythm range can be expressed as

[0065] In the above manner, the speed range of the song is determined by adjusting the speed parameters to constrain the rhythm range.

[0066] In some embodiments, the speed of the song is subtracted from a first value to obtain a minimum speed, and the speed of the song is added to the first value to obtain a maximum speed, wherein the first value is the product of the speed of the song and the speed adjustment parameter; the minimum rhythm is determined based on the minimum speed and the number of beats, and the maximum rhythm is determined based on the maximum speed and the number of beats.

[0067] For example, assuming the speed adjustment parameter is 0.3 and the speed of the song is T, the first value is 0.3*T, and the maximum speed that the song can accept is T. max =T+0.3*T=1.3T, the minimum speed is T min=T-0.3*T=0.7T, that is, the acceptable speed range of the song is 0.7T~1.3T, and the minimum tempo is It can be expressed as:

[0068]

[0069] Maximum rhythm It can be expressed as:

[0070]

[0071] Where n represents the total number of beats in a musical measure, Δ represents the duration of each frame of data in milliseconds, and round represents the rounding operator.

[0072] In some embodiments, the speed of the song is subtracted from a third value to obtain a minimum speed, and the speed of the song is added to a first value to obtain a maximum speed, wherein the first value is a speed adjustment parameter; the minimum rhythm is determined based on the minimum speed and the number of beats, and the maximum rhythm is determined based on the maximum speed and the number of beats.

[0073] For example, assuming that the speed adjustment parameter is 20, the unit is the number of beats per minute, and the speed of the song is T, then the first value is 20, and the maximum speed that the song can accept is T. max =T+20, the minimum speed is T min =T-20, that is, the acceptable speed range of the song is T-20 ~ T+20, the minimum tempo It can be expressed as:

[0074]

[0075] Maximum rhythm It can be expressed as:

[0076]

[0077] Where n represents the total number of beats in a musical measure, Δ represents the duration of each frame of data in milliseconds, and round represents the rounding operator.

[0078] Through the above method, based on the speed adjustment parameter, the speed range accepted by the song is made to fluctuate up and down by a first value, which can effectively reduce the search space of the rhythm range and is conducive to improving the efficiency of subsequent decoding.

[0079] Step 230: Based on the beat detection result and the rhythm range, a dynamic Bayesian network is used to determine a music bar pointer sequence, where the music bar pointer sequence is used to indicate the position of at least one music bar in the song.

[0080] Dynamic Bayesian networks (DBNs) are an extension of Bayesian networks, used to model and infer probabilistic relationships that vary over time. Unlike traditional static Bayesian networks, DNBs can capture the time-varying nature of system states. In a DNB, nodes represent the system's state variables, and edges represent the dependencies between these variables. By defining conditional probability distributions and transition probabilities between nodes, DNBs can describe the temporal evolution of system states. DNBs can be used to perform a variety of inference tasks, such as predicting future states, updating system states based on observed data, and inferring hidden states. DNBs have broad applications in many fields, including machine learning, artificial intelligence, biomedicine, and finance.

[0081] The music bar pointer sequence includes the best music bar pointers corresponding to all the frame data in the song. The music bar pointer includes the position of the frame data in the music bar and the corresponding frame rhythm.

[0082] In some embodiments, based on a dynamic Bayesian network, a music measure pointer model is established according to an initial probability model, a state transition probability model and an observable model; wherein, the initial probability model is used to initialize the music measure pointer model, the state transition probability model is used to describe the change pattern of the music measure pointer over time, and the observable model is used to describe the relationship between the beat detection result and the music measure pointer; based on the rhythm range, the music measure pointer model is used to decode the beat detection result to obtain a music measure pointer sequence.

[0083] In some embodiments, the music bar pointers correspond to nodes in a dynamic Bayesian network, and the initial probability model, the state transition probability model, and the observable model correspond to edges in the dynamic Bayesian network.

[0084] In some embodiments, the initial probability model is a uniform distribution, or it may be prior knowledge and historical data, etc., which is not limited in this application.

[0085] In some embodiments, the state transition probability model is used to indicate the probability that the music bar pointer of the current frame data transfers to the music bar pointer of the next frame data.

[0086] For example, P(x k |x k-1 ) represents the state transition probability model, which represents the probability that the music bar pointer of the k-1th frame data transfers to the music bar pointer of the kth frame data, where k is a positive integer greater than 1.

[0087] In some embodiments, the observable model represents the conditional probability of the beat probability value of the corresponding frame data given a music bar pointer of the frame data.

[0088] For example, P(yk |x k ) represents the observable model, which represents the music measure pointer x given the k-th frame data k The beat probability value y corresponding to the k-th frame data under the condition k The conditional probability of , k is a positive integer.

[0089] In some embodiments, a dynamic Bayesian network is used to determine a musical measure pointer sequence based on a first constant, the beat detection result, and the tempo range, wherein the first constant is an optimal hyperparameter of the dynamic Bayesian network. In some embodiments, the hyperparameters of the dynamic Bayesian network are adjusted based on the beat detection result, and the hyperparameter that best fits the beat detection result is determined as the first constant.

[0090] In some embodiments, a Viterbi algorithm is used to decode the dynamic Bayesian network in the music measure, and a music measure pointer sequence is decoded from the beat detection result.

[0091] The Viterbi algorithm is a dynamic programming algorithm used to find the Viterbi path—the hidden state sequence—that is most likely to produce an observation time sequence. In this application, the observation time sequence corresponds to the beat detection result, and the Viterbi path corresponds to the musical measure pointer sequence.

[0092] Please refer to the following for the process of establishing a music measure pointer model based on a dynamic Bayesian network.

[0093] Through the above method, a dynamic Bayesian network is used to decode the optimal music measure pointer sequence from the beat detection results, so as to more accurately predict the position of each frame data in the music measure of the song.

[0094] Step 240: Based on the music bar pointer sequence and the rhythm type, obtain the position of the beat in at least one music bar in the music bar.

[0095] In some embodiments, the music section is evenly divided into a plurality of positions. Optionally, the music section can be divided into positions according to the length of the frame data, or according to the length set by relevant personnel, which is not limited in this application.

[0096] Exemplarily, a music measure includes M positions, and a music measure includes M frame data, that is, the music measure is divided into positions according to the length of the frame data, and M is a positive integer.

[0097] In some embodiments, the first frame data of at least one beat in a music measure is determined as the position of the beat.

[0098] In some embodiments, each musical measure includes at least one beat; based on the musical measure pointer sequence, the position of at least one musical measure is determined; based on the rhythm type, the number of beats included in each musical measure is determined; based on the position and number of beats of at least one musical measure, the position of the beats in at least one musical measure in the musical measure is determined.

[0099] For example, based on the position of each frame data in the music measure and the frame rhythm in the song, the starting and ending positions of each music measure can be inferred. Based on the rhythm type of the song, the number of beats included in each music measure can be determined, and the position of the beats in each music measure in the corresponding music measure can be determined.

[0100] In some embodiments, the position of the first frame of each measure of music is determined as the position of the first type of beat.

[0101] For example, assuming that the first type of beat is a strong beat, set D is used to represent the set of strong beats, then the set Among them, k represents the kth frame data in the song, Indicates the optimal position of the k-th frame data in the music measure, It represents the frame data whose position in each music measure of the song is 1. Then, according to the rhythm type of the song, the number of beats in each music measure can be obtained, and then the positions of other beats in each music measure can be obtained. k is a positive integer.

[0102] By using the above method, the position of each music measure and the position of its beat in the music measure can be obtained more accurately.

[0103] In some embodiments, when a music measure includes multiple beats, the first beat in the music measure is a first type of beat, and at least one beat in the music measure other than the first beat is a second type of beat.

[0104] In some embodiments, the first type of beat refers to a strong beat and the second type of beat refers to a weak beat.

[0105] For example, assuming that the rhythm type of the song is 2 / 4, each music measure in the song includes 2 beats, in the order of strong beat and weak beat, and each beat is a half note. The first beat in the music measure is the first type of beat, that is, the strong beat, and the other beat in the music measure except the first beat is the second type of beat, that is, the weak beat.

[0106] For example, assuming that the rhythm type of the song is 3 / 8, each music measure in the song includes 3 beats, in the order of strong beat, weak beat, and weak beat, and each beat is an eighth note. The first beat in the music measure is the first type of beat, that is, the strong beat, and the other two beats in the music measure except the first beat are the second type of beats, that is, the weak beats.

[0107] For example, assuming that the rhythm type of the song is 3 / 4, each music measure in the song includes 3 beats, in the order of strong beat, weak beat, and weak beat, and each beat is a quarter note. The first beat in the music measure is the first type of beat, that is, the strong beat, and the other two beats in the music measure except the first beat are the second type of beats, that is, the weak beats.

[0108] In some embodiments, the first type of beat refers to a strong beat, and the second type of beat includes a secondary strong beat and a weak beat.

[0109] For example, assuming that the rhythm type of the song is 6 / 8, each music measure in the song includes 6 beats, in the order of strong beat, weak beat, weak beat, secondary strong beat, weak beat, and weak beat, and each beat is an eighth note. The first beat in the music measure is the first type of beat, that is, the strong beat, and the other five beats in the music measure except the first beat are the second type of beats, including weak beats and secondary strong beats.

[0110] For example, assuming that the rhythm type of the song is 4 / 4, each music measure in the song contains 4 beats, in the order of strong beat, weak beat, secondary strong beat, and weak beat, and each beat is a quarter note. The first beat in the music measure is the first type of beat, that is, the strong beat, and the other three beats in the music measure except the first beat are the second type of beats, including weak beats and secondary strong beats.

[0111] In some embodiments, there is only one beat in a measure of music and the beat is the strong beat.

[0112] For example, assuming that the rhythm type of the song is 1 / 4 beat, each music measure in the song includes 1 beat, the beat is the strong beat, and the beat is a quarter note.

[0113] Through the above-mentioned method, the position of each beat in a certain music measure is determined based on different rhythm types.

[0114] To sum up, through the above method, the beat detection result of the song is obtained, and the beat detection result includes the beat probability value of each frame data in multiple frame data of the song. According to the speed and rhythm type of the song, the rhythm range of the song is determined. Based on the above beat detection result and rhythm range, the dynamic Bayesian network is used to determine the music bar pointer sequence corresponding to the song. Finally, based on the music bar pointer sequence and the rhythm type of the song, the position of the beat of at least one music bar in the corresponding music bar can be obtained. The beat detection result of the song is analyzed by using a dynamic Bayesian network, and the position of the beat of the music bar in the music bar is deduced from the beat detection result of the song, which effectively reduces the probability of missed beats and multiple beats and improves the accuracy of beat detection.

[0115] The following briefly introduces the process of establishing a music measure pointer model based on a dynamic Bayesian network.

[0116] By using a dynamic Bayesian network to infer a series of hidden variables from a series of observable variables, in this application, the observable variable is the beat detection result, that is, the beat probability value y of each frame data in the song k ={y1,y2,…,y K}, K is the number of frame data of the song, k is a positive integer less than K, and the hidden variable is the music measure pointer Among them, Φ k Indicates the position of the k-th frame data in the music measure, Φ k ∈{1,2,…,M}, M represents the total number of positions of the music section, Indicates the frame rhythm of the k-th frame data, and The frame data is determined by the tempo of the song, and its calculation method is shown in the above method embodiment.

[0117] The goal is to generate a list of K}Calculate the optimal hidden state sequence The conditional probability distribution P(x) between the observable sequence and the hidden state sequence 1:K |y 1:K )maximize:

[0118]

[0119] in, Indicates the optimal music measure pointer from the 1st frame data to the Kth frame data in the song.

[0120] The conditional probability distribution between the observable sequence and the hidden state sequence satisfies the following relationship:

[0121]

[0122] Among them, P(x1) represents the initial probability model, and uniform distribution is used as the distribution of the initial probability model to initialize each hidden state so that the initial probability of each hidden state is equal.

[0123] P(x k |x k-1 ) represents the state transition probability model, indicating that the hidden state changes from x k-1 Transfer to x k The probability of the dynamic Bayesian network describes the evolution of the hidden state over time through the state transition probability. The state transition probability model P(x k |x k-1 ) can be decomposed into the product of the following two conditional probabilities:

[0124]

[0125] in, is an indicator function whose value is 0 or 1:

[0126]

[0127] mod represents the modulo operation, which is used to calculate the remainder after dividing two numbers; The calculation is as follows:

[0128]

[0129] Here, exp represents the natural exponential function, that is, e to the power of x, and λ represents a constant ranging from 1 to 300.

[0130] Among them, P(y k |x k ) represents the observable model, which represents the music measure pointer x given the k-th frame data k The beat probability value y corresponding to the k-th frame data under the condition k The conditional probability of , k is a positive integer.

[0131] In an embodiment of the present application, the tempo of the song is used as a constraint of the dynamic Bayesian network, so that the dynamic Bayesian network decodes the beat detection results within an acceptable speed range to obtain the position of the beats in each music measure in the corresponding music measure.

[0132] A specific embodiment is briefly described below.

[0133] Please refer to Figure 3 , which shows a flowchart of a simple example of a post-processing method for music beat detection provided by an embodiment of the present application.

[0134] First, the song is input into the neural network model used to analyze the song to obtain the beat detection result and the speed of the song corresponding to the above song. Based on the speed and rhythm type of the song, the rhythm range of the song is obtained. After the beat detection result, rhythm range and first constant are given in the dynamic Bayesian network of the music bar pointer model, the music bar pointer sequence is decoded by the bit algorithm. According to the music bar pointer sequence and the rhythm type, the position of the beat in at least one music bar in the song in the corresponding music bar can be derived.

[0135] Assuming that the method of the present application is applied to music creation software, when the user needs to create based on an accompaniment, the user needs to understand the beat position, chord progression, rhythm type, melody fragment and other elements of the accompaniment. If the user only has information about the accompaniment and rhythm type, the music bar pointer model established by this method can be used to extract the beat position of the accompaniment from the accompaniment, and then, based on the extracted beat position, additional rhythm elements such as percussion and drum beat changes can be added according to the style the user wants to create. Lyrics can also be created for the accompaniment based on the extracted beat position.

[0136] The method of the present application can be applied to various scenarios in the field of music. The application ideas are similar to those above, so they will not be repeated here.

[0137] The following are device embodiments of the present application, which can be used to implement the method embodiments of the present application. For details not disclosed in the device embodiments of the present application, please refer to the method embodiments of the present application.

[0138] Please refer to Figure 4 , which shows a block diagram of a post-processing device for music beat detection provided by an embodiment of the present application. The device has the function of implementing the above-mentioned method example, and the function can be implemented by hardware or by hardware executing corresponding software. The device can be the computer device 10 introduced above, or it can be set in the computer device 10. Figure 4 As shown, the apparatus 400 may include a detection result acquisition module 410 , a rhythm range determination module 420 , a pointer sequence determination module 430 and a beat position determination module 440 .

[0139] The detection result acquisition module 410 is used to obtain the beat detection result of the song, wherein the song includes multiple frame data, and the beat detection result includes the beat probability value of each frame data in the multiple frame data, and the beat probability value of the frame data refers to the probability value of the frame data being a beat.

[0140] The rhythm range determination module 420 is used to determine the rhythm range of the song based on the speed and rhythm type of the song, wherein the speed of the song is used to indicate the number of beats of the song in unit time, the rhythm type is used to indicate the number of beats included in a music measure and the regularity of the beats, and the rhythm range includes multiple candidate frame rhythms, and the frame rhythm is used to indicate the speed between adjacent frame data in the song.

[0141] The pointer sequence determination module 430 is used to determine a music measure pointer sequence based on the beat detection result and the rhythm range using a dynamic Bayesian network, where the music measure pointer sequence is used to indicate the position of at least one music measure in the song.

[0142] The beat position determination module 440 is configured to obtain the position of the beats in the at least one music measure in the music measure based on the music measure pointer sequence and the rhythm type.

[0143] In some embodiments, the rhythm range determination module 420 includes a beat number determination submodule, a rhythm determination submodule, and a rhythm range determination submodule (in Figure 4 not shown).

[0144] The number determination submodule is used to determine the number of beats included in each music measure according to the rhythm type.

[0145] a tempo determination submodule, configured to determine a minimum tempo and a maximum tempo according to the number of beats, a tempo adjustment parameter, and the tempo of the song, wherein the tempo adjustment parameter is used to indicate the minimum tempo and the maximum tempo accepted by the song;

[0146] The rhythm range determination submodule is configured to determine the rhythm range based on the minimum rhythm and the maximum rhythm.

[0147] In some embodiments, the rhythm determination submodule is used to subtract a first value from the speed of the song to obtain the minimum speed, and add the first value to the speed of the song to obtain the maximum speed, wherein the first value is the product of the speed of the song and the speed adjustment parameter; determine the minimum rhythm based on the minimum speed and the number of beats, and determine the maximum rhythm based on the maximum speed and the number of beats.

[0148] In some embodiments, each music measure includes at least one beat; the beat position determination module 440 is used to determine the position of the at least one music measure based on the music measure pointer sequence; determine the number of beats included in each music measure based on the rhythm type; and determine the position of the beats in the at least one music measure in the music measure based on the position of the at least one music measure and the number of beats.

[0149] In some embodiments, when the music measure includes multiple beats, the first beat in the music measure is a first type beat, and at least one beat in the music measure other than the first beat is a second type beat.

[0150] In some embodiments, the beat position determination module 440 is used to establish a music measure pointer model based on the dynamic Bayesian network according to the initial probability model, the state transition probability model and the observable model; wherein the initial probability model is used to initialize the music measure pointer model, the state transition probability model is used to describe the change pattern of the music measure pointer over time, and the observable model is used to describe the relationship between the beat detection result and the music measure pointer; based on the rhythm range, the music measure pointer model is used to decode the beat detection result to obtain the music measure pointer sequence.

[0151] To sum up, by obtaining the beat detection result of the song, the beat detection result includes the beat probability value of each frame data in multiple frame data of the song, and determining the rhythm range of the song according to the speed and rhythm type of the song, based on the above beat detection result and rhythm range, a dynamic Bayesian network is used to determine the music measure pointer sequence corresponding to the song, and finally based on the music measure pointer sequence and the rhythm type of the song, the position of the beat of at least one music measure in the corresponding music measure can be obtained, which realizes the analysis of the beat detection result of the song by using a dynamic Bayesian network, and the deduction of the beat position of the music measure in the music measure from the beat detection result of the song, thereby effectively reducing the probability of missed beats and multiple beats, and improving the accuracy of beat detection.

[0152] Please refer to Figure 5 , which shows a block diagram of a computer device 500 provided in one embodiment of the present application. The computer device 500 may be Figure 1 The computer device 10 in the illustrated implementation environment is used to implement the post-processing method for music beat detection provided in the above embodiment. Specifically:

[0153] Typically, the computer device 500 includes a processor 510 and a memory 520 .

[0154] The processor 510 may include one or more processing cores, such as a 5-core processor, an 8-core processor, and the like. The processor 510 may be implemented in at least one hardware form of digital signal processing (DSP), field programmable gate array (FPGA), and programmable logic array (PLA). The processor 510 may also include a main processor and a coprocessor. The main processor is a processor for processing data in an awake state, also known as a central processing unit (CPU); the coprocessor is a low-power processor for processing data in a standby state. In some embodiments, the processor 510 may be integrated with a graphics processing unit (GPU), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 510 may also include an AI processor for processing computing operations related to machine learning.

[0155] The memory 520 may include one or more computer-readable storage media, which may be non-transitory. The memory 520 may also include high-speed random access memory and non-volatile memory, such as one or more magnetic disk storage devices or flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 520 is used to store a computer program, which is configured to be executed by one or more processors to implement the above-mentioned post-processing method for music beat detection.

[0156] Those skilled in the art will understand that Figure 5 The structure shown in the figure does not constitute a limitation on the computer device 500, and the computer device 500 may include more or fewer components than shown in the figure, or combine some components, or adopt a different component arrangement.

[0157] In an exemplary embodiment, a computer-readable storage medium is further provided, wherein a computer program is stored in the storage medium, and when the computer program is executed by the processor, the post-processing method for music beat detection is implemented. Optionally, the computer-readable storage medium may include: a read-only memory (ROM), a random access memory (RAM), a solid-state drive (SSD), or an optical disc. Among them, the random access memory may include a resistance random access memory (ReRAM) and a dynamic random access memory (DRAM).

[0158] In an exemplary embodiment, a computer program product is also provided, comprising a computer program stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium and executes the computer program, causing the computer device to perform the above-described post-processing method for music beat detection.

[0159] It should be understood that the "multiple" mentioned in this article refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent three situations: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the previous and subsequent associated objects are in an "or" relationship. In addition, the step numbers described in this article only illustrate a possible execution sequence between the steps. In some other embodiments, the above steps may not be executed in the order of the numbers, such as two steps with different numbers are executed at the same time, or two steps with different numbers are executed in the opposite order to the diagram. The embodiments of the present application do not limit this.

[0160] The above description is merely an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A post-processing method for music beat detection, characterized in that: The method comprises: Obtaining a beat detection result of a song, wherein the song includes a plurality of frame data, and the beat detection result includes a beat probability value of each frame data in the plurality of frame data, wherein the beat probability value of the frame data refers to a probability value that the frame data is a beat; Determining a rhythm range of the song based on the tempo and rhythm type of the song, wherein the tempo of the song is used to indicate the number of beats in the song per unit time, the rhythm type is used to indicate the number of beats included in a musical measure and the regularity of the beats, the rhythm range includes a plurality of candidate frame rhythms, and the frame rhythm is used to indicate the speed between adjacent frame data in the song; Determining a music measure pointer sequence using a dynamic Bayesian network based on the beat detection result and the rhythm range, wherein the music measure pointer sequence is used to indicate a position of at least one music measure in the song; Based on the music measure pointer sequence and the rhythm type, the position of the beat in the at least one music measure in the music measure is obtained.

2. The method according to claim 1, characterized in that Determining the rhythm range of the song based on the speed and rhythm type of the song includes: Determining the number of beats included in each music measure according to the rhythm type; determining a minimum tempo and a maximum tempo based on the number of beats, a tempo adjustment parameter, and the tempo of the song, wherein the tempo adjustment parameter is used to indicate a minimum tempo and a maximum tempo acceptable to the song; The cadence range is determined based on the minimum cadence and the maximum cadence.

3. The method according to claim 2, characterized in that The determining of the minimum tempo and the maximum tempo according to the number of beats, the tempo adjustment parameter, and the tempo of the song comprises: Subtracting a first value from the speed of the song to obtain the minimum speed, and adding the first value to the speed of the song to obtain the maximum speed, wherein the first value is the product of the speed of the song and the speed adjustment parameter; The minimum tempo is determined based on the minimum speed and the number of beats, and the maximum tempo is determined based on the maximum speed and the number of beats.

4. The method according to claim 1, wherein Each measure of music includes at least one beat; The step of obtaining a position of a beat in the at least one music bar in the music bar based on the music bar pointer sequence and the rhythm type includes: determining a position of the at least one music measure based on the music measure pointer sequence; Based on the rhythm type, determining the number of beats included in each music measure; Based on the position of the at least one music measure and the number of beats, positions of the beats in the at least one music measure in the music measure are determined.

5. The method according to claim 4, characterized in that In the case that the music measure includes multiple beats, the first beat in the music measure is a first type beat, and at least one beat in the music measure other than the first beat is a second type beat.

6. The method according to claim 1, characterized in that Determining a music measure pointer sequence using a dynamic Bayesian network based on the beat detection result and the rhythm range includes: Based on the dynamic Bayesian network, a music measure pointer model is established according to an initial probability model, a state transition probability model, and an observable model; wherein the initial probability model is used to initialize the music measure pointer model, the state transition probability model is used to describe the change pattern of the music measure pointer over time, and the observable model is used to describe the relationship between the beat detection result and the music measure pointer; Based on the rhythm range, the music measure pointer model is used to decode the beat detection result to obtain the music measure pointer sequence.

7. A post-processing device for music beat detection, characterized in that: The device comprises: a detection result acquisition module, configured to acquire a beat detection result of a song, wherein the song includes a plurality of frame data, the beat detection result includes a beat probability value of each frame data in the plurality of frame data, and the beat probability value of the frame data refers to a probability value that the frame data is a beat; a rhythm range determination module, configured to determine a rhythm range of the song based on the tempo and rhythm type of the song, wherein the tempo of the song indicates the number of beats in the song per unit time, the rhythm type indicates the number of beats in a musical measure and the regularity of the beats, the rhythm range includes a plurality of candidate frame rhythms, and the frame rhythm indicates the speed between adjacent frame data in the song; a pointer sequence determination module, configured to determine a music measure pointer sequence using a dynamic Bayesian network based on the beat detection result and the rhythm range, wherein the music measure pointer sequence is used to indicate a position of at least one music measure in the song; The beat position determination module is used to obtain the position of the beat in the at least one music measure in the music measure based on the music measure pointer sequence and the rhythm type.

8. A computer device, characterized in that: The computer device includes a processor and a memory, wherein a computer program is stored in the memory, and the processor executes the computer program to implement the method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that The storage medium stores a computer program, and the computer program is configured to be executed by a processor to implement the method according to any one of claims 1 to 6.

10. A computer program product, characterized in that The computer program product comprises a computer program, which is loaded and executed by a processor to implement the method according to any one of claims 1 to 6.

Images