Piano roller shutter score note labeling method

By constructing the dynamic behavior of piano pedals and dividing different pedal behaviors, the problem of inaccurate notation in piano automatic music notation in the existing technology is solved, more accurate notation is achieved, and the performance of automatic music notation is improved .

CN118918870APending Publication Date: 2024-11-08BEIJING UNIV OF TECH
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202410983674.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-22
Publication Date
2024-11-08

AI Technical Summary

Technical Problem

The existing piano automatic music notation method is difficult to accurately mark the notes duration when dealing with the dynamic behavior of the piano sustain pedal and the selection of sustain pedal, resulting in a large difference between the audio signal and the note duration annotation, which affects the automatic music notation. performance.

Method used

By obtaining the digital instrument interface information during piano performance, we construct a dynamic behavior representation of the piano sustain pedal and the selection of sustain pedals, divide different pedal behaviors, and combine the key information to mark the notes in the piano roller blind score, improving the notes' time mark Methods, consider the effect of the dynamic behavior of the pedal on the note duration.

Benefits of technology

This method more accurately describes the playing process of the piano pedal, effectively distinguishes and represents different pedal behaviors, improves the existing pedal single threshold note duration method, obtains note markings that are more consistent with the actual vocalization process of piano notes, and reduces the The difference between audio signals and note duration labels improves the performance of automatic music notation.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118918870A_ABST
    Figure CN118918870A_ABST
Patent Text Reader

Abstract

The invention discloses a piano roller shutter score note labeling method and belongs to an automatic music score recording technology. The note labeling method aims at obtaining the piano note roller shutter score label which is more matched with the actual piano note sounding process. In the playing process of a modern piano, a player can modify the length and the tone of a played note by means of the piano pedal, and the expression form of piano music is enriched. However, according to an existing piano automatic notation method, only a pedal single threshold value division method is adopted to describe the tenuto effect of a piano pedal. The invention designs a piano pedal dynamic behavior representation method based on the tenuto pedal widely used in modern piano playing and selection of the tenuto pedal, and tenuto the note length influencing the half-pedal playing technology based on the representation method. And the piano note roller shutter score label which better conforms to the actual sounding process of the piano notes is obtained.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention is based on the piano automatic music notation technology and studies a piano roll score note marking method. The digital instrument interface information of the piano keys, piano sustain pedal and piano select sustain pedal is obtained during piano performance, and the dynamic behavior representation of the piano sustain pedal and the piano select sustain pedal is constructed. Different pedal behaviors are divided according to the pedal dynamic behavior representation, and notes are marked in the piano roll score according to the divided pedal behaviors and the piano key digital instrument interface information. The present invention belongs to the field of computer hearing, and specifically relates to music information retrieval, automatic music notation and other technologies. Background Art

[0002] Automatic Music Transcription (AMT) is one of the basic technologies in the field of music information retrieval, which aims to convert music audio signals into musical notation, such as simple notation, five-line notation, piano roll notation, and digital interface for musical instruments. At present, the cost of manual music notation is high, and it can only be done by trained music professionals, and it is necessary to listen repeatedly to confirm the content of the recorded notes. The notation process is complicated, mechanical and repetitive, and requires a lot of time and energy. The emergence of automatic music notation will greatly simplify and improve this process. In addition, with the help of automatic music notation technology, MIR tasks such as multi-pitch estimation, note starting point detection, beat detection and melody tracking can be achieved, which can provide support for application directions such as music theme classification, automatic music generation, music education, music-assisted creation, and musicology research.

[0003] Conducting targeted research on the piano as a typical instrument for polyphonic music notation is a key breakthrough in solving various difficulties in this technology. In the process of playing modern pianos, performers can use piano pedals to modify the duration and timbre of the notes played, enriching the expression of piano music. Among all piano pedals, the piano sustain pedal is the most frequently used. The piano sustain pedal can extend the duration of the free vibration of the strings of the hammer, which has a direct impact on the duration of the notes. The existing piano automatic notation method has taken into account the effect of the piano sustain pedal on the duration of the notes: when the piano sustain pedal value is less than a certain threshold, it is determined that the sustain effect of the pedal ends. In addition to the sustain pedal, the piano's selector sustain pedal also has the effect of extending the notes. Slightly different from the sustain pedal, the selector sustain pedal only extends the notes that have been played when the pedal is pressed. When using these piano pedals, the duration of the notes will be affected differently. To this end, the present invention takes into account the pedal playing techniques of the above-mentioned two piano pedals, and proposes a method for marking notes with reference to the dynamic behavior of the piano pedal, so as to reduce the difference between the piano audio signal and the note duration marking, thereby further improving the automatic music notation performance of the neural network in supervised learning. Summary of the invention

[0004] Based on digital musical instrument interface information, the present invention proposes a note duration marking method for piano notes represented by piano roll score, which takes into account the influence of different piano pedal behaviors on note duration. The method is composed of the following steps:

[0005] (1) Constructing a representation of the dynamic behavior of the piano pedal

[0006] The state value curve of the piano pedal is obtained from the digital musical instrument interface information recorded during piano performance, and the speed value curve of the piano pedal at each moment during the performance is calculated based on the state value curve.

[0007] (2) Classification of different pedal behaviors according to pedal dynamic behavior representation

[0008] The pedal behavior is divided according to the state value and speed value of the piano pedal at each moment.

[0009] (3) Marking notes in the piano roll score according to the divided pedal behavior and piano key digital instrument interface information

[0010] The results are divided according to the pedal behavior, and the notes are judged whether they are sustained by the piano pedal, and the notes are marked in the piano roll.

[0011] A piano roll score note marking method, characterized by comprising:

[0012] Acquire digital musical instrument interface information of piano keys, piano sustain pedal and piano select sustain pedal during piano performance, construct dynamic behavior representations of piano sustain pedal and piano select sustain pedal, divide different pedal behaviors according to the pedal dynamic behavior representations, and mark notes in the piano roll score according to the divided pedal behaviors and piano key digital musical instrument interface information;

[0013] The dynamic behavior representation method adopts the same construction process on the piano sustain pedal and the piano selective sustain pedal;

[0014] The dynamic behavior representation method is constructed according to the digital instrument interface information of the piano sustain pedal and the piano select sustain pedal obtained. In the digital instrument interface information file containing complete performance information, the piano pedal behavior is recorded by independent control parameters, and the process mapping from the release of the pedal to the full depression of the pedal is recorded as 128 integers of [0,127]. The specific piano pedal dynamic behavior representation construction process includes: the piano pedal state value curve y frame by frame p(t) can be directly obtained from the piano pedal information recorded in the original digital musical instrument interface information file in time; the state value curve changes continuously with the time frame dimension, and the time frame at each moment here corresponds to each time frame obtained after the piano audio signal is pre-processed by signal framing, and the time frame at each moment here also corresponds to each time frame in the piano roll score note annotation; specify θ p =64 represents the pedal threshold, R p =2 represents the pedal speed detection range; point A(t1,y1), B(t2,y2) is y p (t), these points to y p =θ p The projection is denoted as A¢(t1,θ p ),B¢(t2,θ p ); specifies the velocity value v at point A of the piano pedal A for:

[0015]

[0016] Where t2-t1=4 frames; by the same logic, we can get curve y p (t) The pedal speed value curve v at any point in any frame p (t); So far, the dynamic behavior of the piano pedal time value information is respectively calculated by the state value y of the piano pedal obtained above p (t), and the velocity value v p (t) indicates.

[0017] According to the pedal dynamic behavior representation, different pedal behaviors are divided into:

[0018] The pedal behavior classification conditions include three categories in total. The first category is when the pedal state value is greater than the sum of the pedal threshold and the pedal speed detection range:

[0019] y p (t)>θ p +R p (2)

[0020] When this condition is met, the pedal behavior is classified as pedal effective; the second category is when the pedal state value is less than the pedal threshold:

[0021] y p (t)<θ p (3)

[0022] When this condition is met, the pedal behavior is classified as invalid pedal; the third category is when the pedal state value is greater than or equal to the pedal threshold, and the pedal state value is less than or equal to the sum of the pedal threshold and the pedal speed detection range:

[0023]

[0024] When this condition is met, the pedal behavior is classified as pending pedal validity, and in the pending pedal validity category, it is monitored along the increasing direction of the time frame whether there is a situation where the pedal speed value of the current frame is greater than the pedal speed value of the frame after the current frame for the first time:

[0025]

[0026] If this condition is met, all pending pedal valid time frames located in the direction of increasing time frames pointing to the current frame will be classified as invalid pedals, and the remaining pending pedal valid time frames will be classified as valid pedals; if this condition is not met, all pending pedal valid time frames will be classified as valid pedals; therefore, for each time frame, the pedal behavior classification results include two categories, namely, valid pedals and invalid pedals.

[0027] Marking notes in the piano roll score according to the divided pedal behavior and piano key digital musical instrument interface information includes: according to the two types of pedal behavior division results, for each piano key note played in the pedal valid result time frame, ignoring the note end time frame recorded in the digital musical instrument interface information, extending the note end time frame of the note along the time frame increasing direction to the time frame where the pedal invalid result first appears, and marking each extended note in the piano roll score.

[0028] Compared with the prior art, the present invention has the following obvious advantages and beneficial effects:

[0029] (1) Taking into account the dynamic behavior of the piano pedal, the piano pedal playing process is described more accurately. (2) Based on the dynamic behavior of the piano pedal, different piano pedal behaviors are effectively distinguished and represented. (3) Based on the dynamic behavior of the piano pedal and the distinction between different half-pedal playing techniques, the existing pedal single-threshold note duration sustain method is improved, and a piano roll score note annotation that is more in line with the actual sounding process of the piano notes is obtained. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 Schematic diagram of piano pedal status value and piano pedal velocity value calculation DETAILED DESCRIPTION

[0031] According to the above description, the following is a specific implementation process, but the scope of protection of this patent is not limited to this implementation process.

[0032] Step 1: Get the MIDI file of the piano performance information

[0033] Get the complete MIDI file of the piano performance with the help of a piano instrument that automatically records the complete performance information, such as Yamaha's DISKLAVIER series of acoustic intelligent pianos. In the MIDI file containing the complete performance information, the piano pedal behavior is recorded by an independent control parameter, and the process from the release of the pedal to the full depression is mapped to 128 integers in [0,127].

[0034] Step 2: Set up the calculation process for the dynamic behavior of the piano pedal

[0035] Based on the dynamic representation method of the pedal duration information of the piano pedal control parameter MIDI value, the present invention takes into account the sustain pedal and the note duration sustain effect of the selected sustain pedal. In order to obtain a detailed representation of the dynamic behavior of the piano pedal, it is necessary to calculate the state value and velocity value of the piano pedal frame by frame. First, as shown in the attached figure Figure 1 As shown, the piano pedal state value curve y frame by frame p (t) can be directly obtained from the piano pedal information recorded in the original MIDI file in time. The state value curve changes continuously with the time frame dimension. Here, the time frame of each moment corresponds to each time frame obtained after the piano audio signal is pre-processed by signal framing. Here, the time frame of each moment also corresponds to each time frame in the piano roll score. Figure 1 In the specification θ p =64 represents the pedal threshold, R p =2 represents the pedal speed detection range, where the pedal threshold value follows the middle value of the total pedal state value commonly used in the past, that is, 64. The pedal speed detection range represents the focus of the half-pedal technology detection. Point A (t1, y1), B (t2, y2) is y p (t), these points to y p =θ p The projection is denoted as A¢(t1,θ p ),B¢(t2,θ p ). The velocity value v when the piano pedal is at point A is specified A for:

[0036]

[0037] Where t2-t1=4 frames. By the same logic, we can get the curve y p (t) The pedal speed value curve v at any point in any frame p So far, the dynamic behavior of the piano pedal duration information can be obtained by the state value y of the piano pedal obtained by the above calculation. p (t), and the velocity value v p (t) indicates.

[0038] Step 3: Divide the different piano pedal playing behaviors

[0039] Half-pedal Technique is the most widely used piano pedal technique: by fine-tuning the depth of stepping on the piano pedal, the performer can control the degree of contact between the damper in the piano action and the strings, thereby changing the duration of the played notes. In previous pedal single-threshold note duration sustaining methods, the regulating effect of half-pedal technology on note length was ignored. The present invention takes into account that half-pedal technology can gradually weaken the vibration degree of piano strings, showing a weakened musical performance effect. According to the dynamic behavior representation of the piano pedal, different pedal behaviors can be divided. The specific division conditions include a total of three categories. The first category is when the pedal state value is greater than the sum of the pedal threshold and the pedal speed detection range:

[0040] y p (t)>θ p +R p (7)

[0041] When this condition is met, the pedal behavior is classified as pedal effective; the second category is when the pedal state value is less than the pedal threshold:

[0042] y p (t)<θ p (8)

[0043] When this condition is met, the pedal behavior is classified as invalid pedal; the third category is when the pedal state value is greater than or equal to the pedal threshold, and the pedal state value is less than or equal to the sum of the pedal threshold and the pedal speed detection range:

[0044]

[0045] When this condition is met, the pedal behavior is classified as pending pedal validity, and in the pending pedal validity category, it is monitored along the increasing direction of the time frame whether there is a situation where the pedal speed value of the current frame is greater than the pedal speed value of the frame after the current frame for the first time:

[0046]

[0047] If the condition is met, all pending pedal effective time frames in the direction of increasing time frames pointed by the current frame are classified as pedal invalid, and the remaining pending pedal effective time frames are classified as pedal effective; if the condition is not met, all pending pedal effective time frames are classified as pedal effective. Therefore, for each time frame, the pedal behavior classification results include two categories, namely pedal effective and pedal invalid.

[0048] Step 4: Mark the notes in the piano roll according to the results of the piano pedal behavior

[0049] According to the three-category pedal behavior division result described in claim 3, for each piano key note played in the pedal valid result time frame, ignore the note end time frame recorded in the digital musical instrument interface information, extend the note end time frame of the note along the time frame increasing direction to the time frame where the pedal invalid result first appears, and mark each extended note in the piano roll score.

[0050] Regarding the above advantages and beneficial effects:

[0051] (1) “More accurately describes the playing process of the piano pedal” means that the existing pedal single threshold method abstracts the time-varying values ​​of the pedal recorded in the MIDI file into a binary representation of {0,1}, while the “dynamic behavior of the piano pedal” representation method proposed in the present invention retains the original time-varying values ​​of the pedal as much as possible. In terms of representing the time-varying values ​​of the pedal, the proposed invention is more accurate.

[0052] (2) “Effectively distinguish and represent different piano pedal behaviors”, that is, several types of pedal behaviors designed for the present invention.

[0053] (3) "Piano roll notation that is more consistent with the actual sounding process of piano notes". This effect is demonstrated using the following steps: first, the piano roll notation of the same piece of music is obtained using the existing method and the proposed method respectively; then, the synthetic audio corresponding to the above two annotations is synthesized using the same piano timbre file and audio synthesis method; finally, the mean absolute error and mean square error between all time domain audio sampling points of the synthetic audio and the original real audio are compared. The results show that in the two types of synthetic audio, the synthetic audio corresponding to the proposed method has a smaller mean absolute error and mean square error, which proves the effect.

Claims

1. A piano roll score note marking method, It is characterized in that include: Acquire digital musical instrument interface information of piano keys, piano sustain pedal and piano select sustain pedal during piano performance, construct dynamic behavior representations of piano sustain pedal and piano select sustain pedal, divide different pedal behaviors according to the pedal dynamic behavior representations, and mark notes in the piano roll score according to the divided pedal behaviors and piano key digital musical instrument interface information; The dynamic behavior representation method adopts the same construction process on the piano sustain pedal and the piano selective sustain pedal; The dynamic behavior representation method is constructed according to the digital instrument interface information of the piano sustain pedal and the piano select sustain pedal obtained. In the digital instrument interface information file containing complete performance information, the piano pedal behavior is recorded by independent control parameters, and the process mapping from the release of the pedal to the full depression of the pedal is recorded as 128 integers of [0,127]. The specific piano pedal dynamic behavior representation construction process includes: the piano pedal state value curve y frame by frame p (t) can be directly obtained from the piano pedal information recorded in the original digital musical instrument interface information file in time; the state value curve changes continuously with the time frame dimension, and the time frame at each moment here corresponds to each time frame obtained after the piano audio signal is pre-processed by signal framing, and the time frame at each moment here also corresponds to each time frame in the piano roll score note annotation; specify θ p =64 represents the pedal threshold, R p =2 indicates the pedal speed detection range; point A (t 1 ,y 1 ),B(t 2 ,y 2 ) is y p (t), these points to y p =θ p The projection is denoted by A¢(t 1 ,θ p ),B¢(t 2 ,θ p ); specifies the velocity value v at point A of the piano pedal A for: where t 2 -t 1 = 4 frames; by extending the same logic, we can get the curve y p (t) The pedal speed value curve v at any point in any frame p (t); So far, the dynamic behavior of the piano pedal time value information is respectively calculated by the state value y of the piano pedal obtained above p (t), and the velocity value v p (t) indicates.

2. The marking method according to claim 1, It is characterized in that According to the pedal dynamic behavior representation, different pedal behaviors are divided into: The pedal behavior classification conditions include three categories in total. The first category is when the pedal state value is greater than the sum of the pedal threshold and the pedal speed detection range: y p (t)>θ p +R p (2) When this condition is met, the pedal behavior is classified as pedal effective; the second category is when the pedal state value is less than the pedal threshold: y p (t)<θ p (3) When this condition is met, the pedal behavior is classified as invalid pedal; the third category is when the pedal state value is greater than or equal to the pedal threshold, and the pedal state value is less than or equal to the sum of the pedal threshold and the pedal speed detection range: When this condition is met, the pedal behavior is classified as pending pedal validity, and in the pending pedal validity category, it is monitored along the increasing direction of the time frame whether there is a situation where the pedal speed value of the current frame is greater than the pedal speed value of the frame after the current frame for the first time: If this condition is met, all pending pedal valid time frames located in the direction of increasing time frames pointing to the current frame will be classified as invalid pedals, and the remaining pending pedal valid time frames will be classified as valid pedals; if this condition is not met, all pending pedal valid time frames will be classified as valid pedals; therefore, for each time frame, the pedal behavior classification results include two categories, namely, valid pedals and invalid pedals.

3. The method according to claim 2, It is characterized in that The notes in the piano roll are marked according to the divided pedal behavior and piano key digital instrument interface information, including: According to the results of the two types of pedal behavior division, for each piano key note played in the pedal valid result time frame, ignore the note end time frame recorded in the digital musical instrument interface information, and extend the note end time frame of the note along the direction of increasing time frame to the time frame where the pedal invalid result first appears, and mark each extended note in the piano roll score.

Citation Information

Cited By

  • Pedal control method and system

    CN120783610A

  • Method and system for inserting pedal data in playing file

    CN121640970A