A method and system for inserting foot pedal data into a playback file

Abstract

This invention relates to the field of automatic foot pedal accompaniment technology, and more particularly to a method and system for inserting foot pedal data into a playback file. The method includes: identifying change nodes in the score file that produce chord changes; selecting at least one change node as a first insertion node; acquiring the foot pedal data corresponding to the first insertion node, and identifying the adjacent basic information of the first insertion node as an index array; retrieving matching event groups in the playback file that match the index array; setting a second insertion node in the playback file according to the position of the matching event group; and inserting the foot pedal data into the second insertion node. This invention can provide users with automatic foot pedal accompaniment, solving the problem of some players not being able to press the pedals during playing, and also meeting the needs of some players who want to use other people's foot pedal data for accompaniment, thereby greatly reducing the difficulty of playing and improving the performance effect.

Description

Technical Field

[0001] This invention relates to the field of automatic foot pedal accompaniment technology, and in particular to a method and system for inserting foot pedal data into a playback file. Background Technology

[0002] The demand for piano learning is currently very strong, covering multiple scenarios such as children's music enlightenment, teenagers' art exam preparation, and adults' interest cultivation, making it a popular choice for the public to improve their artistic literacy. The use of the piano pedals is key to enhancing the performance: the right pedal extends the pitch and enriches resonance, the left pedal softens the tone and reduces volume, and the middle pedal locks in the bass and avoids reverberation. The flexible use of these three pedals can make the melody more layered and emotional. Piano pedals are usually integrated under the piano body, allowing users to extend the tone, adjust reverberation, and create dynamic changes by pressing the pedals.

[0003] However, some pianists (such as children with short legs, elderly or disabled people who cannot exert force with their legs, or beginners who do not know how to use the pedals) may not be able to use the pedals correctly to achieve the desired playing effect. Traditional technologies have also attempted to propose some technical solutions to assist users in driving the pedals.

[0004] For example, patent application CN109036339A discloses a piano pedal auxiliary foot pedal device. This device includes a housing, a support plate disposed on the top plate of the housing and adjacent to and parallel to the back plate of the housing, and three auxiliary pedals hinged to the support plate. Each auxiliary pedal has a pressure rod connected to its free end. An opening in the back plate allows the three piano pedals to extend into the inner cavity of the housing. The three pressure rods pass through the top plate and are positioned directly above the three piano pedals. In other words, the piano pedal auxiliary foot pedal device provided by this patent can transmit the movement of the auxiliary pedals to the piano pedals, enabling children to operate the piano pedals normally even when using an adult piano.

[0005] For example, patent application CN120510757A discloses a piano pedal movement amplitude indicator device for music teaching, including a placement plate and an indicator structure disposed on the placement plate. A suction cup is installed at the bottom of the placement plate. The indicator structure includes an indicator frame, on the front of which, from left to right, are arranged a soft pedal display screen, a mute pedal display screen, and a sustain pedal display screen. The indicator frame also has a reminder element for pedal switching. The soft pedal display screen has a first indicator element for displaying the soft pedal movement. This invention, through the setting of the indicator structure, can display the movement and amplitude of the piano pedals, thereby achieving precise reminders of the piano pedal movements. This allows learners to accurately control the range of piano pedal movements, facilitating piano teaching demonstrations, improving teaching effectiveness, and promoting rapid piano learning.

[0006] However, for the diverse scenarios of piano performance, the above traditional technical solutions still cannot meet users' personalized needs for piano foot pedal accompaniment. Summary of the Invention

[0007] The purpose of this invention is to provide a method for inserting pedal data into a playback file, which partially solves or alleviates the above-mentioned shortcomings in the prior art, and can greatly reduce the difficulty of playing and improve the performance effect.

[0008] To solve the aforementioned technical problems, the present invention specifically adopts the following technical solution:

[0009] A first aspect of the present invention is to provide a method for inserting foot pedal data into a playback file, comprising:

[0010] S100, acquire the playback file of the track, the playback file records: trigger information of at least one trigger event, the trigger information includes: the identifier of the trigger element, the trigger time, and the release time;

[0011] S101, Obtain the sheet music file corresponding to the piece of music. The sheet music file includes: basic information of the symbolic elements. The basic information includes: the category and position of the symbolic elements. The category of the symbolic elements includes: notes, clefs, performance technique markings and / or key signatures.

[0012] S102, Identify the change nodes in the score file that produce chord changes;

[0013] S103, select at least one of the changed nodes as the first insertion node;

[0014] S104, obtain the foot pedal data corresponding to the first insertion node, and identify the neighboring basic information of the first insertion node as an index array, wherein the index array includes basic information of at least one of the semantic elements;

[0015] S105, retrieve matching event groups that match the index array in the playback file;

[0016] S106, Set a second insertion node in the playback file according to the position of the matching event group;

[0017] S107, insert the foot pedal data into the second insertion node to obtain the new playback file.

[0018] In some embodiments, the pedal data includes: pedaling time, pedaling release time, and at least one pedal depth; wherein at least one pedal depth is timestamped.

[0019] In some embodiments, selecting at least one changed node as the first insertion node includes the steps of:

[0020] When a foot marker is set within the associated interval of the changed node, the changed node is used as the first inserted node.

[0021] In some embodiments, the music track includes at least one musical phrase, and selecting at least one variation node as the first insertion node includes the steps of:

[0022] Obtain valid information for at least one of the triggering events within a musical segment, the valid information including: the number of events;

[0023] Generate the note density of the triggering events in the musical segment based on the valid information;

[0024] Select the insertion quantity based on the stated note density;

[0025] Select the specified number of changed nodes from among the multiple changed nodes as the first insertion node.

[0026] In some embodiments, the note density and the set number of insertions are negatively correlated.

[0027] In some embodiments, the effective information further includes: an influence factor, the influence factor including at least one of the following: register position, duration of articulation, and whether it is a root note; correspondingly, the step of generating the note density of the triggering event in the musical segment based on the effective information includes:

[0028] Generate a real count for the triggering event;

[0029] When the valid information includes the impact factor, the corresponding true count is updated to a weighted count based on the impact factor.

[0030] The note density of the musical phrase is generated based on the weighted count and / or the true count.

[0031] In some embodiments, selecting at least one changed node as the first insertion node includes the steps of:

[0032] Identify the style of the track;

[0033] An insertion rule is selected for the track according to the style, and the insertion rule includes: position rule or quantity rule;

[0034] At least one of the changed nodes is selected as the first insertion node according to the insertion rules.

[0035] In some embodiments, the location rules include:

[0036] The staccato rule states that when the pitch difference between two notes before and after the staccato node is greater than a preset pitch level, the staccato node is designated as the first insertion node.

[0037] And / or, a bass rule, wherein the bass rule is that when the pitch of the next lower part after the change node is lower than a preset pitch threshold, the duration of the lower part is higher than a preset duration threshold, and the note density of the upper part is greater than a preset density threshold, then the change node is selected as the first insertion node.

[0038] In some embodiments, obtaining the foot pedal data corresponding to the first insertion node includes:

[0039] The performance preferences for the aforementioned pieces are obtained, including: pedal depth, performer style, or musical style.

[0040] The pedal data is obtained based on the performance preferences.

[0041] A second aspect of the present invention is to provide a system for inserting pedal data into a playback file, comprising:

[0042] The playback file acquisition module is used to acquire the playback file of the track. The playback file records the trigger information of at least one trigger event. The trigger information includes the identifier of the trigger element, the trigger time, and the release time.

[0043] The music score file acquisition module is used to acquire the music score file corresponding to the piece. The music score file includes: basic information of the symbolic elements, the basic information including: the category and position of the symbolic elements, and the category of the symbolic elements includes: notes, clefs, performance technique markings and / or key signatures;

[0044] A change node identification module is used to identify change nodes in the musical score file that produce chord changes;

[0045] An insertion node selection module is used to select at least one of the changed nodes as the first insertion node;

[0046] The index array acquisition module is used to acquire the foot data corresponding to the first insertion node, and to identify the neighboring basic information of the first insertion node as an index array, wherein the index array includes basic information of at least one of the semantic elements;

[0047] An event matching module is used to retrieve matching event groups that match the index array in the playback file;

[0048] An insertion node setting module is used to set a second insertion node in the playback file according to the position of the matching event group;

[0049] The pedal data insertion module is used to insert the pedal data into the second insertion node to obtain the new playback file.

[0050] Beneficial technical effects:

[0051] This invention selects the chord change nodes in the score file as the index (i.e., the first insertion node), then finds the corresponding position in the playback file (i.e., the second insertion node) and inserts the corresponding pedal data. This user-driven (i.e., the final insertion position is determined by the user's playback file) and score alignment (i.e., the relative position of the insertion node is determined by the standard score file) fusion mechanism has at least the following technical effects:

[0052] Firstly, by inserting pedal data at the change nodes based on chord changes, the pedal accompaniment is kept highly consistent with the logic of the music itself, greatly reducing the complexity of pedal data insertion node calculation, and avoiding harmonic chaos caused by misaligned time nodes.

[0053] Secondly, based on the chord change nodes in the score file, the insertion position of the pedal data can be determined more accurately, which helps to synchronize the standard pedal data with the actual performance rhythm. In other words, with the synergistic effect of the dual benchmarks of the score file and the playback file, a positioning basis that is both standardized and personalized can be provided for the insertion of pedal data.

[0054] Meanwhile, this invention proposes a restrictive pedal node selection mechanism. Instead of uniformly selecting all variable nodes as the first insertion node, this invention filters the range of insertion nodes based on the effective note density to avoid negative impacts on the playback file caused by too many selected nodes (such as excessive sustain effects leading to muddy sound effects), while also reducing the difficulty of automatically inserting pedal data.

[0055] From another perspective, restrictively inserting pedal data not only helps reduce the computational power consumption during the pedal data insertion process, but also effectively reduces the operational complexity of data matching and other steps, thereby significantly reducing the overall difficulty of pedal data insertion and improving the stability and feasibility of the pedal data insertion process.

[0056] In addition, the restrictive pedal node selection mechanism proposed in this invention can also determine the insertion rules according to the style of the music (for example, constrain the number of inserted nodes according to the quantity rule, or select the position of the inserted nodes according to the position rule). Therefore, it can largely ensure that the selected inserted nodes meet the specific needs of the music performance and avoid or reduce the problem of style inconsistency caused by adding pedal accompaniment to all the changing nodes. Attached Figure Description

[0057] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. In all the drawings, similar elements or parts are generally identified by similar reference numerals. The elements or parts in the drawings are not necessarily drawn to scale. Obviously, the drawings described below are some embodiments of the present invention, and those skilled in the art can obtain other drawings based on these drawings without any creative effort.

[0058] Figure 1 A comparative example diagram showing the insertion of pedal data into a MusicXML file, as provided by this invention;

[0059] Figure 2 A schematic diagram illustrating the retrieval of matching event groups that match the index array provided by the present invention;

[0060] Figure 3 This is an example diagram illustrating the determination of the first insertion node based on the changing node provided by the present invention.

[0061] Figure 4 A flowchart illustrating the foot pedal accompaniment insertion method provided by the present invention;

[0062] Figure 5 This is a flowchart illustrating a method for inserting foot pedal data into a playback file according to the present invention.

[0063] Figure 6 A schematic diagram of a system for inserting foot pedal data into a playback file, provided by the present invention;

[0064] Figure 7 This is a schematic block diagram of the structure of a computer device provided by the present invention.

[0065] Summary of attached labeling and identification:

[0066] 1. First inserted node; 2. Second inserted node. Detailed Implementation

[0067] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0068] In this document, suffixes such as "module," "part," or "unit" used to denote elements are used only for the purpose of illustrative purposes and have no specific meaning in themselves. Therefore, "module," "part," or "unit" may be used interchangeably.

[0069] In this document, the terms "upper," "lower," "inner," "outer," "front," "rear," "one end," and "the other end," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing the present invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the present invention. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0070] In this document, unless otherwise explicitly specified and limited, the terms "installed," "equipped with," "connected," etc., should be interpreted broadly. For example, "connection" can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection, a direct connection, or an indirect connection through an intermediate medium; it can be a connection within two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0071] In this document, "and / or" includes any and all combinations of one or more of the listed related items.

[0072] In this article, "multiple" means two or more, that is, it includes two, three, four, five, etc.

[0073] As used in this specification, the term "about" typically means + / -5% of the value, more typically + / -4% of the value, more typically + / -3% of the value, more typically + / -2% of the value, even more typically + / -1% of the value, and even more typically + / -0.5% of the value.

[0074] In this specification, certain embodiments may be disclosed in a range-bound format. It should be understood that this "range-bound" description is merely for convenience and brevity and should not be construed as a rigid limitation on the disclosed range. Therefore, the description of a range should be considered as having specifically disclosed all possible subranges and the individual numerical values ​​within those ranges. For example, a description of the range 1-6 should be considered as having specifically disclosed subranges such as from 1 to 3, from 1 to 4, from 1 to 5, from 2 to 4, from 2 to 6, from 3 to 6, etc., and the individual numbers within those ranges, such as 1, 2, 3, 4, 5, and 6. This rule applies regardless of the breadth of the range.

[0075] Definition of the noun:

[0076] MusicXML is a standardized file format for digital music score exchange and distribution, providing a universal representation of musical notation.

[0077] Example 1:

[0078] In some embodiments, users may have recording needs (such as recording to generate playback files for entertainment, practice, etc.), but some users may not be able to control the foot pedals well enough to achieve the desired performance effect. In this case, foot pedal data can be inserted into the MIDI file to improve the performance effect of the user's playback file.

[0079] The following will provide an exemplary description of how the present invention generates playback files with foot pedal data:

[0080] In some embodiments, see Figure 5 This invention proposes a method for inserting foot pedal data into a playback file, comprising:

[0081] S100, acquire the playback file of the track, the playback file records: trigger information of at least one trigger event, the trigger information includes: the identifier of the trigger element, the trigger time, and the release time;

[0082] In some embodiments, the pedal can refer to the pedal of a piano or the pedal of a modern harp.

[0083] Specifically, the triggering element may be a piano key, or the triggering element may be a piano string.

[0084] It should be understood that the foot pedal in this invention can refer to the foot pedal of an instrument that intervenes in the duration, loudness, or texture of sound through mechanical / electronic means, as long as the foot pedal function is similar. This invention does not limit the type of instrument.

[0085] S101, Obtain the sheet music file corresponding to the piece of music. The sheet music file includes: basic information of the symbolic elements. The basic information includes: the category and position of the symbolic elements. The category of the symbolic elements includes: notes, clefs, performance technique markings and / or key signatures.

[0086] S102, Identify the change nodes in the score file that produce chord changes;

[0087] S103, select at least one of the changed nodes as the first insertion node;

[0088] S104, obtain the foot pedal data corresponding to the first insertion node, and identify the neighboring basic information of the first insertion node as an index array, wherein the index array includes basic information of at least one of the semantic elements;

[0089] S105, retrieve matching event groups that match the index array in the playback file;

[0090] S106, Set a second insertion node in the playback file according to the position of the matching event group;

[0091] S107, insert the foot pedal data into the second insertion node to obtain the new playback file.

[0092] In some embodiments, the playback file may be a music instruction file, such as a MIDI file, that records performance data generated by a user playing the piano.

[0093] In some embodiments, a MIDI file (Musical Instrument Digital Interface) can be a type of instruction-based music file. It does not directly store sound waveform data, but records digital instructions related to performance, such as pitch, duration, dynamics, timbre selection, pedal control, and tempo. These instructions can be recognized by electronic musical instruments, music production software, or automatic playing devices and converted into corresponding sounds. It is characterized by its small size and flexible editing capabilities, and is widely used in the music field.

[0094] In some embodiments, the playback file can be used for direct playback or for repetition (i.e., driving the piano to reproduce the performance process, such as automatically playing according to the playback file).

[0095] For example, the automatic piano playing method described in patent application CN120472871A can be used to drive the piano to repeat the playback file.

[0096] Among them, sensors (such as speed sensors, displacement sensors or acceleration sensors, which can detect the speed, displacement or acceleration of the corresponding piano key being triggered in real time) and / or image acquisition modules (such as cameras) can be used to identify the trigger information of the trigger event (such as key touch).

[0097] In some embodiments, the sheet music file may be a digital sheet music that records standard performance information, such as MusicXML (Music Extensible Markup Language).

[0098] In some embodiments, ideographic elements refer to musical elements that make up a musical score file, which may include notes (such as half notes, quarter notes, etc.), rests (such as half rests, quarter rests, etc.), performance technique markings (such as trills, arpeggios, etc.), dynamic markings (such as crescendos, diminuendos, etc.), clefs, key signatures (such as sharps, flats, etc.), or structural markings (such as repeats, leaps, etc.). It should be understood that ideographic elements are symbols or graphics that have specific musical meanings in the musical score.

[0099] In some embodiments, the position of an ideographic element is used to define the position of the ideographic element in the score file (e.g., the position of the note in which section).

[0100] For example, the position of a symbolic element can refer to its position on a musical line in the score. The position can be a height position (such as the height position of a note on a musical line, which can be used to indicate pitch, such as a line or space in the musical line).

[0101] Alternatively, position can refer to the order in which the symbolic elements appear in the score file, such as on a particular beat of a measure.

[0102] In other words, the position of a symbolic element can refer to its position on the spectrum.

[0103] In this embodiment, please refer to Figure 2 This invention selects the chord change nodes in the score file as the index (i.e., the first insertion node), then finds the corresponding position in the playback file (i.e., the second insertion node) and inserts the corresponding pedal data. This user-driven (i.e., the final insertion position is determined by the user's playback file) and score alignment (i.e., the relative position of the insertion node is determined by the standard score file) fusion mechanism has at least the following technical effects:

[0104] Firstly, by inserting pedal data at the change nodes based on chord changes, the pedal accompaniment is kept highly consistent with the logic of the music itself, greatly reducing the complexity of pedal data insertion node calculation, and avoiding harmonic chaos caused by misaligned time nodes.

[0105] Secondly, based on the chord change nodes in the score file, the insertion position of the pedal data can be determined more accurately, which helps to synchronize the standard pedal data with the actual performance rhythm. In other words, with the synergistic effect of the dual benchmarks of the score file and the playback file, a positioning basis that is both standardized and personalized can be provided for the insertion of pedal data.

[0106] In some embodiments, pedal data may refer to a data sequence that records information such as pedaling time / speed, pedal release time / speed, and pedal depth.

[0107] In some embodiments, a pedal database may be pre-set, which records different types of pedal data. For example, it may include: full pedal data of slow pedaling and fast release, half full pedal data of slow pedaling and fast release, full pedal data of fast pedaling and fast release, light pedaling data of fast pedaling and fast release, full pedal data of slow pedaling and slow release, etc.

[0108] In some embodiments, pedal data can be selected from the pedal database according to the performance requirements of the current piece.

[0109] For example, if the current piece is a prelude, then you can choose clearer, more restrained (e.g., light pedaling) pedal data based on the performance requirements of this genre.

[0110] Alternatively, users can select the foot pedal data.

[0111] For example, users can choose the artistic effect they want to achieve (such as style), and further, the foot pedal data corresponding to the style can be determined based on the user's choice.

[0112] In some embodiments, a chord corresponds to at least one pedal action; corresponding pedal actions (i.e., recommended pedal data) can be recommended based on the type of chord. For example, for a major triad, a slow, fast full pedal data can be recommended; for a dominant seventh chord, a fast, fast half pedal data can be recommended.

[0113] Alternatively, in some embodiments, the pedal data can be selected based on the type between the preceding and following chords. For example, if the preceding and following chords share a common note, the harmonic transition is natural, and there is no need to adjust the pedal depth; simply keeping the pedal fully depressed to maximize resonance is sufficient. For instance, the pedal depth for the preceding and following chords can be the same, requiring only a switching action (i.e., releasing the pedal and pressing it down again) when switching.

[0114] For example, in some embodiments, the two chords share no or very few common notes. For instance, if the first chord is a C major triad, its pedal parameters are: slow pressing and fast releasing (full pedal). If the second chord is an Eb major triad, its pedal parameters are set to: fast pressing and fast releasing (half pedal). The switching action between the two is to only half-lift the pedal to switch to the next chord.

[0115] It is understood that this invention allows users to manually adjust the selection rules for pedal data, such as adding or deleting data.

[0116] Alternatively, it could recommend different types of pedal data when the chord changes.

[0117] In some embodiments, please refer to Table 1, chord types can be preset to identify chord types in the score file and the transition nodes between different types of chords.

[0118] Table 1:

[0119]

[0120] For example, please see Figure 1 The chord progressions are as follows: measure 12 is "FCFAC", measure 13 is "CGCEG", measure 14 is "DADFA", measure 15 is "AEACECA", and measure 16 is "FCFCEBGB". Based on the preset chord types in Table 1, measure 12 belongs to the 4th basic group, measure 13 to the 1st basic group, measure 14 to the 2nd basic group, measure 15 to the 6th basic group, and measure 16 is a combination chord of the 4th and 3rd basic groups.

[0121] Furthermore, the position where the chord changes (i.e., the change node) can be determined as the first insertion node.

[0122] For example, please see Figure 3 The "measure start time" on the left side of the score corresponds to key points of chord changes (e.g., measure 7 starts at 24:00; measure 8 starts at 28:00; measure 9 starts at 32:00). These time points are the points of change for chord switching. At the time points of chord changes (e.g., 24:00, 28:00, 32:00), perform the "up" action (i.e., release) of the pedal; immediately following the time point (e.g., 24:01, 28:01, 32:01), perform the "down" action (i.e., press down) of the pedal.

[0123] For example, the original MusicXML or MIDI file may not contain pedal information. Pedal information is added at chord changes. For instance, in measures 12, 13, 14, 15, and 16, each measure has a different chord, so each measure begins with the pedal pressed and ends with the pedal released. Measure 16 contains chord changes, so it also shows a situation where the pedal is released before being pressed again.

[0124] It should be understood that, taking the sustain pedal as an example, its mechanism is to lift the damper, allowing the strings to continue vibrating, thereby producing reverberation and resonance. In other words, this invention uses the chord switching point as the first insertion node, meaning that before entering a new chord, the pedal can be released to cut off the residual sound of the previous chord, and then, according to the needs of the new chord, the pedal can be pressed down again to form a new sound.

[0125] In some embodiments, S102 can also be used to obtain the pedal data corresponding to the first insertion node directly from the records in the score file (such as which measure and which beat the pedal should be stepped on).

[0126] For example, the first insertion node can be determined based on the foot pedal markings in the score file or user annotations (such as user-annotated notes).

[0127] In some embodiments, the pedal data corresponding to the first insertion node can also be generated based on chord changes. For example, if the analysis finds that the new chord has a long bass note, the system will generate a set of pedal data that is deeply pressed and maintained for a long time; if it is a light staccato passage, it will generate pedal data that is quickly tapped.

[0128] In some embodiments, the neighboring basic information of the first insertion node may refer to the basic information of at least one set of ideographic elements that make up at least one chord before and after the first insertion node.

[0129] For example, the first insertion node is used to define the position where pedal data needs to be inserted in the playback file, and its adjacent basic information can be the basic information of at least two or at least three ideographic elements (such as musical notes) preceding the first insertion node.

[0130] Specifically, an indexed array can refer to at least one semantic element, such as "FAC".

[0131] In some embodiments, matching may mean that the sequence of semantic elements of the index array and the matching event group are completely identical; or it may mean that the similarity between the sequence of semantic elements of the index array and the matching event group is greater than a preset similarity (such as 80%).

[0132] For example, similarity can be determined based on the percentage overlap between the index array and the sequence of semantic elements of the matching event group.

[0133] For example, if the index array contains 5 notes (such as "GBDFA"), and the matching event group contains 4 notes in the exact same order as the index array (such as "CBDFA"), then the overlap percentage is 80%, and the index array can be considered to match the matching event group.

[0134] In some embodiments, an audio-to-score matching method described in CN120508674A can be used to quickly locate and match the index array (i.e., score) and the matching event group (i.e., audio).

[0135] In some embodiments, inserting the pedal data into the second insertion node to obtain a new playback file may include: using the second insertion node as a reference, writing at least one piece of pedal data into the corresponding position of the original playback file (i.e., the second insertion node); thereafter, integrating the original playback file and the pedal data to regenerate a new playback file.

[0136] In some embodiments, a set of pedal data includes: pedaling time, pedaling release time, and at least one pedal depth.

[0137] In some embodiments, the pedal pressing time preferably corresponds to the chord start time, and the pedal releasing time preferably corresponds to the chord end time.

[0138] In other words, the time window between pedaling and releasing should preferably cover at least one complete chord.

[0139] In some embodiments, pedal depth may include multiple pedal depths within a time window between pedaling time and pedal release time.

[0140] In some embodiments, pedal curves can be plotted based on multiple sets of pedal data. For example, pedal curves can be plotted according to a performance data visualization method described in patent application CN120833765A to form an intuitive mapping between pedals and time, thereby improving the user experience.

[0141] In some embodiments, at least one of the pedal depths is timestamped. For example, in this embodiment, a pedal curve may be inserted into the playback file, which records the changes in pedal depth at multiple time points.

[0142] In some embodiments, selecting at least one changed node as the first insertion node includes the steps of:

[0143] When a foot marker is set within the associated interval of the changed node, the changed node is used as the first inserted node.

[0144] In some embodiments, the associated region of a pedal mark refers to the range of target notes covered in time by the pedal event corresponding to the pedal mark. For example, a pedal mark may be used to mark a measure, in which case the associated region is the measure in which the pedal mark is located. Alternatively, a pedal mark may be used to mark a musical phrase, in which case the associated region of the pedal mark is the musical phrase in which the pedal mark is located.

[0145] In some embodiments, the associated area corresponding to a foot pedal mark can be predetermined by the user (e.g., a composer). In some embodiments, the foot pedal mark includes at least one of a depressed mark, a released mark, or a sustain mark. Taking the sustain pedal as an example, Figure 1 In this context, “ped” means “step down”, “*” means “release”, and “—” means “maintain”.

[0146] It should be understood that the present invention preferably determines the change node corresponding to the associated interval with the pedal mark as the first insertion node, which can ensure to a certain extent that this part of the pedal data is inserted into the corresponding position according to the composer's creative intention, and meets the hard constraints of the pedal mark.

[0147] In other words, the present invention preferably first quickly and unambiguously identifies the "must-process" change nodes corresponding to these foot pedal marks, and then judges the "to-process" change nodes that do not have clear foot pedal marks and require further reasoning, thereby improving the efficiency and reliability of the overall algorithm.

[0148] In some embodiments, the music track includes at least one musical phrase, and selecting at least one variation node as the first insertion node includes the steps of:

[0149] Obtain valid information for at least one of the triggering events within a musical segment, the valid information including: the number of events;

[0150] Generate the note density of the triggering events in the musical segment based on the valid information;

[0151] Select the insertion quantity based on the stated note density;

[0152] Select the specified number of changed nodes from among the multiple changed nodes as the first insertion node.

[0153] In some embodiments, the note density and the set insertion number are negatively correlated. For example, the higher the note density of a musical phrase, the lower the set insertion number, and correspondingly, the fewer the number of phrases identified as the first insertion node.

[0154] In some embodiments, when a musical phrase (such as a measure) has a large number of notes, i.e., a high note density, the use of the foot pedal can be reduced or avoided to allow for relatively clear intervals between different sounds / voices, thus making the main melody stand out. Conversely, if the note density is low, the use of the foot pedal can be increased or retained without worrying that the sustain pedal will make the different voices too closely connected, making them difficult to distinguish.

[0155] In some embodiments, a recommended number of insertions can be provided to the user based on the note density, such as recommending a range of 5-8 insertions. The user can then select their desired number of insertions from the recommended range.

[0156] In some embodiments, the effective information further includes: an influence factor, the influence factor including at least one of the following: register position, duration of articulation, and whether it is a root note; correspondingly, the step of generating the note density of the triggering event in the musical segment based on the effective information includes:

[0157] Generate a real count for the triggering event;

[0158] When the valid information includes the impact factor, the corresponding true count is updated to a weighted count based on the impact factor.

[0159] The note density of the musical phrase is generated based on the weighted count and / or the true count.

[0160] In some embodiments, the note density may not be an absolute number, but may be a weighted number.

[0161] In some embodiments, the vocal register may include a high register, a low register, a mid register, etc.

[0162] In particular, the present invention preferably assigns higher weight to low-frequency notes, that is, when low-frequency notes are played, the use of the pedal is reduced more carefully to avoid the continuous bass sound masking the melody and harmony in the mid and high-frequency range, thus compromising the clarity of the music in terms of listening experience.

[0163] In some embodiments, for notes of different durations (such as long notes, medium-long notes, and short notes), the present invention preferably assigns higher weight to long notes, that is, it more carefully reduces the use of the pedal when a note is sung for a long duration, so as to avoid the sustain effect affecting the listening experience.

[0164] The root note of a chord is the lowest note in the chord, also known as the fundamental note.

[0165] In some embodiments, the present invention preferably assigns higher weight to the root note of the chord, that is, the use of the pedal is reduced more carefully when the root note of the chord is sounded, which to a certain extent ensures that the chord is clearly distinguishable and avoids the root note from mixing with notes that do not belong to the same chord in terms of sound effect, resulting in blurred harmonic properties and affecting the listening experience.

[0166] For example, a musical phrase may have 4 notes (such as note A, note B, note C, and note D), and its actual count is 4.

[0167] For example, taking the position of the pitch as an influence factor, note A is in the high pitch range and is counted as 0.8, note B is in the high pitch range and is counted as 0.8, note C is in the low pitch range and is counted as 1.2, and note D is in the middle pitch range and is counted as 1. In this case, the weighted count of the four notes A, B, C, and D is 3.8.

[0168] For example, taking the duration of pronunciation as an influencing factor, note A is a long tone and is counted as 1.3, note B is a short tone and is counted as 0.7, note C is a short tone and is counted as 0.7, and note D is a medium-long tone and is counted as 1.1. At this time, the weighted count of the four notes A, B, C and D is 3.8.

[0169] For example, if we take whether a note is the root note as an influencing factor (e.g., root note, not root note), note A is the root note and is counted as 1.5, note B is not the root note and is counted as 0.8, note C is not the root note and is counted as 0.8, and note D is the root note and is counted as 1.5. In this case, the weighted count of the four notes A, B, C, and D is 4.6.

[0170] In some embodiments, different weighted counts can be pre-set for different influence factors.

[0171] In some embodiments, when the effective information includes the influence factor, the present invention preferably updates the corresponding true count to a weighted count based on the influence factor. That is, when deciding whether to use a change node as the first insertion node, the present invention considers not only "how many notes are played", but also identifies the notes that need to be focused on in combination with the context of the score, and appropriately increases their "weight" (i.e., weighted count) to calculate a note density that is more in line with the real scene. This helps to ensure sound clarity with fewer pedals (i.e., less sustain effect) when the note density is low, and helps to ensure smoothness with more pedals when the note density is high.

[0172] In other words, this invention proposes a restrictive foot pedal node selection mechanism. Instead of uniformly selecting all variable nodes as the first insertion node, this invention determines the note density that better matches the actual performance situation through the context information of the score itself, and further determines whether to select the variable node as the first insertion node, so as to avoid the negative impact of foot pedal accompaniment on the playback file due to too many node selections (such as excessive sustain effects causing muddy sound effects).

[0173] It should be understood that the restrictive pedal node selection mechanism proposed in this invention helps to balance computing resources and artistic expression, enabling high-quality pedal accompaniment insertion even under limited computing power. This allows high-performance computing tasks to run efficiently and stably, greatly reducing the processing difficulty of computers and also reducing the difficulty of automatic pedal accompaniment insertion to some extent.

[0174] In some embodiments, selecting at least one changed node as the first insertion node may include the steps of:

[0175] Identify the style of the track;

[0176] An insertion rule is selected for the track according to the style, and the insertion rule includes: position rule or quantity rule;

[0177] At least one of the changed nodes is selected as the first insertion node according to the insertion rules.

[0178] In some embodiments, the style of a piece (such as romantic, passionate, melancholic, etc.) can be identified based on the title of the score (such as "Sonata", "Prelude", "Etude", etc.), tonality (such as C major, A minor, etc.), playing techniques (such as vibrato, arpeggio, etc.), or tempo.

[0179] For example, in terms of playing technique, dense arpeggios may indicate that the style of the piece is romantic.

[0180] In some embodiments, the weighted count or true count in a musical phrase can be directly used as the note density.

[0181] In some embodiments, if the style of the actual track corresponds to both positional rules and quantity rules, either positional rules or quantity rules can be selected as the insertion rule.

[0182] Alternatively, in some embodiments, if the style of the actual track corresponds to both positional rules and quantity rules, the rules can be applied according to the priority set by the user for positional rules and quantity rules respectively.

[0183] For example, if a user pre-sets the priority of the quantity rule to 1 and the priority of the location rule to 2, with the location rule having a higher priority than the quantity rule, then the location rule can be applied first.

[0184] In some embodiments, different musical styles have different requirements for the use of pedals. The quantity rule refers to the constraints imposed by the musical style on the number of pedals used, i.e., the limit on the number of pedal data points allowed to be inserted in that musical style. For example, the Baroque style typically requires fewer pedal data points (e.g., only allowing 3 first insertion points out of 12 variations) to ensure the separation and clarity of each voice as much as possible; conversely, the Romantic style typically requires more pedal data points (e.g., allowing 8 first insertion points out of 12 variations) to create a rich harmonic atmosphere and abundant sonic resonance.

[0185] In some embodiments, the position rule refers to the restriction of the position of the node where the pedal data is inserted by the style of the track. That is, when the changing node satisfies the position rule, the present invention preferably uses the changing node as the first insertion node.

[0186] Taking staccato rules as an example, if there is a large leap of more than an octave in a chord, such as jumping from low C to high G, you can insert pedal data between the two notes. By increasing the pedal connection, you can improve the smoothness of the performance and help avoid the disjointed sound caused by the inability to connect the fingerings.

[0187] Taking the bass rule as an example, if there is a bass note that lasts for a long time (e.g., the duration is higher than the preset duration threshold) in a musical passage (e.g., the pitch of the lower voice is lower than the preset pitch threshold), and there are many high notes (e.g., the note density of the upper voice is greater than the preset density threshold), then the corresponding change node can be used as the first insertion node (e.g., inserting a faster, shallower pedal data) to maintain the continuity of the bass note while providing basic resonance for the dense high notes.

[0188] In some embodiments, the restrictive pedal node selection mechanism proposed in this invention can also determine the insertion rules according to the style of the piece (e.g., constrain the number of inserted nodes according to quantity rules, or select the position of the inserted nodes according to position rules). Therefore, it can largely ensure that the selected inserted nodes meet the specific needs of the piece performance, and avoid or reduce the problem of style inconsistency caused by adding pedal accompaniment to all the changing nodes.

[0189] It should be understood that this invention uses effective note density and insertion rules to restrict the insertion of pedal data from two levels: the number of nodes and the position of nodes (such as setting the number of insertions), thereby imposing progressive restrictions on the insertion of automatic pedal accompaniment, reducing the difficulty of inserting automatic pedal accompaniment, and improving the basic compatibility between pedal accompaniment and playback files.

[0190] In some embodiments, obtaining the foot pedal data corresponding to the first insertion node includes:

[0191] The performance preferences for the aforementioned pieces are obtained, including: pedal depth, performer style, or musical style.

[0192] The pedal data is obtained based on the performance preferences.

[0193] In some embodiments, performance preferences are used to define a user’s preference for different types of pedal data, such as pedal depth.

[0194] Among them, pedal depth can be used to define a user's preference for the depth to which they press the pedals. For example, some users (such as beginners) have weaker pedal control and generally press the pedals too shallowly; or, they may not fully understand the function of the pedals, and therefore may often experience sudden pressing or stopping.

[0195] Among them, "performer style" is used to define a user's preference for a particular performer's pedaling style. For example, a database of performer pedaling styles can be created by analyzing a large number of playback files from different performers; then, a user's performer style can be determined based on the pedaling styles of performers they frequently select.

[0196] Among them, the musical style is used to define the user's preference for different styles of music. For example, the romantic style usually requires pressing the pedal deeper to use resonance to create a full and lyrical tone.

[0197] In some embodiments, pedal data can be selected in the pedal data according to any of the performance preferences.

[0198] In some embodiments, the present invention also proposes a foot-pedal accompaniment insertion system for performing... Figure 4 The system includes the following steps for inserting foot pedal accompaniment:

[0199] The chord category classification information module is used to classify and organize information on various types of chords (such as major triads, minor triads, seventh chords, etc.) and clarify the type affiliation of different chords.

[0200] The chord-related note information extraction and classification numbering module is used to extract the constituent note information corresponding to each chord (such as the specific pitches contained in a certain chord), and organize and assign numbers to these note information to facilitate the subsequent identification and management of the note content of the chord.

[0201] The chord variation comparison module is used to compare chord differences in different scenarios, such as changes in the composition of chords in the same melody passage, and adjustments to chords in different versions, to analyze chord transformations and evolution.

[0202] The pedal release and pedal press description injection module is used to associate the operation description information of "pedal release" and "pedal press" in piano performance with the corresponding chord performance content, and supplement the relevant annotation information of pedal operation and chord coordination.

[0203] In some embodiments, the above-described foot-operated accompaniment insertion system further includes a CPU (Central Processing Unit) module for executing algorithm software; a DDR memory module (Double Data Rate SDRAM) for running programs; and a FLASH (Adobe Flash) storage module for storing the original MusicXML file and the modified MusicXML file.

[0204] In some embodiments, the foot pedal mentioned in this invention mainly refers to the sustain pedal.

[0205] In other embodiments, similar strategies and methods can be used to generate corresponding MusicXML files for the soft pedal, which will not be elaborated upon in this invention.

[0206] In some embodiments, see Figure 6 The present invention also proposes a system for inserting foot pedal data into a playback file, comprising:

[0207] The playback file acquisition module is used to acquire the playback file of the track. The playback file records the trigger information of at least one trigger event. The trigger information includes the identifier of the trigger element, the trigger time, and the release time.

[0208] The music score file acquisition module is used to acquire the music score file corresponding to the piece. The music score file includes: basic information of the symbolic elements, the basic information including: the category and position of the symbolic elements, and the category of the symbolic elements includes: notes, clefs, performance technique markings and / or key signatures;

[0209] A change node identification module is used to identify change nodes in the musical score file that produce chord changes;

[0210] An insertion node selection module is used to select at least one of the changed nodes as the first insertion node;

[0211] The index array acquisition module is used to acquire the foot data corresponding to the first insertion node, and to identify the neighboring basic information of the first insertion node as an index array, wherein the index array includes basic information of at least one of the semantic elements;

[0212] An event matching module is used to retrieve matching event groups that match the index array in the playback file;

[0213] An insertion node setting module is used to set a second insertion node in the playback file according to the position of the matching event group;

[0214] The pedal data insertion module is used to insert the pedal data into the second insertion node to obtain the new playback file.

[0215] It should be understood that the system proposed in this invention can be used to implement the steps described in any embodiment of this invention.

[0216] In some embodiments, the applicant notes that MusicXML typically does not contain foot pedal information.

[0217] To address this, the present invention can analyze the content of MusicXML, automatically add pedal data (such as descriptions of pedal release / press) to the MusicXML file, and generate a new MusicXML file with pedal data, such as... Figure 1 .

[0218] For example, generating a sheet music file with pedal data may include the following steps:

[0219] Obtain the sheet music file for the track;

[0220] Identify the changing nodes in the score file that produce chord changes, and use at least one changing node as the first insertion node;

[0221] Insert the pedal data into the first insertion node to generate a score file with pedal data.

[0222] It should be understood that the MusicXML file with pedal data generated by this invention can not only be displayed as sheet music, but also drive the piano keys and pedals to play in sync, providing users with automatic pedal accompaniment; at the same time, with the help of the pedal information in the MusicXML file, users can also see more clearly whether their pedals are pressed correctly.

[0223] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0224] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a computer terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0225] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims. All of these forms are within the protection scope of the present invention.

[0226] In some embodiments, this application also provides a schematic block diagram of the structure of a computer device, please see... Figure 7 Computer programs can be used in situations such as Figure 7 It runs on the computer device shown. Figure 7 As shown, the computer device includes a processor, memory, and a network interface connected via a system bus. The memory may include a non-volatile storage medium and internal memory. The non-volatile storage medium may store an operating system and a computer program. The computer program includes program instructions that, when executed, cause the processor to perform the methods described in any embodiment of the present invention. The processor provides computing and control capabilities to support the operation of the entire computer device. The internal memory provides an environment for the execution of the computer program in the non-volatile storage medium. When the computer program is executed by the processor, it causes the processor to perform the methods described in any embodiment of the present invention. The network interface is used for network communication, such as sending assigned tasks. Those skilled in the art will understand that... Figure 7The structures shown are merely block diagrams of a portion of the structure related to the present application and do not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than shown in the figures, or combine certain components, or have different component arrangements. It should be understood that the processor may be a Central Processing Unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

Claims

1. A method of inserting foot pedal data in a play file, characterized by, include: S100, acquire the playback file of the track, the playback file records: trigger information of at least one trigger event, the trigger information includes: the identifier of the trigger element, the trigger time, and the release time; S101, Obtain the sheet music file corresponding to the piece of music. The sheet music file includes: basic information of the symbolic elements. The basic information includes: the category and position of the symbolic elements. The category of the symbolic elements includes: notes, clefs, performance technique markings and / or key signatures. The position of the symbolic elements is used to define the position of the symbolic elements in the sheet music file. S102, Identify the change nodes in the score file that produce chord changes; S103, select at least one of the changed nodes as the first insertion node; S104, obtain the foot pedal data corresponding to the first insertion node, and identify the neighboring basic information of the first insertion node as an index array, wherein the index array includes basic information of at least one of the semantic elements; S105, retrieve matching event groups that match the index array in the playback file; S106, Set a second insertion node in the playback file according to the position of the matching event group; S107, Insert the foot pedal data into the second insertion node to obtain the new playback file; The musical piece includes at least one musical section, and step S103 includes the following steps: Obtain valid information for at least one of the triggering events within a musical segment, the valid information including: the number of events; Generate the note density of the triggering events in the musical segment based on the valid information; Select the insertion quantity based on the stated note density; Select the specified number of changed nodes from among the multiple changed nodes as the first insertion node.

2. The method of claim 1, wherein, The pedal data includes: pedaling time, pedal releasing time, and at least one pedal depth; wherein at least one pedal depth is timestamped.

3. The method according to claim 1, characterized in that, Selecting at least one changed node as the first insertion node includes the following steps: When a foot marker is set within the associated interval of the changed node, the changed node is used as the first inserted node.

4. The method according to claim 1, characterized in that, The note density and the set number of insertions are negatively correlated.

5. The method according to claim 1, characterized in that, The effective information further includes: an influence factor, which includes at least one of the following: register position, duration of articulation, and whether it is a root note; correspondingly, the step of generating the note density of the triggering event in the musical segment based on the effective information includes: Generate a real count for the triggering event; When the valid information includes the impact factor, the corresponding true count is updated to a weighted count based on the impact factor. The note density of the musical phrase is generated based on the weighted count and / or the true count.

6. The method according to claim 1, characterized in that, Selecting at least one changed node as the first insertion node includes the following steps: Identify the style of the track; An insertion rule is selected for the track according to the style, and the insertion rule includes: position rule or quantity rule; At least one of the changed nodes is selected as the first insertion node according to the insertion rules.

7. The method according to claim 6, characterized in that, The location rules include: The staccato rule states that when the pitch difference between two notes before and after the staccato node is greater than a preset pitch level, the staccato node is designated as the first insertion node. And / or, a bass rule, wherein the bass rule is that when the pitch of the next lower part after the change node is lower than a preset pitch threshold, the duration of the lower part is higher than a preset duration threshold, and the note density of the upper part is greater than a preset density threshold, then the change node is selected as the first insertion node.

8. The method according to claim 1, characterized in that, Obtaining the foot pedal data corresponding to the first inserted node includes: The performance preferences for the aforementioned pieces are obtained, including: pedal depth, performer style, or musical style. The pedal data is obtained based on the performance preferences.

9. A system for inserting foot pedal data into a playback file, characterized in that, include: The playback file acquisition module is used to acquire the playback file of the track. The playback file records the trigger information of at least one trigger event. The trigger information includes the identifier of the trigger element, the trigger time, and the release time. The music score file acquisition module is used to acquire the music score file corresponding to the piece of music. The music score file includes: basic information of the symbolic elements, the basic information including: the category and position of the symbolic elements, and the category of the symbolic elements includes: notes, clefs, performance technique markings and / or key signatures; wherein, the position of the symbolic elements is used to define the position of the symbolic elements in the music score file; A change node identification module is used to identify change nodes in the musical score file that produce chord changes; An insertion node selection module is used to select at least one of the changed nodes as the first insertion node; The index array acquisition module is used to acquire the foot data corresponding to the first insertion node, and to identify the neighboring basic information of the first insertion node as an index array, wherein the index array includes basic information of at least one of the semantic elements; An event matching module is used to retrieve matching event groups that match the index array in the playback file; An insertion node setting module is used to set a second insertion node in the playback file according to the position of the matching event group; A pedal data insertion module is used to insert the pedal data into the second insertion node to obtain the new playback file; The music track includes at least one musical phrase; the insertion node selection module is used for: Obtain valid information for at least one of the triggering events within a musical segment, the valid information including: the number of events; Generate the note density of the triggering events in the musical segment based on the valid information; Select the insertion quantity based on the stated note density; Select the specified number of changed nodes from among the multiple changed nodes as the first insertion node.

Images