Method and system for inserting pedal data in playing file

Abstract

The invention relates to the technical field of automatic pedal accompaniment, in particular to a method and system for inserting pedal data in a playing file. The method comprises the following steps: identifying change nodes generating chord changes in a music score file; selecting at least one change node as a first insertion node; acquiring pedal data corresponding to the first insertion node, and identifying adjacent basic information of the first insertion node as an index array; searching a matching event group matched with the index array in the playing file; setting a second insertion node in the playing file according to the position of the matching event group; and inserting the pedal data into the second insertion node. Automatic pedal accompaniment can be provided for a user, the problem that part of players cannot step on pedals in the playing process is solved, meanwhile, the requirement that part of players want to quote pedal data of other people for accompaniment can be met, and therefore the playing difficulty is greatly reduced, and the playing effect is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of automatic foot accompaniment, and particularly relates to a method and system for inserting foot pedal data in a playing file. BACKGROUND

[0002] At present, the demand for piano learning is very strong, covering multiple scenarios such as children's music enlightenment, teenagers' art examination and education, and adult interest cultivation, and is a popular choice for the public to improve their artistic literacy. The use of piano pedals (pedals) is the key to improving the performance effect: the right pedal prolongs the note value and enriches the resonance, the left pedal softens the tone and reduces the volume, and the middle pedal locks the bass and avoids reverberation. Flexible use of the three can make the melody more layered and emotional. The piano pedal is usually integrated under the piano body, and the user can step on the pedal to achieve tone extension, reverberation and dynamic change.

[0003] However, for some piano players (such as children with insufficient leg length, the elderly or disabled who cannot exert force with their legs, or beginners who cannot step on the pedal), they may not be able to correctly use the pedal to achieve the desired performance effect. Some technical solutions for assisting users to drive the pedal have been proposed in traditional technologies.

[0004] For example, patent application No. CN109036339A proposes a piano pedal auxiliary foot pedal device, which includes a shell, a support plate arranged on the top plate of the shell, adjacent to the back plate of the shell and parallel to the back plate of the shell, a first auxiliary pedal, a second auxiliary pedal and a third auxiliary pedal respectively hinged to the support plate; the free end of each auxiliary pedal is connected to a pressure rod; the back plate is provided with an opening for the three pedals of the piano to extend into the inner cavity of the shell, and the three pressure rods pass through the top plate and are located directly above the three pedals of the piano. That is, the piano pedal auxiliary foot pedal device provided by the patent can transmit the movement of the auxiliary pedal to the piano pedal, so that children can normally operate the piano pedal when using an adult's piano.

[0005] For example, the patent application with publication number CN120510757A proposes a piano pedal action amplitude indicating device for music teaching, which includes a placement plate and an indicating structure arranged on the placement plate. The bottom of the placement plate is installed with a suction cup. The indicating structure includes an indicating frame. The front surface of the indicating frame is sequentially provided with a soft pedal display screen, a mute pedal display screen and a sustain pedal display screen from left to right. A reminding member for reminding pedal switching is arranged on the indicating frame. The soft pedal display screen is provided with a first indicating member for displaying soft pedal action. That is, through the arrangement of the indicating structure, the action and amplitude of the piano pedal can be displayed, thereby realizing accurate reminding of the piano pedal action, facilitating accurate control of the action amplitude of the piano pedal by the practice personnel, thereby facilitating the teaching and display of the piano, improving the teaching effect of the piano, and being beneficial to rapid learning of the piano.

[0006] However, the above-mentioned traditional technical solutions still cannot meet the personalized needs of users for piano foot pedal accompaniment in diversified scenarios of piano performance. SUMMARY

[0007] The purpose of the present application is to provide a method for inserting pedal data in a playing file, which partially solves or alleviates the above-mentioned deficiencies in the prior art, can greatly reduce the difficulty of performance, and improve the performance effect.

[0008] In order to solve the above-mentioned technical problems, the present application specifically adopts the following technical solutions: The first aspect of the present application is to provide a method for inserting pedal data in a playing file, comprising: S100, obtaining a playing file of a piece, the playing file recording: trigger information of at least one trigger event, the trigger information including: identification of a trigger element, trigger time, release time; S101, obtaining a score file corresponding to the piece, the score file including: basic information of a semantic element, the basic information including: category and position of the semantic element, and the category of the semantic element including: note, staff line, performance technique mark and / or key signature; S102, identifying a change node generating a chord change in the score file; S103, selecting at least one of the change nodes as a first insertion node; S104, obtaining pedal data corresponding to the first insertion node, identifying adjacent basic information of the first insertion node as an index array, the index array including basic information of at least one of the semantic elements; S105, searching for a matching event group matching the index array in the playing file; S106, setting a second insertion node in the play file according to the position of the matched event group; S107, inserting the foot pedal data into the second insertion node to obtain a new play file.

[0009] In some embodiments, the foot pedal data comprises: foot pedal pressing time, foot pedal releasing time, at least one foot pedal depth; wherein at least one foot pedal depth is marked with a timestamp.

[0010] In some embodiments, selecting at least one variation node as the first insertion node comprises the steps of: When the variation node is provided with a foot pedal mark within the associated interval, the variation node is selected as the first insertion node.

[0011] In some embodiments, the piece of music comprises at least one section, and selecting at least one variation node as the first insertion node comprises the steps of: Obtaining effective information of at least one trigger event in a section, the effective information comprising: event quantity; Generating note density of the trigger event in the section according to the effective information; Selecting a set insertion number according to the note density; Selecting the variation node of the set insertion number from a plurality of variation nodes as the first insertion node.

[0012] In some embodiments, the note density and the set insertion number are negatively correlated.

[0013] In some embodiments, the effective information further comprises: an impact factor, the impact factor comprising at least one of: note position, sound duration, and whether it is a root note; correspondingly, the step of generating note density of the trigger event in the section according to the effective information comprises: Generating real count for the trigger event; When the effective information comprises the impact factor, updating the corresponding real count to a weighted count according to the impact factor; Generating note density of the section according to the weighted count and / or the real count.

[0014] In some embodiments, selecting at least one variation node as the first insertion node comprises the steps of: Identifying the style of the piece of music; Selecting an insertion rule for the piece of music according to the style, the insertion rule comprising: position rule or quantity rule; Selecting at least one variation node as the first insertion node according to the insertion rule.

[0015] In some embodiments, the position rule comprises: a skip note rule, the skip note rule being that when a pitch difference between two notes before and after the change node is greater than a preset pitch degree, the change node is selected as the first insertion node; and / or, a bass rule, the bass rule being that when a pitch of a next bass part after the change node is lower than a preset pitch threshold, a time value of the bass part is higher than a preset time value threshold, and a note density of a treble part is greater than a preset density threshold, the change node is selected as the first insertion node.

[0016] In some embodiments, obtaining the pedal data corresponding to the first insertion node comprises: obtaining a performance preference of the piece, the performance preference comprising: a pedal depth, a performer style, or a style style; obtaining the pedal data according to the performance preference.

[0017] In a second aspect, the present application provides a system for inserting pedal data in a playing file, comprising: a playing file obtaining module, configured to obtain a playing file of a piece, the playing file recording trigger information of at least one trigger event, the trigger information comprising: an identification of a trigger element, a trigger time, and a release time; a score file obtaining module, configured to obtain a score file corresponding to the piece, the score file comprising: basic information of a semantic element, the basic information comprising: a category of the semantic element, and a position of the semantic element, and the category of the semantic element comprising: a note, a staff line, a performance technique mark, and / or a key signature; a change node identifying module, configured to identify a change node producing a chord change in the score file; an insertion node selecting module, configured to select at least one of the change nodes as a first insertion node; an index array obtaining module, configured to obtain pedal data corresponding to the first insertion node, and identify adjacent basic information of the first insertion node as an index array, the index array comprising basic information of at least one of the semantic elements; an event matching module, configured to search for a matching event group matching the index array in the playing file; an insertion node setting module, configured to set a second insertion node in the playing file according to a position of the matching event group; a pedal data inserting module, configured to insert the pedal data into the second insertion node to obtain a new playing file.

[0018] Beneficial technical effects: The present application selects the chord change node according to the score file as the index (i.e. the first insertion node), then finds the corresponding position in the play file (i.e. the second insertion node) and inserts the corresponding pedal data. This user-driven (i.e. the user's play file determines the final insertion position) and score alignment (i.e. the relative position of the insertion node is determined according to the standard score file) fusion mechanism has at least the following technical effects: Firstly, based on the chord change, the pedal data is inserted at the change node, which makes the pedal accompaniment highly consistent with the logic of the music itself, greatly reduces the complexity of the calculation of the pedal data insertion node, and avoids the dislocation of the time node leading to the confusion of the harmony.

[0019] Secondly, according to the change node of the chord change in the score file, the insertion position of the pedal data can be accurately determined, which helps to synchronize the standard pedal data with the actual performance rhythm, or in other words, with the help of the dual reference of the score file and the play file, the positioning basis with standardization and individualization can be provided for the pedal data insertion.

[0020] At the same time, the present application proposes a restrictive pedal node selection mechanism, i.e. the present application does not blindly select all change nodes as the first insertion node, but filters the number range of the insertion node from the dimension of effective note density, so as to avoid the negative impact of the pedal accompaniment on the play file caused by too many nodes (such as too many sustain effects leading to muddy sound), and reduce the difficulty of automatic insertion of pedal data.

[0021] From another angle, by limiting the insertion of pedal data, not only the computing power consumption in the pedal data insertion process can be reduced, but also the operation complexity in the data matching process can be effectively reduced, thereby significantly reducing the overall difficulty of pedal data insertion and improving the stability and feasibility of the pedal data insertion process.

[0022] In addition, the restrictive pedal node selection mechanism proposed by the present application can also determine the insertion rule according to the style of the piece (for example, the number of insertion nodes is constrained according to the number rule, or the position of the insertion node is selected according to the position rule), so that the selected insertion node can meet the specific needs of the piece performance to a large extent, avoiding or reducing the problem of inconsistent style caused by adding pedal accompaniment to all change nodes. BRIEF DESCRIPTION OF DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. In all the drawings, similar elements or parts are generally identified by similar reference signs. In the drawings, the elements or parts are not necessarily drawn according to the actual proportions. Obviously, the drawings described below are some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0024] Figure 1 A comparison example diagram provided by the present application after inserting pedal data into a MusicXML file; Figure 2 A schematic diagram of the present application for retrieving a matching event group matching the index array; Figure 3 An example diagram provided by the present application for determining a first insertion node according to a change node; Figure 4 A flowchart of the pedal accompaniment insertion method provided by the present application; Figure 5 A flowchart of a method for inserting pedal data in a playing file provided by the present application; Figure 6 A structural schematic diagram of a system for inserting pedal data in a playing file provided by the present application; Figure 7 A structural schematic diagram of a computer device provided by the present application.

[0025] Summary of the reference signs: 1, first insertion node; 2, second insertion node. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical solutions and advantages of the embodiments of the present application more clear, the following will combine the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are some embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0027] In this paper, the suffix such as "module", "component" or "unit" used to represent elements is only for the convenience of the description of the present application, and has no specific meaning. Therefore, "module", "component" or "unit" can be mixedly used.

[0028] In this document, the terms "upper", "lower", "inner", "outer", "front", "back", "one end", "the other end" and the like indicate relative positions or orientation as shown in the drawings and are used only for convenience in describing the present application and simplifying the description, and do not indicate or imply that the referred device or element must have a particular orientation, be constructed and operated in a particular orientation, and therefore should not be construed as limiting the present application. In addition, the terms "first", "second" are only for descriptive purposes and should not be construed as indicating or implying relative importance.

[0029] In this document, unless otherwise explicitly specified and limited, the terms "mount", "provided with", "connected" and the like should be understood broadly, for example, "connected" can be fixedly connected, can be detachably connected, or integrally connected; can be mechanically connected, can be directly connected, can be indirectly connected through an intermediate medium, can be internal communication of two elements. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.

[0030] In this document, "and / or" includes any and all combinations of one or more listed related items.

[0031] In this document, "a plurality of" means two or more, that is, it includes two, three, four, five, etc.

[0032] In this specification, the term "about" typically means + / - 5% of the stated value, more typically + / - 4% of the stated value, more typically + / - 3% of the stated value, more typically + / - 2% of the stated value, even more typically + / - 1% of the stated value, even more typically + / - 0.5% of the stated value.

[0033] In this specification, certain embodiments can be disclosed in a format that is a range. It is to be understood that such a "range" format is merely used for convenience and brevity and should be interpreted in the context of the description unless otherwise indicated. Therefore, the description of a range should be considered to have specifically disclosed all possible sub-ranges as well as individual numerical values within that range. For example, a description of a range 1-6 should be considered to have specifically disclosed the sub-ranges like 1-3, 1-4, 1-5, 2-4, 2-6, 3-6, etc., as well as individual numbers within that range, for example, 1, 2, 3, 4, 5, and 6. The above rules apply regardless of the breadth of the range.

[0034] Glossary: MusicXML is a standardized file format for digital music score exchange and distribution, which can provide a common representation for music notation.

[0035] Example one: In some embodiments, users may have recording needs (such as recording to generate playback files for entertainment, practice, etc.), but some users may not be able to control the foot pedals well enough to achieve the desired performance effect. In this case, foot pedal data can be inserted into the MIDI file to improve the performance effect of the user's playback file.

[0036] The following will provide an exemplary description of how the present invention generates playback files with foot pedal data: In some embodiments, see Figure 5 This invention proposes a method for inserting foot pedal data into a playback file, comprising: S100, acquire the playback file of the track, the playback file records: trigger information of at least one trigger event, the trigger information includes: the identifier of the trigger element, the trigger time, and the release time; In some embodiments, the pedal can refer to the pedal of a piano or the pedal of a modern harp.

[0037] Specifically, the triggering element may be a piano key, or the triggering element may be a piano string.

[0038] It should be understood that the foot pedal in this invention can refer to the foot pedal of an instrument that intervenes in the duration, loudness, or texture of sound through mechanical / electronic means, as long as the foot pedal function is similar. This invention does not limit the type of instrument.

[0039] S101, Obtain the sheet music file corresponding to the piece of music. The sheet music file includes: basic information of the symbolic elements. The basic information includes: the category and position of the symbolic elements. The category of the symbolic elements includes: notes, clefs, performance technique markings and / or key signatures. S102, Identify the change nodes in the score file that produce chord changes; S103, select at least one of the changed nodes as the first insertion node; S104, obtain the foot pedal data corresponding to the first insertion node, and identify the neighboring basic information of the first insertion node as an index array, wherein the index array includes basic information of at least one of the semantic elements; S105, retrieve matching event groups that match the index array in the playback file; S106, Set a second insertion node in the playback file according to the position of the matching event group; S107, insert the foot pedal data into the second insertion node to obtain the new playback file.

[0040] In some embodiments, the playback file may be a music instruction file, such as a MIDI file, that records performance data generated by a user playing the piano.

[0041] In some embodiments, a MIDI file (Musical Instrument Digital Interface) can be a type of instruction-based music file. It does not directly store sound waveform data, but records digital instructions related to performance, such as pitch, duration, dynamics, timbre selection, pedal control, and tempo. These instructions can be recognized by electronic musical instruments, music production software, or automatic playing devices and converted into corresponding sounds. It is characterized by its small size and flexible editing capabilities, and is widely used in the music field.

[0042] In some embodiments, the playback file can be used for direct playback or for repetition (i.e., driving the piano to reproduce the performance process, such as automatically playing according to the playback file).

[0043] For example, the automatic piano playing method described in patent application CN120472871A can be used to drive the piano to repeat the playback file.

[0044] Among them, sensors (such as speed sensors, displacement sensors or acceleration sensors, which can detect the speed, displacement or acceleration of the corresponding piano key being triggered in real time) and / or image acquisition modules (such as cameras) can be used to identify the trigger information of the trigger event (such as key touch).

[0045] In some embodiments, the sheet music file may be a digital sheet music that records standard performance information, such as MusicXML (Music Extensible Markup Language).

[0046] In some embodiments, ideographic elements refer to musical elements that make up a musical score file, which may include notes (such as half notes, quarter notes, etc.), rests (such as half rests, quarter rests, etc.), performance technique markings (such as trills, arpeggios, etc.), dynamic markings (such as crescendos, diminuendos, etc.), clefs, key signatures (such as sharps, flats, etc.), or structural markings (such as repeats, leaps, etc.). It should be understood that ideographic elements are symbols or graphics that have specific musical meanings in the musical score.

[0047] In some embodiments, the position of an ideographic element is used to define the position of the ideographic element in the score file (e.g., the position of the note in which section).

[0048] For example, the position of a symbolic element can refer to its position on a musical line in the score. The position can be a height position (such as the height position of a note on a musical line, which can be used to indicate pitch, such as a line or space in the musical line).

[0049] Alternatively, position can refer to the order in which the symbolic elements appear in the score file, such as on a particular beat of a measure.

[0050] In other words, the position of a symbolic element can refer to its position on the spectrum.

[0051] In this embodiment, please refer to Figure 2 This invention selects the chord change nodes in the score file as the index (i.e., the first insertion node), then finds the corresponding position in the playback file (i.e., the second insertion node) and inserts the corresponding pedal data. This user-driven (i.e., the final insertion position is determined by the user's playback file) and score alignment (i.e., the relative position of the insertion node is determined by the standard score file) fusion mechanism has at least the following technical effects: Firstly, by inserting pedal data at the change nodes based on chord changes, the pedal accompaniment is kept highly consistent with the logic of the music itself, greatly reducing the complexity of pedal data insertion node calculation, and avoiding harmonic chaos caused by misaligned time nodes.

[0052] Secondly, based on the chord change nodes in the score file, the insertion position of the pedal data can be determined more accurately, which helps to synchronize the standard pedal data with the actual performance rhythm. In other words, with the synergistic effect of the dual benchmarks of the score file and the playback file, a positioning basis that is both standardized and personalized can be provided for the insertion of pedal data.

[0053] In some embodiments, pedal data may refer to a data sequence that records information such as pedaling time / speed, pedal release time / speed, and pedal depth.

[0054] In some embodiments, a pedal database may be pre-set, which records different types of pedal data. For example, it may include: full pedal data of slow pedaling and fast release, half full pedal data of slow pedaling and fast release, full pedal data of fast pedaling and fast release, light pedaling data of fast pedaling and fast release, full pedal data of slow pedaling and slow release, etc.

[0055] In some embodiments, pedal data can be selected from the pedal database according to the performance requirements of the current piece.

[0056] For example, if the current piece is a prelude, then you can choose clearer, more restrained (e.g., light pedaling) pedal data based on the performance requirements of this genre.

[0057] Alternatively, users can select the pedal data.

[0058] For example, users can choose the artistic effect they want to achieve (such as style), and further, the foot pedal data corresponding to the style can be determined based on the user's choice.

[0059] In some embodiments, a chord corresponds to at least one pedal action; corresponding pedal actions (i.e., recommended pedal data) can be recommended based on the type of chord. For example, for a major triad, a slow, fast full pedal data can be recommended; for a dominant seventh chord, a fast, fast half pedal data can be recommended.

[0060] Alternatively, in some embodiments, the pedal data can be selected based on the type between the preceding and following chords. For example, if the preceding and following chords share a common note, the harmonic transition is natural, and there is no need to adjust the pedal depth; simply keeping the pedal fully depressed to maximize resonance is sufficient. For instance, the pedal depth for the preceding and following chords can be the same, requiring only a switching action (i.e., releasing the pedal and pressing it down again) when switching.

[0061] For example, in some embodiments, the two chords share no or very few common notes. For instance, if the first chord is a C major triad, its pedal parameters are: slow pressing and fast releasing (full pedal). If the second chord is an Eb major triad, its pedal parameters are set to: fast pressing and fast releasing (half pedal). The switching action between the two is to only half-lift the pedal to switch to the next chord.

[0062] It is understood that this invention allows users to manually adjust the selection rules for pedal data, such as adding or deleting data.

[0063] Alternatively, it could recommend different types of pedal data when the chord changes.

[0064] In some embodiments, please refer to Table 1, chord types can be preset to identify chord types in the score file and the transition nodes between different types of chords.

[0065] Table 1: For example, please see Figure 1 The chord progressions are as follows: measure 12 is "FCFAC", measure 13 is "CGCEG", measure 14 is "DADFA", measure 15 is "AEACECA", and measure 16 is "FCFCEBGB". Based on the preset chord types in Table 1, measure 12 belongs to the 4th basic group, measure 13 to the 1st basic group, measure 14 to the 2nd basic group, measure 15 to the 6th basic group, and measure 16 is a combination chord of the 4th and 3rd basic groups.

[0066] Furthermore, the position where the chord changes (i.e., the change node) can be determined as the first insertion node.

[0067] For example, please see Figure 3 The "measure start time" on the left side of the score corresponds to key points of chord changes (e.g., measure 7 starts at 24:00; measure 8 starts at 28:00; measure 9 starts at 32:00). These time points are the points of change for chord switching. At the time points of chord changes (e.g., 24:00, 28:00, 32:00), perform the "up" action (i.e., release) of the pedal; immediately following the time point (e.g., 24:01, 28:01, 32:01), perform the "down" action (i.e., press down) of the pedal.

[0068] For example, the original MusicXML or MIDI file may not contain pedal information. Pedal information is added at chord changes. For instance, in measures 12, 13, 14, 15, and 16, each measure has a different chord, so each measure begins with the pedal pressed and ends with the pedal released. Measure 16 contains chord changes, so it also shows a situation where the pedal is released before being pressed again.

[0069] It should be understood that, taking the sustain pedal as an example, its mechanism is to lift the damper, allowing the strings to continue vibrating, thereby producing reverberation and resonance. In other words, this invention uses the chord switching point as the first insertion node, meaning that before entering a new chord, the pedal can be released to cut off the residual sound of the previous chord, and then, according to the needs of the new chord, the pedal can be pressed down again to form a new sound.

[0070] In some embodiments, S102 can also be used to obtain the pedal data corresponding to the first insertion node directly from the records in the score file (such as which measure and which beat the pedal should be stepped on).

[0071] For example, the first insertion node can be determined based on the foot pedal markings in the score file or user annotations (such as user-annotated notes).

[0072] In some embodiments, the pedal data corresponding to the first insertion node can also be generated based on chord changes. For example, if the analysis finds that the new chord has a long bass note, the system will generate a set of pedal data that is deeply pressed and maintained for a long time; if it is a light staccato passage, it will generate pedal data that is quickly tapped.

[0073] In some embodiments, the neighboring basic information of the first insertion node may refer to the basic information of at least one set of ideographic elements that make up at least one chord before and after the first insertion node.

[0074] For example, the first insertion node is used to define the position where pedal data needs to be inserted in the playback file, and its adjacent basic information can be the basic information of at least two or at least three ideographic elements (such as musical notes) preceding the first insertion node.

[0075] Specifically, an indexed array can refer to at least one semantic element, such as "FAC".

[0076] In some embodiments, matching may mean that the sequence of semantic elements of the index array and the matching event group are completely identical; or it may mean that the similarity between the sequence of semantic elements of the index array and the matching event group is greater than a preset similarity (such as 80%).

[0077] For example, similarity can be determined based on the percentage overlap between the index array and the sequence of semantic elements of the matching event group.

[0078] For example, if the index array contains 5 notes (such as "GBDFA"), and the matching event group contains 4 notes in the exact same order as the index array (such as "CBDFA"), then the overlap percentage is 80%, and the index array can be considered to match the matching event group.

[0079] In some embodiments, an audio-to-score matching method described in CN120508674A can be used to quickly locate and match the index array (i.e., score) and the matching event group (i.e., audio).

[0080] In some embodiments, inserting the pedal data into the second insertion node to obtain a new playback file may include: using the second insertion node as a reference, writing at least one piece of pedal data into the corresponding position of the original playback file (i.e., the second insertion node); thereafter, integrating the original playback file and the pedal data to regenerate a new playback file.

[0081] In some embodiments, a set of pedal data includes: pedaling time, pedaling release time, and at least one pedal depth.

[0082] In some embodiments, the pedal pressing time preferably corresponds to the chord start time, and the pedal releasing time preferably corresponds to the chord end time.

[0083] In other words, the time window between pedaling and releasing should preferably cover at least one complete chord.

[0084] In some embodiments, pedal depth may include multiple pedal depths within a time window between pedaling time and pedal release time.

[0085] In some embodiments, pedal curves can be plotted based on multiple sets of pedal data. For example, pedal curves can be plotted according to a performance data visualization method described in patent application CN120833765A to form an intuitive mapping between pedals and time, thereby improving the user experience.

[0086] In some embodiments, at least one of the pedal depths is timestamped. For example, in this embodiment, a pedal curve may be inserted into the playback file, which records the changes in pedal depth at multiple time points.

[0087] In some embodiments, selecting at least one changed node as the first insertion node includes the steps of: When a foot marker is set within the associated interval of the changed node, the changed node is used as the first inserted node.

[0088] In some embodiments, the associated region of a pedal mark refers to the range of target notes covered in time by the pedal event corresponding to the pedal mark. For example, a pedal mark may be used to mark a measure, in which case the associated region is the measure in which the pedal mark is located. Alternatively, a pedal mark may be used to mark a musical phrase, in which case the associated region of the pedal mark is the musical phrase in which the pedal mark is located.

[0089] In some embodiments, the associated area corresponding to a foot pedal mark can be predetermined by the user (e.g., a composer). In some embodiments, the foot pedal mark includes at least one of a depressed mark, a released mark, or a sustain mark. Taking the sustain pedal as an example, Figure 1 In this context, “ped” means “step down”, “*” means “release”, and “—” means “maintain”.

[0090] It should be understood that the present invention preferably determines the change node corresponding to the associated interval with the pedal mark as the first insertion node, which can ensure to a certain extent that this part of the pedal data is inserted into the corresponding position according to the composer's creative intention, and meets the hard constraints of the pedal mark.

[0091] In other words, the present invention preferably first quickly and unambiguously identifies the "must-process" change nodes corresponding to these foot pedal marks, and then judges the "to-process" change nodes that do not have clear foot pedal marks and require further reasoning, thereby improving the efficiency and reliability of the overall algorithm.

[0092] In some embodiments, the music track includes at least one musical phrase, and selecting at least one variation node as the first insertion node includes the steps of: Obtain valid information for at least one of the triggering events within a musical segment, the valid information including: the number of events; Generate the note density of the triggering events in the musical segment based on the valid information; Select the insertion quantity based on the stated note density; Select the specified number of changed nodes from among the multiple changed nodes as the first insertion node.

[0093] In some embodiments, the note density and the set insertion number are negatively correlated. For example, the higher the note density of a musical phrase, the lower the set insertion number, and correspondingly, the fewer the number of phrases identified as the first insertion node.

[0094] In some embodiments, when a musical phrase (such as a measure) has a large number of notes, i.e., a high note density, the use of the foot pedal can be reduced or avoided to allow for relatively clear intervals between different sounds / voices, thus making the main melody stand out. Conversely, if the note density is low, the use of the foot pedal can be increased or retained without worrying that the sustain pedal will make the different voices too closely connected, making them difficult to distinguish.

[0095] In some embodiments, a recommended number of insertions can be provided to the user based on the note density, such as recommending a range of 5-8 insertions. The user can then select their desired number of insertions from the recommended range.

[0096] In some embodiments, the effective information further includes: an influence factor, the influence factor including at least one of the following: register position, duration of articulation, and whether it is a root note; correspondingly, the step of generating the note density of the triggering event in the musical segment based on the effective information includes: Generate a real count for the triggering event; When the valid information includes the impact factor, the corresponding true count is updated to a weighted count based on the impact factor. The note density of the musical phrase is generated based on the weighted count and / or the true count.

[0097] In some embodiments, the note density may not be an absolute number, but may be a weighted number.

[0098] In some embodiments, the vocal register may include a high register, a low register, a mid register, etc.

[0099] In particular, the present invention preferably assigns higher weight to low-frequency notes, that is, when low-frequency notes are played, the use of the pedal is reduced more carefully to avoid the continuous bass sound masking the melody and harmony in the mid and high-frequency range, thus compromising the clarity of the music in terms of listening experience.

[0100] In some embodiments, for notes of different durations (such as long notes, medium-long notes, and short notes), the present invention preferably assigns higher weight to long notes, that is, it more carefully reduces the use of the pedal when a note is sung for a long duration, so as to avoid the sustain effect affecting the listening experience.

[0101] The root note of a chord is the lowest note in the chord, also known as the fundamental note.

[0102] In some embodiments, the present invention preferably assigns higher weight to the root note of the chord, that is, the use of the pedal is reduced more carefully when the root note of the chord is sounded, which to a certain extent ensures that the chord is clearly distinguishable and avoids the root note from mixing with notes that do not belong to the same chord in terms of sound effect, resulting in blurred harmonic properties and affecting the listening experience.

[0103] For example, a musical phrase may have 4 notes (such as note A, note B, note C, and note D), and its actual count is 4.

[0104] For example, taking the position of the pitch as an influence factor, note A is in the high pitch range and is counted as 0.8, note B is in the high pitch range and is counted as 0.8, note C is in the low pitch range and is counted as 1.2, and note D is in the middle pitch range and is counted as 1. In this case, the weighted count of the four notes A, B, C, and D is 3.8.

[0105] For example, taking the duration of pronunciation as an influencing factor, note A is a long tone and is counted as 1.3, note B is a short tone and is counted as 0.7, note C is a short tone and is counted as 0.7, and note D is a medium-long tone and is counted as 1.1. At this time, the weighted count of the four notes A, B, C and D is 3.8.

[0106] For example, if we take whether a note is the root note as an influencing factor (e.g., root note, not root note), note A is the root note and is counted as 1.5, note B is not the root note and is counted as 0.8, note C is not the root note and is counted as 0.8, and note D is the root note and is counted as 1.5. In this case, the weighted count of the four notes A, B, C, and D is 4.6.

[0107] In some embodiments, different weighted counts can be pre-set for different influence factors.

[0108] In some embodiments, when the effective information includes the influence factor, the present invention preferably updates the corresponding true count to a weighted count based on the influence factor. That is, when deciding whether to use a change node as the first insertion node, the present invention considers not only "how many notes are played", but also identifies the notes that need to be focused on in combination with the context of the score, and appropriately increases their "weight" (i.e., weighted count) to calculate a note density that is more in line with the real scene. This helps to ensure sound clarity with fewer pedals (i.e., less sustain effect) when the note density is low, and helps to ensure smoothness with more pedals when the note density is high.

[0109] In other words, this invention proposes a restrictive foot pedal node selection mechanism. Instead of uniformly selecting all variable nodes as the first insertion node, this invention determines the note density that better matches the actual performance situation through the context information of the score itself, and further determines whether to select the variable node as the first insertion node, so as to avoid the negative impact of foot pedal accompaniment on the playback file due to too many node selections (such as excessive sustain effects causing muddy sound effects).

[0110] It should be understood that the restrictive pedal node selection mechanism proposed in this invention helps to balance computing resources and artistic expression, enabling high-quality pedal accompaniment insertion even under limited computing power. This allows high-performance computing tasks to run efficiently and stably, greatly reducing the processing difficulty of computers and also reducing the difficulty of automatic pedal accompaniment insertion to some extent.

[0111] In some embodiments, selecting at least one changed node as the first insertion node may include the steps of: Identify the style of the track; An insertion rule is selected for the track according to the style, and the insertion rule includes: position rule or quantity rule; At least one of the changed nodes is selected as the first insertion node according to the insertion rules.

[0112] In some embodiments, the style of a piece (such as romantic, passionate, melancholic, etc.) can be identified based on the title of the score (such as "Sonata", "Prelude", "Etude", etc.), tonality (such as C major, A minor, etc.), playing techniques (such as vibrato, arpeggio, etc.), or tempo.

[0113] For example, in terms of playing technique, dense arpeggios may indicate that the style of the piece is romantic.

[0114] In some embodiments, the weighted count or true count in a musical phrase can be directly used as the note density.

[0115] In some embodiments, if the style of the actual track corresponds to both positional rules and quantity rules, either positional rules or quantity rules can be selected as the insertion rule.

[0116] Alternatively, in some embodiments, if the style of the actual track corresponds to both positional rules and quantity rules, the rules can be applied according to the priority set by the user for positional rules and quantity rules respectively.

[0117] For example, if a user pre-sets the priority of the quantity rule to 1 and the priority of the location rule to 2, with the location rule having a higher priority than the quantity rule, then the location rule can be applied first.

[0118] In some embodiments, different musical styles have different requirements for the use of pedals. The quantity rule refers to the constraints imposed by the musical style on the number of pedals used, i.e., the limit on the number of pedal data points allowed to be inserted in that musical style. For example, the Baroque style typically requires fewer pedal data points (e.g., only allowing 3 first insertion points out of 12 variations) to ensure the separation and clarity of each voice as much as possible; conversely, the Romantic style typically requires more pedal data points (e.g., allowing 8 first insertion points out of 12 variations) to create a rich harmonic atmosphere and abundant sonic resonance.

[0119] In some embodiments, the position rule refers to the restriction of the position of the node where the pedal data is inserted by the style of the track. That is, when the changing node satisfies the position rule, the present invention preferably uses the changing node as the first insertion node.

[0120] Taking staccato rules as an example, if there is a large leap of more than an octave in a chord, such as jumping from low C to high G, you can insert pedal data between the two notes. By increasing the pedal connection, you can improve the smoothness of the performance and help avoid the disjointed sound caused by the inability to connect the fingerings.

[0121] Taking the bass rule as an example, if there is a bass note that lasts for a long time (e.g., the duration is higher than the preset duration threshold) in a musical passage (e.g., the pitch of the lower voice is lower than the preset pitch threshold), and there are many high notes (e.g., the note density of the upper voice is greater than the preset density threshold), then the corresponding change node can be used as the first insertion node (e.g., inserting a faster, shallower pedal data) to maintain the continuity of the bass note while providing basic resonance for the dense high notes.

[0122] In some embodiments, the restrictive pedal node selection mechanism proposed in this invention can also determine the insertion rules according to the style of the piece (e.g., constrain the number of inserted nodes according to quantity rules, or select the position of the inserted nodes according to position rules). Therefore, it can largely ensure that the selected inserted nodes meet the specific needs of the piece performance, and avoid or reduce the problem of style inconsistency caused by adding pedal accompaniment to all the changing nodes.

[0123] It should be understood that this invention uses effective note density and insertion rules to restrict the insertion of pedal data from two levels: the number of nodes and the position of nodes (such as setting the number of insertions), thereby imposing progressive restrictions on the insertion of automatic pedal accompaniment, reducing the difficulty of inserting automatic pedal accompaniment, and improving the basic compatibility between pedal accompaniment and playback files.

[0124] In some embodiments, obtaining the foot pedal data corresponding to the first insertion node includes: The performance preferences for the aforementioned pieces are obtained, including: pedal depth, performer style, or musical style. The pedal data is obtained based on the performance preferences.

[0125] In some embodiments, performance preferences are used to define a user’s preference for different types of pedal data, such as pedal depth.

[0126] Among them, pedal depth can be used to define a user's preference for the depth to which they press the pedals. For example, some users (such as beginners) have weaker pedal control and generally press the pedals too shallowly; or, they may not fully understand the function of the pedals, and therefore may often experience sudden pressing or stopping.

[0127] Among them, "performer style" is used to define a user's preference for a particular performer's pedaling style. For example, a database of performer pedaling styles can be created by analyzing a large number of playback files from different performers; then, a user's performer style can be determined based on the pedaling styles of performers they frequently select.

[0128] Among them, the musical style is used to define the user's preference for different styles of music. For example, the romantic style usually requires pressing the pedal deeper to use resonance to create a full and lyrical tone.

[0129] In some embodiments, pedal data can be selected in the pedal data according to any of the performance preferences.

[0130] In some embodiments, the present invention also proposes a foot-pedal accompaniment insertion system for performing... Figure 4 The system includes the following steps for inserting foot pedal accompaniment: The chord category classification information module is used to classify and organize information on various types of chords (such as major triads, minor triads, seventh chords, etc.) and clarify the type affiliation of different chords.

[0131] The chord-related note information extraction and classification numbering module is used to extract the constituent note information corresponding to each chord (such as the specific pitches contained in a certain chord), and organize and assign numbers to these note information to facilitate the subsequent identification and management of the note content of the chord.

[0132] The chord variation comparison module is used to compare chord differences in different scenarios, such as changes in the composition of chords in the same melody passage, and adjustments to chords in different versions, to analyze chord transformations and evolution.

[0133] The pedal release and pedal press description injection module is used to associate the operation description information of "pedal release" and "pedal press" in piano performance with the corresponding chord performance content, and supplement the relevant annotation information of pedal operation and chord coordination.

[0134] In some embodiments, the above-described foot-operated accompaniment insertion system further includes a CPU (Central Processing Unit) module for executing algorithm software; a DDR memory module (Double Data Rate SDRAM) for running programs; and a FLASH (Adobe Flash) storage module for storing the original MusicXML file and the modified MusicXML file.

[0135] In some embodiments, the foot pedal mentioned in this invention mainly refers to the sustain pedal.

[0136] In other embodiments, similar strategies and methods can be used to generate corresponding MusicXML files for the soft pedal, which will not be elaborated upon in this invention.

[0137] In some embodiments, see Figure 6 The present invention also proposes a system for inserting foot pedal data into a playback file, comprising: The playback file acquisition module is used to acquire the playback file of the track. The playback file records the trigger information of at least one trigger event. The trigger information includes the identifier of the trigger element, the trigger time, and the release time. The music score file acquisition module is used to acquire the music score file corresponding to the piece. The music score file includes: basic information of the symbolic elements, the basic information including: the category and position of the symbolic elements, and the category of the symbolic elements includes: notes, clefs, performance technique markings and / or key signatures; A change node identification module is used to identify change nodes in the musical score file that produce chord changes; An insertion node selection module is used to select at least one of the changed nodes as the first insertion node; The index array acquisition module is used to acquire the foot data corresponding to the first insertion node, and to identify the neighboring basic information of the first insertion node as an index array, wherein the index array includes basic information of at least one of the semantic elements; An event matching module is used to retrieve matching event groups that match the index array in the playback file; An insertion node setting module is used to set a second insertion node in the playback file according to the position of the matching event group; The pedal data insertion module is used to insert the pedal data into the second insertion node to obtain the new playback file.

[0138] It should be understood that the system proposed in this invention can be used to implement the steps described in any embodiment of this invention.

[0139] In some embodiments, the applicant notes that MusicXML typically does not contain foot pedal information.

[0140] To address this, the present invention can analyze the content of MusicXML, automatically add pedal data (such as descriptions of pedal release / press) to the MusicXML file, and generate a new MusicXML file with pedal data, such as... Figure 1 .

[0141] For example, generating a sheet music file with pedal data may include the following steps: Obtain the sheet music file for the track; Identify the changing nodes in the score file that produce chord changes, and use at least one changing node as the first insertion node; Insert the pedal data into the first insertion node to generate a score file with pedal data.

[0142] It should be understood that the MusicXML file with pedal data generated by this invention can not only be displayed as sheet music, but also drive the piano keys and pedals to play in sync, providing users with automatic pedal accompaniment; at the same time, with the help of the pedal information in the MusicXML file, users can also see more clearly whether their pedals are pressed correctly.

[0143] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0144] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a computer terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0145] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims. All of these forms are within the protection scope of the present invention.

[0146] In some embodiments, this application also provides a schematic block diagram of the structure of a computer device, please see... Figure 7 Computer programs can be used in situations such as Figure 7 It runs on the computer device shown. Figure 7 As shown, the computer device includes a processor, memory, and a network interface connected via a system bus. The memory may include a non-volatile storage medium and internal memory. The non-volatile storage medium may store an operating system and a computer program. The computer program includes program instructions that, when executed, cause the processor to perform the methods described in any embodiment of the present invention. The processor provides computing and control capabilities to support the operation of the entire computer device. The internal memory provides an environment for the execution of the computer program in the non-volatile storage medium. When the computer program is executed by the processor, it causes the processor to perform the methods described in any embodiment of the present invention. The network interface is used for network communication, such as sending assigned tasks. Those skilled in the art will understand that... Figure 7The structures shown are merely block diagrams of a portion of the structure related to the present application and do not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than shown in the figures, or combine certain components, or have different component arrangements. It should be understood that the processor may be a Central Processing Unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

Claims

1. A method of inserting foot pedal data in a play file, characterized by, The method comprises the steps of: S100, obtaining a playing file of a piece, wherein the playing file records trigger information of at least one trigger event, and the trigger information comprises identification of a trigger element, a trigger time and a release time; S101, obtaining a score file corresponding to the piece, wherein the score file comprises basic information of a semantic element, and the basic information comprises a category and a position of the semantic element, and the category of the semantic element comprises a note, a staff line, a performance technique mark and / or a key signature; and the position of the semantic element is used to define a position of the semantic element in the score file; S102, identifying a change node that produces a chord change in the score file; S103, selecting at least one change node as a first insertion node; S104, obtaining foot pedal data corresponding to the first insertion node, and identifying adjacent basic information of the first insertion node as an index array, wherein the index array comprises basic information of at least one semantic element; S105, searching for a matching event group matching the index array in the playing file; S106, setting a second insertion node in the playing file according to a position of the matching event group; S107, inserting the foot pedal data into the second insertion node to obtain a new playing file.

2. The method of claim 1, wherein, The foot pedal data comprises a foot pedal pressing time, a foot pedal releasing time and at least one foot pedal depth; and at least one foot pedal depth is marked with a timestamp.

3. The method of claim 1, wherein, The step of selecting at least one change node as a first insertion node comprises the step of: When a foot pedal mark is set in an associated interval of the change node, the change node is selected as the first insertion node.

4. The method of claim 1, wherein, The piece comprises at least one section, and the step of selecting at least one change node as a first insertion node comprises the step of: Obtaining effective information of at least one trigger event in a section, wherein the effective information comprises a number of events; Generating a note density of the trigger event in the section according to the effective information; Setting an insertion number according to the note density; Selecting the change nodes of the insertion number from a plurality of change nodes as the first insertion nodes.

5. The method of claim 4, wherein, The note density and the insertion number are negatively correlated.

6. The method of claim 4, wherein, The effective information further comprises an influence factor, and the influence factor comprises at least one of the following: a position of a sound area, a sound duration and whether it is a root note; and correspondingly, the step of generating a note density of the trigger event in the section according to the effective information comprises the steps of: Generating a real count for the trigger event; When the effective information comprises the influence factor, updating the corresponding real count to a weighted count according to the influence factor; Generating a note density of the section according to the weighted count and / or the real count.

7. The method of claim 1, wherein, The step of selecting at least one change node as a first insertion node comprises the steps of: Identifying a style of the piece; Selecting an insertion rule for the piece according to the style, wherein the insertion rule comprises a position rule or a number rule; Selecting at least one change node as the first insertion node according to the insertion rule.

8. The method of claim 7, wherein, The position rule comprises: A jump rule, the jump rule being that when a pitch difference between two notes before and after the change node is greater than a preset pitch degree, the change node is selected as the first insertion node; And / or, a bass rule, the bass rule being that when a pitch of a next bass part after the change node is lower than a preset pitch threshold, a time value of the bass part is higher than a preset time value threshold, and a note density of a treble part is greater than a preset density threshold, the change node is selected as the first insertion node.

9. The method of claim 1, wherein, The pedal data corresponding to the first insertion node is obtained, including: Obtaining a performance preference of the piece, the performance preference including a pedal depth, a performer style, or a style style; The pedal data is obtained according to the performance preference.

10. A system for inserting foot pedal data in a play file, characterized by Including: A playing file obtaining module is configured to obtain a playing file of a piece, the playing file recording trigger information of at least one trigger event, the trigger information including an identifier of a trigger element, a trigger time, and a release time; A score file obtaining module is configured to obtain a score file corresponding to the piece, the score file including basic information of a semantic element, the basic information including a category and a position of the semantic element, and the category of the semantic element including a note, a staff line, a performance technique mark, and / or a key signature; wherein the position of the semantic element is used to define a position of the semantic element in the score file; A change node identifying module is configured to identify a change node producing a chord change in the score file; An insertion node selecting module is configured to select at least one change node as a first insertion node; An index array obtaining module is configured to obtain pedal data corresponding to the first insertion node, identify adjacent basic information of the first insertion node as an index array, and the index array including basic information of at least one semantic element; An event matching module is configured to search for a matching event group matching the index array in the playing file; An insertion node setting module is configured to set a second insertion node in the playing file according to a position of the matching event group; A pedal data inserting module is configured to insert the pedal data into the second insertion node to obtain a new playing file.

Images