Pedal data display method and system

Abstract

The invention relates to the field of playing data visualization, in particular to a pedal data display method and system. The method comprises the steps that at least one section of pedal data of at least one pedal is obtained, and the pedal data comprises a pedal value and a time stamp of the pedal value; generating at least one section of labeling track according to the pedal data; wherein the labeling track comprises a first simulation track and a second simulation track; correspondingly, the method comprises the steps that when the pedal value is larger than a first pedal threshold value, a first simulation track is generated according to the pedal value, and the first simulation track is displayed in a first labeling form; when the pedal value is larger than a second pedal threshold value and smaller than or equal to the first pedal threshold value, a second simulation track is generated according to the pedal value, and the second simulation track is displayed in a second labeling form; and generating a combined track according to the at least one section of labeled track, and displaying the combined track on the music score. According to the invention, the smoothness and integrity of pedal data display can be improved, and the user experience is optimized.

Description

[0001] Divisional application This application is a divisional application of Chinese invention patent application No. 202511288768.8, filed on September 10, 2025, entitled "A Method and System for Visualizing Performance Data". Technical Field

[0002] This invention relates to the field of performance data visualization, specifically to a method and system for displaying pedal data. Background Technology

[0003] In the early stages of piano learning, some users, in an attempt to produce a louder sound, hold the right pedal down for extended periods, resulting in poor performance. Others, despite mastering pedal technique in the practice room, perform chaotically on stage. These improper pedal usage issues largely relate to the level of proficiency in pedal technique. These problems need to be addressed during the learning process to gradually progress to higher levels of piano playing.

[0004] Currently, existing technologies have also attempted to propose some precise methods for stimulating piano pedal movements.

[0005] For example, patent application CN120510757A discloses a piano pedal movement amplitude indicator device for music teaching, including a placement plate and an indicator structure disposed on the placement plate. A suction cup is installed at the bottom of the placement plate. The indicator structure includes an indicator frame, on the front of which, from left to right, are arranged a soft pedal display screen, a mute pedal display screen, and a sustain pedal display screen. The indicator frame also has a reminder element for pedal switching. The soft pedal display screen has a first indicator element for displaying the soft pedal movement. This invention, through the setting of the indicator structure, can display the movement and amplitude of the piano pedals, thereby achieving precise reminders of the piano pedal movements. This allows learners to accurately control the range of piano pedal movements, facilitating piano teaching demonstrations, improving teaching effectiveness, and promoting rapid piano learning.

[0006] For example, patent application CN120412345A provides a computer vision-based intelligent piano teaching projection device and method. The device includes a data acquisition and projection module, a calibration module, a data processing module, a side projection module, and an auxiliary module. The intelligent piano teaching projection method implemented based on this device includes segmenting the top-view image of the performer in the piano key area acquired by the data acquisition and projection module using the Mask2former image segmentation model; constructing the performer's finger skeleton using a finger skeleton modeling algorithm; determining whether the performer's current playing is correct; if the playing is incorrect, the light projection position of the data acquisition and projection module remains unchanged, prompting the performer to press the key again; if the playing is correct, the projection prompts the next key to be pressed and the pedal to be pressed, providing guidance to the performer. This invention does not require modification of the piano; simply fixing the device to the piano allows for piano instruction through projection.

[0007] However, the applicant noted that users still find it difficult to independently judge the gap between their performance and the expected effect while playing. Summary of the Invention

[0008] The purpose of this invention is to provide a method for visualizing performance data, which partially solves or alleviates the above-mentioned shortcomings in the prior art and can improve the smoothness and completeness of pedal data display.

[0009] To solve the aforementioned technical problems, the present invention specifically adopts the following technical solution: A first aspect of the present invention is to provide a method for visualizing performance data, comprising the steps of: S101, acquire at least one segment of pedal data for at least one pedal, wherein the pedal data includes: pedal value, and time stamp of the pedal value; S102, generate at least one labeled trajectory based on the foot pedal data; wherein, the labeled trajectory includes: a first simulated trajectory and a second simulated trajectory; correspondingly, S102 includes the following steps: When the pedal value is greater than the first pedal threshold, a first simulated trajectory is generated based on the pedal value, and the first simulated trajectory is displayed using a first annotation format; When the pedal value is greater than the second pedal threshold and less than or equal to the first pedal threshold, a second simulated trajectory is generated based on the pedal value, and the second simulated trajectory is displayed using a second annotation format; S103, Generate a merged trajectory based on at least one of the marked trajectories, and display the merged trajectory on the musical score.

[0010] In some embodiments, the labeled trajectory further includes: a virtual trajectory; correspondingly, S102 further includes the step: When the pedal value is less than or equal to the second pedal threshold, the pedal value is identified as a virtual target; Select a neighboring step value that is close to the virtual target; wherein the difference between the time stamp of the neighboring step value and the time stamp of the virtual target is less than a preset time difference; A virtual trajectory is generated based on the neighboring footstep values ​​and the virtual target, wherein the virtual trajectory is displayed using a third annotation format.

[0011] In some embodiments, the step of generating a virtual trajectory based on the neighboring footstep values ​​and the virtual target includes: (1) Obtain a group of adjacent footstep data, wherein the group of adjacent footstep data includes: a first adjacent footstep data and a second adjacent footstep data, wherein the first adjacent footstep data and the second adjacent footstep data are adjacent footstep values ​​located before and after the virtual target, respectively. (2) Using a virtual prediction method, at least two predicted trajectories are generated based on adjacent foot data sets; wherein, step (2) includes: Virtual points are generated using the adjacent foot data set, and these virtual points are used to define the starting height of the predicted trajectory; Calculate the changing trajectory of the adjacent foot pedal data group, and predict and generate a virtual changing trajectory based on the changing trajectory; A predicted trajectory is generated based on the initial height and the virtual change trajectory; (3) Generate the virtual trajectory based on at least two predicted trajectories.

[0012] In some embodiments, step (3) includes: At least two predicted trajectories are fused using a trajectory fusion method to generate the virtual trajectory.

[0013] In some embodiments, the trajectory fusion method includes one or more of the following: direct connection method, smooth transition method, and trajectory alignment method based on optimization method.

[0014] In some embodiments, the second adjacent pedal data is obtained from an external data source, which is collected through a music score database, and the external data source records at least one standard pedal value for a note node and a time tag for the standard pedal value.

[0015] In some embodiments, the steps further include: A first auxiliary trajectory is generated based on the first foot pedal threshold, and the first auxiliary trajectory is displayed using a fourth annotation format; And / or, generate a second auxiliary trajectory based on the second foot pedal threshold, and display the second auxiliary trajectory using a fifth annotation format.

[0016] In some embodiments, the steps further include: Acquire multiple segments of first pedal data within the first time period; A pedal score is generated based on multiple segments of the first pedal data and an external data source; wherein, the external data source includes: a standard pedal value and a time label for the standard pedal value; Determine whether the pedal score is less than the first preset score. If so, generate a guide trajectory based on the external data source, and display the guide trajectory using the sixth annotation format.

[0017] In some embodiments, the step of generating a guide trajectory based on the external data source includes: Obtain the external data source corresponding to the second time period; A first guidance trajectory is generated based on the corresponding external data source; Obtain the labeled trajectory in the second time period; The guide trajectory is generated by fusing the labeled trajectory and the first guide trajectory using a trajectory fusion method.

[0018] A second aspect of the present invention is to provide a performance data visualization system, comprising: A pedal data acquisition module is used to acquire at least one segment of pedal data from at least one pedal, wherein the pedal data includes: pedal value and a time stamp of the pedal value; The trajectory generation module is used to generate at least one trajectory based on the foot pedal data; wherein the trajectory includes a first simulation trajectory and a second simulation trajectory; correspondingly, the trajectory generation module is also used to: When the pedal value is greater than the first pedal threshold, a first simulated trajectory is generated based on the pedal value, and the first simulated trajectory is displayed using a first annotation format; When the pedal value is greater than the second pedal threshold and less than or equal to the first pedal threshold, a second simulated trajectory is generated based on the pedal value, and the second simulated trajectory is displayed using a second annotation format; The merged trajectory display module is used to generate a merged trajectory based on at least one of the marked trajectories and display the merged trajectory on the musical score.

[0019] Beneficial technical effects: The foot pedal data display scheme proposed in this invention is particularly suitable for piano teaching scenarios, specifically: 1) Preferably, this invention merges the second simulated trajectory corresponding to invalid data with the first simulated trajectory corresponding to valid data. This has at least the following effects on the smoothness of visual display: 1) This invention treats the user's pedaling movements as a continuous set of motion curves linked to the musical score, rather than several isolated motion points. Continuous display helps to connect with muscle memory and reduces the difficulty of understanding. 2) Furthermore, by observing invalid data, it is possible to determine whether the performer's control of the pedals is sufficiently professional. For example, if the marked trajectory repeatedly bounces at the second pedal threshold, it indicates that the performer's foot strength control is unstable or the pedal position is inappropriate and needs adjustment. Therefore, displaying the process of the pedal changing from invalid to valid values ​​helps the user understand the complete change path of the pedal data, acceleration, and other pedal usage habits.

[0020] 2) By combining virtual and real methods, that is, by using the fusion mechanism of simulated foot tracks and virtual foot tracks, it is possible to generate virtual tracks that are most consistent with the actual situation, effectively avoiding the sudden break of the marked track caused by blank foot data, thus ensuring that the foot track is presented completely and smoothly during the visualization of foot data. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. In all the drawings, similar elements or parts are generally identified by similar reference numerals. The elements or parts in the drawings are not necessarily drawn to scale. Obviously, the drawings described below are some embodiments of the present invention, and those skilled in the art can obtain other drawings based on these drawings without any creative effort.

[0022] Figure 1 A flowchart illustrating a performance data visualization method provided in an embodiment of this application; Figure 2 An example diagram illustrating a performance data visualization method provided in this application embodiment; Figure 3 This is a schematic diagram of the structure of a performance data visualization system provided in an embodiment of this application; Figure 4 This is a schematic diagram of an exemplary piano pedal system to which this application may be applied; Figure 5 This is yet another structural schematic diagram of an exemplary piano pedal system to which this application may be applied; Figure 6 This is another structural schematic diagram of an exemplary piano pedal system to which this application may be applied.

[0023] Figure label: 111. Mounting plate; 112. Mounting block; 113. External left pedal; 114. External middle pedal; 115. External right pedal; 211. Mounting bracket; 212. Pedal motor; 213. Connecting rod; 221. Base plate; 222. Linkage mechanism; 223. Shock-absorbing spring; 2221. Transmission link; 2222. Hinge support; 2223. Connecting bolt; 3. Piano pedal. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0025] In this document, suffixes such as "module," "part," or "unit" used to denote elements are used only for the purpose of illustrative purposes and have no specific meaning in themselves. Therefore, "module," "part," or "unit" may be used interchangeably.

[0026] In this document, the terms "upper," "lower," "inner," "outer," "front," "rear," "one end," and "the other end," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing the present invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the present invention. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0027] In this document, unless otherwise explicitly specified and limited, the terms "installed," "equipped with," "connected," etc., should be interpreted broadly. For example, "connection" can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection, a direct connection, or an indirect connection through an intermediate medium; it can be a connection within two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0028] In this document, "and / or" includes any and all combinations of one or more of the listed related items.

[0029] In this article, "multiple" means two or more, that is, it includes two, three, four, five, etc.

[0030] As used in this specification, the term "about" typically means + / -5% of the value, more typically + / -4% of the value, more typically + / -3% of the value, more typically + / -2% of the value, even more typically + / -1% of the value, and even more typically + / -0.5% of the value.

[0031] In this specification, certain embodiments may be disclosed in a range-bound format. It should be understood that this "range-bound" description is merely for convenience and brevity and should not be construed as a rigid limitation on the disclosed range. Therefore, the description of a range should be considered as having specifically disclosed all possible subranges and the individual numerical values ​​within those ranges. For example, a description of the range 1-6 should be considered as having specifically disclosed subranges such as from 1 to 3, from 1 to 4, from 1 to 5, from 2 to 4, from 2 to 6, from 3 to 6, etc., and the individual numbers within those ranges, such as 1, 2, 3, 4, 5, and 6. This rule applies regardless of the breadth of the range.

[0032] Example 1: Please see Figure 1 The present invention provides a method for visualizing performance data, which, in some embodiments, includes the following steps: S101, acquire at least one segment of pedal data for at least one pedal, wherein the pedal data includes: pedal value, and time stamp of the pedal value; S102, generate at least one labeled trajectory based on the foot pedal data; wherein, the labeled trajectory includes: a first simulated trajectory and a second simulated trajectory; correspondingly, S102 includes the following steps: When the pedal value is greater than the first pedal threshold, a first simulated trajectory is generated based on the pedal value, and the first simulated trajectory is displayed using a first annotation format; When the pedal value is greater than the second pedal threshold and less than or equal to the first pedal threshold, a second simulated trajectory is generated based on the pedal value, and the second simulated trajectory is displayed using a second annotation format; S103, Generate a merged trajectory based on at least one of the marked trajectories, and display the merged trajectory on the musical score.

[0033] It should be understood that the pedal data mentioned in this invention is not a simple pedal value, but a set of time series data with precise time stamps.

[0034] In some embodiments, the pedal value can refer to the depth, angle, or corresponding force value of the pedal as a result of the user's pressing action, obtained by a pedal sensor (e.g., an external position sensor or an added position sensor). The time stamp can be the precise instant the pedal value is acquired, or it can be the elapsed time calculated from the start of the performance. In some embodiments, the present invention preferably aligns the time stamps of the pedal data and the playing data, and the coordinated display of the two greatly enhances the performance effect, strengthening the role of the pedal in the harmonic effect, melodic embellishment, rhythmic enhancement, or stylistic expression of the performance.

[0035] In some embodiments, generating a labeled trajectory based on foot data includes the steps of: converting the corresponding foot values ​​into coordinates in a foot display diagram according to time markers; connecting adjacent coordinates of multiple time markers into line segments; and generating a continuous labeled trajectory based on multiple adjacent line segments.

[0036] Preferably, by setting a first pedal threshold and a second pedal threshold, and displaying pedal values ​​in different annotation formats for different intervals, the present invention can improve the readability of the pedal data visualization display by distinguishing the intervals in which different pedal values ​​are located, helping users to intuitively see whether the pedal data meets the required angle or force, thereby improving the performance effect.

[0037] In some embodiments, the first foot pedal threshold is usually an effective threshold that can trigger the soft, gentle, and sustained sound effects. That is, foot pedal values ​​greater than the first foot pedal threshold can be regarded as valid data, corresponding to the first simulation trajectory, and displayed in the form of the first annotation.

[0038] In some embodiments, the second foot pedal threshold typically refers to an invalid threshold that, although it can be collected by a sensor (e.g., an external position sensor or an added position sensor), is insufficient to trigger the soft, gentle, or sustained sound effects. That is, foot pedal values ​​that are greater than the second foot pedal threshold and less than or equal to the first foot pedal threshold can be considered invalid data, corresponding to the second simulation trajectory, and displayed in the form of a second annotation.

[0039] It should be understood that the present invention preferably merges the second simulated trajectory corresponding to invalid data with the first simulated trajectory corresponding to valid data, which has at least the following effects on the smoothness of visual display: 1) The present invention treats the user's pedaling actions as a series of continuous motion curves linked to the musical score, rather than several isolated motion points. Continuous display helps to associate with muscle memory and reduces the difficulty of understanding. 2) Furthermore, by observing invalid data, it is possible to determine whether the performer's control of the pedals is professional enough. For example, if the marked trajectory repeatedly bounces at the second pedal threshold, it indicates that the performer's foot strength control is unstable or the pedal position is inappropriate and needs adjustment. Therefore, displaying the process of the pedal changing from invalid to valid values ​​helps the performer understand the complete change path of the pedal data, acceleration, and other pedal usage habits.

[0040] In some embodiments, generating a merged trajectory based on at least one segment of the labeled trajectory may include the step of connecting multiple segments of labeled trajectory into a continuous merged trajectory in the order of time stamps.

[0041] In some embodiments, the first annotation form and the second annotation form may employ different visual effects. For example, to facilitate user differentiation, the first annotation form and the second annotation form may use different visual effects such as color and transparency.

[0042] It should be noted that the annotation formats mentioned in this invention can all be different display formats.

[0043] In some embodiments, the display form can be a static shape, a dynamic visual effect, or a combination of both. Specifically, it can be color changes (e.g., highlighting, gradient), flashing effects (e.g., periodic flashing, pulse light effect, water ripple effect), arrow indicators (e.g., directional guidance, dynamic path), icons and symbols, text labels (e.g., brief descriptions, numerical labels), shadows and outlines (e.g., projection effects, halo design, bold outlines), animation effects (e.g., explosion and scattering, floating bubbles), line thickening or highlighting, etc., without limitation.

[0044] Preferably, the first annotation format and the second annotation format are displayed differently.

[0045] In some embodiments, the labeled trajectory further includes: a virtual trajectory; correspondingly, S102 further includes the step: When the pedal value is less than or equal to the second pedal threshold, the pedal value is identified as a virtual target; Select a neighboring step value that is close to the virtual target; wherein the difference between the time stamp of the neighboring step value and the time stamp of the virtual target is less than a preset time difference; A virtual trajectory is generated based on the neighboring footstep values ​​and the virtual target, wherein the virtual trajectory is displayed using a third annotation format.

[0046] In some embodiments, when the pedal value is less than or equal to the second pedal threshold (which may be due to various reasons that the user does not press the pedal or presses it with very little force, and the pedal value corresponding to this time marker is blank), the pedal value that is less than or equal to the second pedal threshold can be identified as a virtual target.

[0047] In other words, this embodiment classifies and continuously displays the validity of pedal data, which on the one hand forms smooth guide lines (i.e., merged tracks). The smoothness of these guide lines is related to the rhythm of the music, making it easier for users to intuitively understand the actual pedal data. In addition, while satisfying the smoothness, the guide lines can also be displayed differently based on binary or tri-classification, thereby marking the differences in pedal playing status (i.e., users can intuitively focus on the overall effect of the pedals).

[0048] It should be understood that the present invention further improves the overall continuity of the marked trajectory by combining virtual and real methods. The following will further introduce the method of combining virtual and real methods.

[0049] In some embodiments, a neighboring footstep value adjacent to the virtual target can be selected; wherein the difference between the timestamp of the neighboring footstep value and the timestamp of the virtual target is less than a preset time difference.

[0050] Preferably, one or more footstep values ​​that are closest to the virtual target in time can be selected from the range of footstep values ​​that are less than or equal to the second footstep threshold as neighboring footstep values.

[0051] It should be understood that the present invention determines the selection range of adjacent pedal values ​​within a preset time difference range to ensure that the generation of the virtual trajectory matches the maximum probability of actual occurrence. For example, in a brief pedal gap (e.g., a rapid lift between two pedaling actions), using adjacent pedal values ​​to construct the virtual trajectory can make the final labeled trajectory (which may include the first simulated trajectory, the second simulated trajectory, and the virtual trajectory) appear to have a smooth, continuous transition, rather than abruptly truncated and starting from zero.

[0052] In some embodiments, a virtual trajectory can be generated based on neighboring footstep values ​​and a virtual target, including the following steps: (1) Obtain a group of adjacent footstep data, wherein the group of adjacent footstep data includes: a first adjacent footstep data and a second adjacent footstep data, wherein the first adjacent footstep data and the second adjacent footstep data are adjacent footstep values ​​located before and after the virtual target, respectively. (2) Using a virtual prediction method, at least two predicted trajectories are generated based on adjacent foot data sets; wherein, step (2) includes: Virtual points are generated using the adjacent foot data set, and these virtual points are used to define the starting height of the predicted trajectory; Calculate the changing trajectory of the adjacent foot pedal data group, and predict and generate a virtual changing trajectory based on the changing trajectory; A predicted trajectory is generated based on the initial height and the virtual change trajectory; (3) Generate the virtual trajectory based on at least two predicted trajectories.

[0053] The virtual trajectory can be displayed using a third annotation format.

[0054] In some embodiments, the acquired neighboring footstep data group (or neighboring footstep data) may include a first neighboring footstep data located before the virtual target and a second neighboring footstep data located after the virtual target.

[0055] In some embodiments, a virtual point can be determined based on the velocity or acceleration of neighboring pedal data. It should be understood that the virtual point is very close to the pedal value of neighboring pedal data; that is, the virtual point can be an approximation or prediction of the neighboring pedal data.

[0056] Preferably, a nearby footstep data point can be directly used as a virtual point.

[0057] For example, among the pedal data that is less than or equal to the second pedal threshold, the largest neighboring pedal data is used as a virtual point.

[0058] In some embodiments, virtual points are used to define the starting height of the predicted trajectory, i.e., to determine the starting point of the predicted trajectory.

[0059] In some embodiments, the changing trajectory of adjacent pedal data can be used as a virtual changing trajectory. Specifically, the direction of the changing trajectory of adjacent pedal data can be used as the direction of the virtual changing trajectory.

[0060] In some embodiments, taking the first adjacent pedal data as an example, calculating the change trajectory of the adjacent pedal data may include: 1) selecting two first adjacent pedal data (timestamps T1 and T2, corresponding pedal values ​​V1 and V2) from multiple first adjacent pedal data; 2) calculating the change speed r (i.e. the slope of the change trajectory) as V2-V1 / T2-T1 based on the two first adjacent pedal data.

[0061] In some embodiments, neighboring pedal data can be determined based on currently collected real data, pre-stored historical data, or a combination of current and historical data. As long as neighboring pedal data that meets the confidence requirements can be determined, no limitation is imposed here.

[0062] It should be understood that the rate of change r calculated based on the previous method is the trend of the pedal value changing over time.

[0063] In some embodiments, the slope of the trajectory can be determined based on the rate of change r between adjacent pedal data.

[0064] In some embodiments, if multiple first neighboring foot data are selected, mathematical methods such as linear regression or moving average can be used to calculate the rate of change r that better reflects the actual situation.

[0065] In some embodiments, a virtual trajectory can be generated based on the predicted trajectory, or the slope of the predicted trajectory can be used as the slope of the virtual trajectory.

[0066] It should be understood that the virtual trajectory is a directional reference predicted from a virtual point based on the rate of change *r* of neighboring footstep data. For example, if it is a linear prediction (i.e., the rate of change *r* remains constant), the virtual trajectory will be a straight line. If the rate of change *r* itself is changing, a higher-order mathematical model can be used for prediction, which is not limited here.

[0067] In some embodiments, a predicted trajectory is generated based on the starting height (i.e., a virtual point) and the virtual change trajectory (direction reference).

[0068] Preferably, the present invention can connect two predicted trajectories (the predicted trajectory before the virtual target is the first predicted trajectory; the predicted trajectory after the virtual target is the second predicted trajectory) to generate a virtual trajectory.

[0069] In some embodiments, the intersection of two predicted trajectories can be identified as a virtual target.

[0070] In some embodiments, step (3) includes: At least two predicted trajectories are fused using a trajectory fusion method to generate the virtual trajectory.

[0071] In some embodiments, the trajectory fusion methods that can be adopted include one or more of the following: direct connection method, smooth transition method, and trajectory alignment method based on optimization method.

[0072] In some embodiments, other existing technologies can also be used to fuse the predicted trajectory, as long as they can achieve the effect of smoothing the virtual trajectory, and no limitation is made here.

[0073] In some embodiments, the second adjacent pedal data is obtained from an external data source, which is collected through a music score database, and the external data source records at least one standard pedal value for a note node and a time tag for the standard pedal value.

[0074] Preferably, the external data source can be the standard pedal data corresponding to each sheet music.

[0075] In some embodiments, second neighboring footstep data after the virtual target can be determined based on an external data source; further, a second predicted trajectory can be generated based on the second neighboring footstep data and the virtual target, and the specific generation method can refer to the first predicted trajectory.

[0076] In some embodiments, the present invention preferably employs a high-precision sensor (e.g., a position sensor, an external position sensor, a force feedback sensor, etc.) to collect and transmit more accurate pedaling data in real time. At the same time, through a display method that combines virtual and real elements, the visualization of the pedaling data not only conforms to the real situation but also has a smoother visual effect.

[0077] In summary, this invention, through a combination of virtual and real methods, namely by using a fusion mechanism of simulated foot pedal data and virtual foot pedal data, can generate virtual trajectories that are most consistent with the actual situation, effectively avoiding sudden breaks in the marked trajectory caused by blank foot pedal data, thereby ensuring that the foot pedal is presented completely and smoothly during the visualization of foot pedal data.

[0078] In some embodiments, the steps further include: A first auxiliary trajectory is generated based on the first foot pedal threshold, and the first auxiliary trajectory is displayed using a fourth annotation format; And / or, generate a second auxiliary trajectory based on the second foot pedal threshold, and display the second auxiliary trajectory using a fifth annotation format.

[0079] Preferably, the fourth annotation format is displayed differently from the fifth annotation format. Please refer to [link / reference]. Figure 2 Different auxiliary trajectories and corresponding annotation formats can be used to display different pedaling thresholds, which can help users intuitively observe the range of pedaling data and help them correct their foot force to the normal range in a timely manner without professional guidance.

[0080] In some embodiments, the steps further include: Acquire multiple segments of first pedal data within the first time period; A pedal score is generated based on multiple segments of the first pedal data and an external data source; wherein, the external data source includes: a standard pedal value and a time label for the standard pedal value; Determine whether the pedal score is less than the first preset score. If so, generate a guide trajectory based on the external data source, and display the guide trajectory using the sixth annotation format.

[0081] In some embodiments, generating a pedal score based on multiple segments of the first pedal data and an external data source includes the following steps: 1) Calculating the degree of difference between the first pedal data and the standard pedal value (obtained from the external data source) (e.g., this can be reflected by a ratio); for example, if the first pedal data reflects that the user's pedal depth is 10% and the corresponding standard pedal depth is about 20%, then the degree of difference between the first pedal data and the standard pedal value can be calculated to be 50%, and the score can be 50 points. 2) Calculating the average degree of difference of multiple degree of difference under the first time period (e.g., one degree of difference can give one score), 3) The average degree of difference (i.e., the average of multiple scores) can be used as the pedal score.

[0082] In some embodiments, the pedal score can also be obtained from a preset score table, wherein the preset table includes the degree of difference between the first pedal data and the standard pedal value and the corresponding pedal score.

[0083] In some embodiments, if the foot pedal score is less than a first preset score, a guide trajectory can be generated based on an external data source.

[0084] Specifically, the guide trajectory can be generated by aligning the time stamps of a standard external data source with the actual pedal data, and generating a corresponding labeled trajectory based on the standard pedal values. Preferably, the guide trajectory is displayed using a sixth-label format.

[0085] In some embodiments, the guide trajectory can be displayed synchronously with the merged trajectory to facilitate comparison by the user and to guide the user.

[0086] In some embodiments, the step of generating a guide trajectory based on the external data source includes: Obtain the external data source corresponding to the second time period; A first guidance trajectory is generated based on the corresponding external data source; Obtain the labeled trajectory in the adjacent time period of the second time period; The guide trajectory is generated by fusing the labeled trajectory and the first guide trajectory using a trajectory fusion method.

[0087] Preferably, in some embodiments, when the pedal score is less than a first preset score but greater than a second preset score, a first guide trajectory of a first length is generated; when the pedal score is less than or equal to the second preset score, a first guide trajectory of a second length is generated, and the first length is less than the second length.

[0088] In this embodiment, guide tracks of different lengths are provided for different pedaling states. This allows for timely guidance when the user's pedaling state is poor. At the same time, the flexible adjustment of the length can avoid or reduce excessive external interference to the user while providing guidance, enabling them to practice pedaling more autonomously.

[0089] In some embodiments, a trajectory fusion method can be used to generate a guide trajectory by fusing the labeled trajectory and the first guide trajectory.

[0090] In some embodiments, generating a first guide trajectory based on the corresponding external data source may include the steps of: aligning the time stamp of the standard external data source with the actual pedal data, and generating a corresponding first guide annotation trajectory based on the standard pedal value.

[0091] It should be understood that the guidance trajectory generated by integrating real data (annotated trajectory) and standard data (first guiding annotated trajectory) is a dynamic trajectory based on the user's current level and gradually approaching the standard, rather than a static trajectory based entirely on standard data. Thus, it can serve a guiding purpose without being too difficult for users to implement.

[0092] Trajectory fusion can be achieved using existing technologies such as interpolation algorithms.

[0093] In some embodiments, the first time period and the second time period can be a performance time, such as ten minutes.

[0094] Please see Figure 3 The present invention also provides a performance data visualization system, comprising: A pedal data acquisition module is used to acquire at least one segment of pedal data from at least one pedal, wherein the pedal data includes: pedal value and a time stamp of the pedal value; The trajectory generation module is used to generate at least one trajectory based on the foot pedal data; wherein the trajectory includes a first simulation trajectory and a second simulation trajectory; correspondingly, the trajectory generation module is also used to: When the pedal value is greater than the first pedal threshold, a first simulated trajectory is generated based on the pedal value, and the first simulated trajectory is displayed using a first annotation format; When the pedal value is greater than the second pedal threshold and less than or equal to the first pedal threshold, a second simulated trajectory is generated based on the pedal value, and the second simulated trajectory is displayed using a second annotation format; The merged trajectory display module is used to generate a merged trajectory based on at least one of the marked trajectories and display the merged trajectory on the musical score.

[0095] It should be understood that the system can also be used to perform the steps described in any embodiment of the present invention.

[0096] Below, to more clearly illustrate the application scenarios of the present invention, an exemplary piano pedal system to which the present invention can be applied (see below) Figure 4 , Figure 5 , Figure 6 To introduce: A piano pedal system includes an external unit, an additional unit, and a control unit. The external unit is independently installed outside the piano, and the additional unit is installed on the piano. Both the external unit and the additional unit are electrically connected to the control unit.

[0097] Specifically, the control unit uses a microcontroller unit (MCU), which can be set independently or integrated into an external unit or added-on unit.

[0098] The external unit includes an external pedal module and a data acquisition module. The external pedal module includes at least one external pedal. The data acquisition module is used to collect the pedaling data of the external pedal. The control unit is used to control the installation unit to press the corresponding piano pedal according to the pedaling data.

[0099] The external pedal module also includes a mounting plate 111 and a mounting block 112 fixedly connected to the top of the mounting plate 111. The external pedal includes an external left pedal 113, an external middle pedal 114 and an external right pedal 115 that are hinged side by side to one side of the mounting block 112.

[0100] In some embodiments, buffer springs are connected between the external left pedal 113, the external middle pedal 114, and the external right pedal 115 and the mounting plate 111. The buffer springs not only buffer the user's stepping force on the external pedals to prevent collision between the external pedals and the mounting plate 111, but also help the external pedals return to their original position after the user releases the stepping force.

[0101] The data acquisition module includes an external position sensor, which is electrically connected to the control unit. The external position sensor is used to collect the pedal height data of the external left pedal 113 and the external right pedal 115 (in other embodiments of the present invention, it is also referred to as the pedal value or foot value). The control unit controls the mounting unit to press down the corresponding piano pedal 3 to the corresponding height based on the pedal height data.

[0102] Alternatively, in some embodiments, the data acquisition module may include a force feedback sensor instead, which is used to acquire the user's pedaling force (i.e., pedaling value or foot pressure).

[0103] Although both upright and grand pianos include three pedals (left, center, and right), only the left pedal (soft pedal) and the right pedal (sustain pedal) actually require grayscale control; the center pedal only needs to be controlled by switching it on and off. Therefore, in this embodiment, the addition unit includes two pedal drive modules, which are respectively set for the left and right pedals of the piano, to operate the left and right pedals according to the pedal height data of the external left pedal 113 and the external right pedal 115.

[0104] The pedal drive module includes a mounting bracket 211 and a pedal motor 212. The mounting bracket 211 is fixedly connected to the piano, and the pedal motor 212 is vertically fixedly connected to the mounting bracket 211 and electrically connected to the control unit. The pedal motor 212 is a linear motor, including a stator and a mover. The stator is cylindrical, and the mover is vertically slidably inserted into the stator. The stator is fixedly connected to the mounting bracket 211. The bottom end of the mover is connected to the corresponding piano pedal 3, driving the corresponding piano pedal 3 to move downwards to achieve pedal operation.

[0105] This embodiment abandons the traditional drive method of rotary motor + gear set / screw mechanism, and adopts a linear motor as the pedal motor 212. The stator and mover of the linear motor directly generate thrust through magnetic field interaction, so that the mover slides vertically in the stator without any intermediate mechanical reversal or deceleration. This modification eliminates the periodic noise and transmission backlash caused by gear meshing or screw friction. Therefore, when the pedal drive module presses the corresponding piano pedal according to the pedal height data, the mover can remain silent and respond sensitively during high-frequency reciprocating motion in the stator, reducing the response time from tens of milliseconds to sub-milliseconds.

[0106] A connecting rod 213 is coaxially fixed to the top of the mover, and a position sensor is fixedly connected to the top of the connecting rod 213. The position sensor is electrically connected to the control unit. The position sensor is used to detect the movement distance of the mover. If the detection result shows that the movement distance of the mover is the same as the pedal height data, the control unit controls the pedal motor 212 to turn off, stopping the pedaling of the piano pedal 3.

[0107] In this embodiment, the external position sensor and the added position sensor can employ proximity switches (such as inductive or capacitive non-contact switches) or oscillating switches (also known as lever limit switches or micro switches) to achieve a position detection accuracy of ±0.1mm: 1) The system can accurately reproduce the formation details of the user's pedaling action, achieving delicate control over pedal effects such as sustain and soft tones; 2) The pedaling drive response of the added unit is highly synchronized with the user's operation, enhancing the realism and realism of the performance. Furthermore, in this embodiment, the added position sensor is directly fixed to the top of the mover via the connecting rod 213. Compared to external or separate sensors, this integrated transmission + sensing structure not only shortens the signal transmission path and reduces wiring complexity but also avoids response lag and data drift caused by component misalignment or secondary installation errors, ensuring detection efficiency and accuracy.

[0108] In some embodiments, the height sensor, external position sensor, and additional position sensor may also employ a high-resolution linear encoder.

[0109] In some embodiments, the external position sensor and the added position sensor can be replaced by a force feedback sensor, or a combination of both can be used.

[0110] In some embodiments, the mounting unit further includes two linkage transmission modules, which are configured one-to-one with the pedal drive module. The linkage transmission module is connected between the output end of the pedal drive module and the corresponding piano pedal 3.

[0111] Specifically, the linkage transmission module includes a base plate 221 and a linkage mechanism 222 mounted on the base plate 221. The linkage mechanism 222 includes a transmission link 2221, a hinge support 2222, and a connecting bolt 2223. The hinge support 2222 is fixedly connected to the top of the base plate 221, and the non-end of the transmission link 2221 is hinged to the hinge support 2222. One end of the transmission link 2221 is fixedly connected to the bottom end of the linear guide rod via the connecting bolt 2223, and the other end of the transmission link 2221 is fixedly connected to the corresponding piano pedal 3 via the connecting bolt 2223.

[0112] The pedal motor 212 is used to drive one end of the transmission link 2221 to move upward, so that the other end of the transmission link 2221 drives the corresponding piano pedal 3 to move downward, thereby realizing the pressing of the piano pedal 3.

[0113] In some embodiments, the linkage transmission module further includes a damping spring 223, which is connected between the transmission link 2221 and the base plate 221, and is located between the hinge support 2222 and the piano pedal 3. The damping spring 223 not only buffers the force applied to the piano pedal 3 by the transmission link 2221 to prevent excessive force from causing structural collision damage, but also assists the transmission link 2221 in resetting after the force applied to the piano pedal 3 by the transmission link 2221 has ended.

[0114] In some embodiments, the transmission link 2221 has a square cross-section. A flange plate is fixedly connected to the bottom end of the linear guide rod. A square hole is opened on the flange plate. The end of the transmission link 2221 away from the piano pedal 3 is inserted into the square hole, and the transmission link 2221 is fixedly connected to the flange plate with bolts. The size of the square hole matches the cross-sectional size of the transmission link 2221, that is, the inner wall of the square hole is clearance-fitted with the outer wall of the transmission link 2221, which limits the transmission link 2221 and prevents it from rotating during use, thereby ensuring the accuracy of force transmission.

[0115] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0116] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a computer terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0117] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims. All of these forms are within the protection scope of the present invention.

Claims

1. A pedal data display method characterized by comprising: The method comprises the steps of: S101, acquiring at least one piece of pedal data of at least one pedal, and the pedal data comprises a pedal value and a time mark of the pedal value; S102, generating at least one piece of marked trajectory according to the pedal data; wherein the marked trajectory comprises a first simulation trajectory and a second simulation trajectory; correspondingly, S102 comprises the steps of: when the pedal value is greater than a first pedal threshold, generating the first simulation trajectory according to the pedal value, and the first simulation trajectory is displayed in a first marked form; when the pedal value is greater than a second pedal threshold and less than or equal to the first pedal threshold, generating the second simulation trajectory according to the pedal value, and the second simulation trajectory is displayed in a second marked form; S103, generating a merged trajectory according to at least one piece of the marked trajectory, and displaying the merged trajectory on a musical score.

2. The method of claim 1, wherein, The pedal value refers to a depth value, an angle value or a corresponding force value of the pedal changed by a user's down-pedal action and acquired by a pedal sensor.

3. The method of claim 1, wherein, The first marked form and the second marked form adopt different visual effects.

4. The method of claim 1, wherein, According to the pedal data, the marked trajectory is generated, comprising the steps of: converting the corresponding pedal value into coordinates in a pedal display graph according to the time mark; connecting the coordinates of a plurality of adjacent time marks into line segments; and generating a continuous marked trajectory according to a plurality of adjacent line segments.

5. The method of claim 1, wherein, The marked trajectory further comprises a virtual trajectory; correspondingly, S102 further comprises the steps of: when the pedal value is less than or equal to the second pedal threshold, identifying the pedal value as a virtual target; selecting an adjacent pedal value adjacent to the virtual target; wherein the difference between the time mark of the adjacent pedal value and the time mark of the virtual target is less than a preset time difference value; generating a virtual trajectory according to the adjacent pedal value and the virtual target, wherein the virtual trajectory is displayed in a third marked form.

6. The method of claim 5, wherein, According to the adjacent pedal value and the virtual target, the virtual trajectory is generated, comprising the steps of: (1) acquiring an adjacent pedal data group, and the adjacent pedal data group comprises first adjacent pedal data and second adjacent pedal data, and the first adjacent pedal data and the second adjacent pedal data are respectively the adjacent pedal values before and after the virtual target; (2) generating at least two predicted trajectories according to the adjacent pedal data group by using a virtual prediction method; wherein step (2) comprises: generating a virtual point by using the adjacent pedal data group, and the virtual point is used to define the starting height of a predicted trajectory; calculating a change trajectory of the adjacent pedal data group, and generating a virtual change trajectory according to the change trajectory; generating a predicted trajectory according to the starting height and the virtual change trajectory; (3) generating the virtual trajectory according to the at least two predicted trajectories.

7. The method of claim 6, wherein, Step (3) comprises: fusing the at least two predicted trajectories by using a trajectory fusion method to generate the virtual trajectory.

8. The method of claim 6, wherein, The second adjacent foot pedal data is obtained from an external data source, the external data source is collected through a music score database, and the external data source records a standard foot pedal value of at least one note node and a time label of the standard foot pedal value.

9. The method of claim 1, wherein, Further comprising steps of: generating a first auxiliary track according to the first foot pedal threshold, the first auxiliary track being displayed in a fourth annotation form; and / or generating a second auxiliary track according to the second foot pedal threshold, the second auxiliary track being displayed in a fifth annotation form.

10. A foot pedal data display system characterized by, Comprise: a foot pedal data acquisition module configured to acquire at least one piece of foot pedal data of at least one foot pedal, and the foot pedal data comprises a foot pedal value and a time label of the foot pedal value; an annotated track generation module configured to generate at least one annotated track according to the foot pedal data, wherein the annotated track comprises a first simulation track and a second simulation track, and the annotated track generation module is further configured to: generate the first simulation track according to the foot pedal value when the foot pedal value is greater than a first foot pedal threshold, and the first simulation track is displayed in a first annotation form; generate the second simulation track according to the foot pedal value when the foot pedal value is greater than a second foot pedal threshold and less than or equal to the first foot pedal threshold, and the second simulation track is displayed in a second annotation form; a merged track display module configured to generate a merged track according to the at least one annotated track, and display the merged track on a music score.

Images