3D musical score display method and system, and medium and program product

Abstract

The present invention relates to the technical field of intelligent display for pianos, and specifically relates to a 3D musical score display method and system, and a medium and a program product. The method comprises: acquiring at least one set of action capture results at a first time, wherein the action capture results include a first duration, a second duration or motion data, the action capture results are associated with a timestamp, the first duration is a time during which a finger executes a pressing action or generates a pressing tendency on a piano key, the second duration is the duration of collected effective sound, and the motion data is force applied to the piano key or a displacement generated when the piano key is pressed; and using a matching rule to associate the at least one set of action capture results with a target voice part, and adaptively displaying the associated action capture results at the position of a note. The present invention provides a solution for graphical display having visual smoothness for piano action capture results, so that users can quickly and intuitively capture key information from graphical display content.

Description

A method, system, medium, and program product for displaying 3D sheet music.

[0001] Priority application

[0002] This application claims priority to Chinese Invention Patent Application No. [202510091360.5], filed on January 20, 2025, entitled "[A Method and System for Displaying 3D Musical Scores]", which is incorporated herein by reference in its entirety. Technical Field

[0003] This invention relates to the field of intelligent piano display technology, specifically to a method, system, medium, and program product for displaying 3D sheet music. Background Technology

[0004] During piano playing, non-professionals typically have very limited ability to judge the final playing effect. This makes the piano learning and training process heavily reliant on the guidance of professional teachers, resulting in a very high learning cost. Currently, existing technologies offer some auxiliary devices that provide data-driven feedback on piano playing effects.

[0005] For example, patent application CN111968675A discloses a string instrument note comparison system based on hand recognition, including a display screen, a camera device, a hand key point recognition device, and a smart terminal. The display screen is communicatively connected to the smart terminal, the smart terminal is communicatively connected to the hand key point recognition device, and the hand key point recognition device is communicatively connected to the camera device. The smart terminal stores the teacher's MIDI audio digital signal. When using practice mode, the notes in the teacher's MIDI audio digital signal are played synchronously in real time. The received user's MIDI audio digital signal is compared with it. If the difference between the notes in the MIDI audio digital signal and the notes in the teacher's MIDI audio digital signal is less than an error threshold, a message indicating correct performance is displayed, and the next note in the teacher's MIDI audio digital signal continues to play. If the difference is greater than the error threshold, a message indicating a performance error is displayed, and the playback of the teacher's MIDI audio digital signal stops.

[0006] For example, patent application CN108597300A discloses a method and apparatus for displaying musical notes. The method includes: activating a timing function; listening to a first message generated by the triggering of a music control; determining a target note based on the first message, wherein the target note is the note associated with the triggered music control; determining whether the timing cycle has ended; if the timing cycle has not ended, storing the target notes in a target note list according to the order in which the target notes were determined; and if the timing cycle ends, displaying the notes in the target note list on a musical staff.

[0007] For example, patent application CN107767847A discloses an intelligent piano performance evaluation method and system, which collects the user's MIDI audio digital signal and converts it into performance data; matches the user's performance time sequence with the master's performance time sequence on the time axis; determines the pitch of the note groups played by the user and the master; determines the rhythm by comparing the relative time values ​​of the notes played by the user and the master; determines the tempo by calculating the relative tempo of the user and the master; determines the dynamics by comparing the dynamics of the notes played by the user and the master; comprehensively evaluates the performance pitch, rhythm, tempo, and dynamics; and marks the comprehensive evaluation results on the staff of the intelligent terminal.

[0008] However, the analysis and presentation of such professional data also places higher demands on beginners' data capture abilities. Therefore, there is an urgent need for a data feedback method that beginners can easily master. Summary of the Invention

[0009] The purpose of this invention is to provide a method and system for displaying 3D sheet music, which partially solves or alleviates the above-mentioned deficiencies in the prior art, enabling users to intuitively capture effective information from graphical data.

[0010] To solve the aforementioned technical problems, the present invention specifically adopts the following technical solution:

[0011] A first aspect of the present invention is to provide a method for displaying 3D sheet music, the sheet music comprising: a plurality of syllables, wherein each syllable is represented by at least one note, comprising the steps of:

[0012] S101, acquire at least one set of motion capture results at the first time, and the motion capture results include one or more of the following parameters: first duration, second duration, motion data, and the motion capture results are associated with a timestamp, wherein the first duration is the time when the finger performs a pressing action or generates a pressing tendency on the piano key, the second duration is the duration of the effective sound collected, and the motion data includes: the force exerted when the piano key is pressed, or the displacement generated when the piano key is pressed;

[0013] S102, at least one set of motion capture results is associated with a target voice part using a matching rule, the target voice part consisting of at least one syllable; wherein, the matching rule requires that the difference between the timestamp of the motion capture result and the target time of the target voice part is less than a first preset difference;

[0014] S103, when the motion capture result is associated with the target voice, the motion capture result is displayed at the corresponding position of the syllable.

[0015] In some embodiments, the method further includes the step of: S104 displaying the motion data using display rules; wherein, S104 includes:

[0016] S41, obtain at least one first dynamic of the first syllable in the target voice part, and determine a first line segment formed by the first target point on the score according to the at least one first dynamic, the first line segment being used to represent the dynamic change of the first syllable;

[0017] S42, obtain at least one second dynamic of the second syllable in the target voice part, and determine the second line segment formed by the second target point on the score according to the at least one second dynamic, the second line segment being used to represent the dynamic change of the second syllable;

[0018] S43, using an arc to connect the first line segment and the second line segment to form a target line segment, the target line segment being used to display the dynamic changes of the target sound part.

[0019] In some embodiments, S104 further includes:

[0020] Obtain the magnitude of the force corresponding to the target point;

[0021] The display color matching the force is queried according to the preset force display rules, wherein the force display rules preset different display colors for at least two different force values;

[0022] Multiple extended regions with a certain area are formed by extending outward from one side of the target line segment, and the multiple extended regions are connected to each other.

[0023] Obtain the display color of the target point corresponding to the extended region, and fill the extended region with the corresponding display color.

[0024] In some embodiments, S104 further includes the step of: when the intensity difference between the first target point and the second target point is greater than a first threshold, a transition region is provided between the two target points; wherein the first target point corresponds to a first display color, the second target point corresponds to a second display color, and the transition region is used to perform a color gradient between the first display color and the second display color.

[0025] In some embodiments, the method further includes: obtaining the peak position and / or trough position of the target line segment; providing a data display area at the peak position and / or the trough position; and displaying at least two of the parameters in the data display area in a predetermined order.

[0026] In some embodiments, it also includes:

[0027] Determine whether the length of the target line segment is less than the target length;

[0028] If so, a supplementary line segment is generated based on the target length and the length of the target line segment. The display intensity of the supplementary line segment is less than the display intensity of the target line segment. The display intensity includes one or more of the following attributes: color depth, width, and transparency.

[0029] A composite line segment is formed based on the supplementary line segment and the target line segment.

[0030] In some embodiments, it also includes:

[0031] The motion capture results are displayed through a data display area; this includes the following steps:

[0032] Determine whether the motion capture results match the reference performance results;

[0033] If so, the motion capture result is displayed using the first display style;

[0034] If not, the motion capture results will be displayed using the second display style.

[0035] The present invention also provides a 3D music score display system, wherein the music score includes: multiple syllables, and each syllable is represented by at least one note; the display system includes:

[0036] The acquisition module is used to acquire at least one set of motion capture results at a first time, and the motion capture results include one or more of the following parameters: a first duration, a second duration, and motion data, and the motion capture results are associated with a timestamp, wherein the first duration is the time when a finger performs a pressing action or generates a pressing tendency on the piano key, the second duration is the duration of the effective sound collected, and the motion data includes: the force exerted when the piano key is pressed, or the displacement generated when the piano key is pressed;

[0037] The association module is used to associate at least one set of motion capture results with a target voice part using matching rules, wherein the target voice part consists of at least one syllable;

[0038] The matching rule requires that the difference between the timestamp of the motion capture result and the target time of the target sound part be less than a first set difference.

[0039] The display module is used to display the motion capture result at the position of the syllable when the motion capture result is associated with the target voice.

[0040] In some embodiments, the display module is further configured to display the motion data using display rules; wherein, the display module includes:

[0041] The first line segment generation unit is used to obtain at least one first dynamic of the first syllable in the target voice part, and determine the first line segment formed by the first target point on the score according to the at least one first dynamic, wherein the first line segment is used to represent the dynamic change of the first syllable.

[0042] The second line segment generation unit is used to obtain at least one second dynamic of the second syllable in the target voice part, and to determine the second line segment formed by the second target point on the score according to the at least one second dynamic. The second line segment is used to represent the dynamic change of the second syllable.

[0043] A target line segment generation unit is used to connect the first line segment and the second line segment with an arc to form a target line segment, which is used to display the dynamic changes of the target sound part.

[0044] In some embodiments, the display module further includes:

[0045] A surface display unit is used to obtain the magnitude of the force corresponding to the target point; query the display color matching the force according to a preset force display rule, wherein the force display rule presets different display colors for at least two different magnitudes of force; extend outward from one side of the target line segment to form multiple extension areas with a certain area, and the multiple extension areas are connected; obtain the display color of the target point corresponding to the extension area, and fill the extension area with the corresponding display color.

[0046] The present invention also provides a computer-readable storage medium storing computer-executable instructions, wherein when a processor executes the computer-executable instructions, the method described in any of the embodiments is implemented. The present invention also provides a computer product comprising a computer program, wherein when the computer program is executed by a processor, it implements the steps of any of the methods as described in any of the embodiments.

[0047] Beneficial Technical Effects: This invention provides a 3D music score display method that dynamically and graphically displays the user's performance data on the music score. In particular, it can convert motion capture data into sound waves, allowing users to perceive the rhythm of the performance more intuitively.

[0048] Preferably, this invention employs a three-dimensional display method (or three-coordinated display method) that combines line display, surface display, and color difference display to graphically display fingering force. This provides a relatively comprehensive feedback on the key aspects of fingering changes (such as the magnitude and transition of force) and the trend of these changes. Simultaneously, this three-dimensional display method ensures smooth graphics while highlighting key points, conforming to the user's playing habits and allowing the user to more intuitively capture crucial information.

[0049] In response to the feedback needs of piano data, this invention proposes a method for smoothly displaying fingering data graphically. Attached Figure Description

[0050] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. In all the drawings, similar elements or parts are generally identified by similar reference numerals. The elements or parts in the drawings are not necessarily drawn to scale. Obviously, the drawings described below are some embodiments of the present invention, and those skilled in the art can obtain other drawings based on these drawings without any creative effort.

[0051] Figure 1 is a schematic diagram of the method flow in an exemplary embodiment of the present invention;

[0052] Figure 2 is a flowchart illustrating the process of generating the target line segment in an exemplary embodiment of the present invention;

[0053] Figure 3 is a schematic diagram of the display interface of the 3D musical score in the first exemplary embodiment of the present invention;

[0054] Figure 4 is a partial display schematic diagram of the 3D musical score in a second exemplary embodiment of the present invention;

[0055] Figure 5 is a schematic diagram of the display interface of the target line segment in the third exemplary embodiment of the present invention;

[0056] Figure 6 is a schematic diagram of the display interface of the target line segment in the fourth exemplary embodiment of the present invention;

[0057] Figure 7 is a schematic diagram of the system module in an exemplary embodiment of the present invention.

[0058] Attached image labels: 01 is a horizontal line, 02 is a subsection. Detailed Implementation

[0059] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0060] In this document, suffixes such as "module," "part," or "unit" used to denote elements are used only for the purpose of illustrative purposes and have no specific meaning in themselves. Therefore, "module," "part," or "unit" may be used interchangeably.

[0061] In this document, the terms "upper," "lower," "inner," "outer," "front," "rear," "one end," and "the other end," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing the present invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the present invention. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0062] In this document, unless otherwise explicitly specified and limited, the terms "installed," "equipped with," "connected," etc., should be interpreted broadly. For example, "connection" can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection, a direct connection, or an indirect connection through an intermediate medium; it can be a connection within two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0063] In this document, "and / or" includes any and all combinations of one or more of the listed related items. "Multiple" in this document means two or more, i.e., it includes two, three, four, five, etc. As used in this specification, the term "about" typically means + / -5% of the value, more typically + / -4%, more typically + / -3%, more typically + / -2%, even more typically + / -1%, even more typically + / -0.5% of the value.

[0064] In this specification, certain embodiments may be disclosed in a range-bound format. It should be understood that this "range-bound" description is merely for convenience and brevity and should not be construed as a rigid limitation on the disclosed range. Therefore, the description of a range should be considered as having specifically disclosed all possible subranges and the individual numerical values ​​within those ranges. For example, a description of the range 1-6 should be considered as having specifically disclosed subranges such as from 1 to 3, from 1 to 4, from 1 to 5, from 2 to 4, from 2 to 6, from 3 to 6, etc., and the individual numbers within those ranges, such as 1, 2, 3, 4, 5, and 6. This rule applies regardless of the breadth of the range.

[0065] In this article, "musical score" (or musical notation) refers to a regular combination of written symbols used to record musical pitch or rhythm. Common examples of musical scores include numbered musical notation and staff notation. Taking the piano as an example, the musical score used on the piano is usually staff notation. Staff notation is a method of recording music by marking notes of different durations and other symbols on five equally spaced parallel horizontal lines. Each line of the staff and the spaces between the lines are, from bottom to top, called the first line, second line, third line, fourth line, fifth line, first space, second space, third space, and fourth space. If there are not enough lines and spaces, additional lines and spaces can be added above or below the staff. These ledger lines and spaces are called ledger first line, ledger first space, ledger first line, ledger first space, etc., each representing a pitch. The fixed pitch of these pitches is determined by the clef used.

[0066] In musical notation, a syllable is the smallest unit of sound that can be produced independently, used to represent rhythm and melody. Syllables are represented by specific symbols in sheet music, such as notes, rests, and slurs. The musical characteristics of each syllable, such as its length, pitch, and dynamics, are specifically expressed through these symbols. The combination of these symbols constitutes a complete musical work. In particular, a single note constitutes one syllable.

[0067] In musical works with two or more melodies or two or more notes moving simultaneously, each line of melody or each note that constitutes a harmonic progression can be called a voice part. For example, each part of a four-part harmony is called a voice part.

[0068] In other words, syllables can be the basic elements that make up a voice part. Different combinations of syllables form different melodic lines, and thus different voice parts. Each voice part is composed of a series of orderly arranged syllables, which are arranged according to specific rhythms and pitches to form a unique melody.

[0069] A piano generally consists of keys (including white and black keys) and a metal soundboard. The range of an 88-key piano is generally from about 27.5 Hz to about 4186.01 Hz, while that of a 108-key piano is up to about 7902.13 Hz.

[0070] Due to the specialized nature of piano playing, piano learners often rely on in-person instruction from professional piano teachers to hone their skills. Therefore, learning piano is both challenging and costly. To address this, this invention proposes a graphical feedback scheme for piano performance, providing users (such as pianists) with an easy-to-understand and quick-to-capture feedback mode to quickly recognize their playing results and facilitate timely self-reflection. Referring to Figures 1-7, this invention provides a method for displaying sheet music.

[0071] Example 1

[0072] Referring to Figure 1, this invention provides a method for displaying 3D musical scores. Typically, the musical score includes multiple syllables, and each syllable is represented by at least one note. Correspondingly, the method includes the following steps:

[0073] S101, acquire at least one set of motion capture results at a first time, wherein the motion capture results include one or more of the following parameters: first duration, second duration, motion data, and the motion capture results are associated with a timestamp (for example, the generation time or occurrence time of each parameter can be recorded by a detection device). The first duration is the time for the finger to perform a pressing action or generate a pressing trend on the piano key, the second duration is the duration of the effective sound collected, and the motion data includes: the force exerted when the piano key is pressed (i.e., fingering force), or the displacement generated when the piano key is pressed. The motion capture results in this embodiment are also referred to as fingering data, which can perform data analysis and display of the user's fingering in different dimensions.

[0074] In some embodiments, motion data includes pitch, intensity, or volume.

[0075] In some embodiments, the first duration can be the time it takes for a finger to press a key on a piano key, and can be acquired by a detection device, such as a sensor. For example, a velocity sensor, displacement sensor, or acceleration sensor can be placed under the keys to detect the velocity, displacement, or acceleration of the corresponding keys in real time, thereby indirectly calculating the pressure applied to the keys. In some embodiments, the first duration is the time it takes for a finger to generate a pressing tendency on the keys. Correspondingly, at least one detection device, such as an image acquisition module (e.g., a camera), can be used to acquire an image of the subject's (or user's) hand, identify the degree of finger bending based on the hand image, and then determine whether a pressing tendency has been generated based on the degree of bending.

[0076] In some embodiments, the second duration can be acquired by a sound detection module (such as a sound sensor). Alternatively, in some embodiments, the second duration can be determined by detecting the time it takes for the hammer (or string hammer, sledgehammer) to strike the strings (correspondingly, a displacement sensor is provided on the hammer, or a force sensor is provided at its lower end to detect whether a striking action has occurred). In this case, the piano strikes the strings with the hammer, causing the strings to vibrate and produce sound. Of course, in other embodiments, sound production can also be measured by measuring the vibration of the strings. For example, the displacement sensor in this embodiment can be a grating tachometer.

[0077] S102, at least one motion capture result is associated with at least one syllable using a matching rule, wherein the matching rule requires that the difference between the timestamp of the motion capture result and the target time of the syllable is less than a first set difference.

[0078] The target time refers to the estimated time / sequence at which the syllable is produced (or played) based on the reference playing rhythm of the score. Specifically, the target time can be a preset time point based on professional experience, or it can be obtained based on historical performance data (such as performance data pre-stored by teachers, or piano pieces in a music library).

[0079] In other words, musical scores typically have standardized performance methods defined by music theory rules (such as relatively fixed pitches, durations, and order of notes, which can also be called reference performance data). Therefore, the user's performance data (i.e., motion capture data) and the reference performance data of the score can be matched from a time perspective to locate the user's performance points. Subsequently, the user's performance data can be displayed at the corresponding position on the score.

[0080] Preferably, step S102 involves: associating at least one set of motion capture results with a target voice part using a matching rule, wherein the target voice part consists of at least one syllable. The matching rule requires that the difference between the timestamp of the motion capture result and the target time of the target voice part is less than a first preset difference (in this case, the motion capture data is considered to match the target time).

[0081] In some embodiments, the target vocal part is a set of multiple syllables. Correspondingly, the target time of the target vocal part can also be the target sub-time of multiple syllables. When the difference between each motion capture result and the corresponding target sub-time is less than a first preset difference, the multiple motion capture results are considered to match the target time. That is, in this embodiment, it is preferable to perform correlation analysis between the set of multiple motion capture results and the musical score on a vocal part basis.

[0082] In other words, in this embodiment, the voice part can be a combination of one or more syllables. Matching by voice part helps improve the accuracy and reliability of the matching process. The specific length of the voice part can be customized by the user. It is understood that in some embodiments, existing sound matching techniques can be directly used to complete the matching; this invention does not limit this.

[0083] S103, when the motion capture result is associated with the target vocal part, the motion capture result (or fingering data) is displayed at the position of the syllable, that is, the fingering data is visualized.

[0084] For example, in some embodiments, displaying the motion capture result (or fingering data) at the location of the syllable includes providing a display box at the location of the syllable, the display box displaying the motion capture result.

[0085] Specifically, the display box can show numerical values ​​of the motion capture results and / or display icons associated with the motion capture results.

[0086] For example, different display icons can be provided for different performance effects, such as pitch, such as high and low pitch icons.

[0087] For example, depending on the performance level, when the motion capture result is completely consistent with the reference performance data, a reward icon can be displayed (such as the text icon "perfect" or a graphic icon, such as a smiley face). Alternatively, when the motion capture result collected at the corresponding syllable's vocalization time point deviates significantly from the reference performance data, an encouragement icon can be displayed.

[0088] In other words, in this embodiment, different types of display icons can be pre-stored for different types of motion capture data or different types of matching results. These display icons can be text icons or graphic icons.

[0089] In this embodiment, the target voice part is a set of multiple syllables that need to be played in the next time period. When the motion capture data sets corresponding to the multiple syllables are all matched with the target time of the target voice part, it is considered that the target voice part has successfully captured complete motion capture data.

[0090] In this embodiment, the voice parts can be predefined by the user before display, such as based on general music theory knowledge. Preferably, this embodiment uses voice parts as the unit for data collection and matching, which can improve the accuracy and reliability of the data.

[0091] It should be noted that, typically, the order in which the user plays the notes is the same as the order in which the notes in the sheet music are played. Therefore, the focus can be on matching the motion capture data with the notes in the sheet music based on the temporal sequence. Alternatively, in some embodiments, the matching and localization can be achieved by combining the numerical values ​​of the motion capture data (such as pitch and duration) with the time.

[0092] In one exemplary embodiment, the matching process may include: acquiring motion capture data of the user performing a piece of music, which may include pitch and duration; obtaining the sheet music of the piece, which pre-records note information, namely the pitch and duration of the notes.

[0093] When the user's motion capture data matches the note information, the user's current performance node can be located.

[0094] Specifically, when a user's motion capture data matches the note information (e.g., the same pitch and similar duration), and the timestamp of the motion capture data matches the appearance time of the corresponding note information in the score, then the motion capture data is associated with the note information.

[0095] In some embodiments, the time value can be represented by a second duration.

[0096] In some embodiments, "similar time values" means that the difference between two time values ​​is less than a set time value difference.

[0097] Preferably, in some embodiments, motion capture data of at least two notes are used as the smallest unit for matching.

[0098] It is worth noting that, unlike the intelligent piano performance evaluation method and system disclosed in patent application CN107767847A, this invention focuses on fingering data as the primary data source and displays the dynamic and time dimensions of the fingering data in a hierarchical manner. The dynamic dimension is preferably visualized graphically, while the time dimension is adapted to be displayed graphically, meaning that data points are displayed in a limited manner at key points in the graphical representation. This hierarchical display, in conjunction with the technical aspects, provides users with an interface display scheme that facilitates quick and easy capture of key information. The hierarchical display scheme provided by this invention will be explained exemplarily below.

[0099] In some embodiments, the steps further include:

[0100] S104 uses display rules to display the motion data. These display rules can be used to convert multiple finger pressure data into a smooth curve. Referring to Figure 2, S104 includes:

[0101] S41, acquire at least one first dynamic (of the first syllable in the target voice part), and determine a first line segment formed by the first target point on the score according to the at least one first dynamic, wherein the first line segment is used to represent the dynamic change of the first syllable;

[0102] S42, acquire at least one second dynamic of the second syllable (in the target voice or in another target voice), and determine a second line segment formed by the second target point on the score based on the at least one second dynamic, the second line segment being used to represent the dynamic change of the second syllable.

[0103] In this system, a target point is associated with a dynamic measurement data point (such as first dynamic or second dynamic, etc.), and its height position on the score is used to reflect the fingering dynamic at that moment.

[0104] S43, connect the first line segment and the second line segment to form the target line segment.

[0105] Understandably, the above display rules can be applied to other types of motion data, such as pitch or volume. In other words, the display rules can also be used to convert multiple pitch or volume data into a smooth curve.

[0106] The target line segment fluctuates with the magnitude of the motion data, which can produce a visual effect similar to waves, hence this article also refers to it as a sound wave.

[0107] Preferably, in some embodiments, S43 is: using an arc to connect the first line segment and the second line segment to form a target line segment, the target line segment being used to display the intensity change of the target sound part.

[0108] Specifically, referring to Figure 5, which shows a schematic diagram of the staff display interface, the horizontal line 01 in the staff is displayed on the first layer. Further, the target line segment L1 is displayed on the second layer above the first layer. The exemplary formation process of the target line segment L1 is as follows: The first line segment l1 is formed based on multiple dynamic point values ​​(equivalent to multiple first dynamics) of a syllable (which can be equivalent to the first syllable); the second line segment l2 is formed based on multiple dynamic point values ​​(equivalent to multiple second dynamics) of another syllable (which can be equivalent to the second syllable). Subsequently, the first and second line segments are connected by a third line segment l3 (such as an arc) to form the target line segment L1.

[0109] In some embodiments, the above display rules can be used to connect one target sound part into a smooth curve, or the display rules can also be used to connect two or more target sound parts into a smooth curve.

[0110] In some embodiments, a syllable corresponds to one or more force values ​​(which can be acquired by velocity, displacement, or acceleration sensors). Preferably, one force can correspond to one target point, and a continuous curve (or line segment) can be fitted based on two or more target points.

[0111] In some embodiments, the target point may correspond to the pronunciation position of the core syllable. Alternatively, in some embodiments, the target point may be a key transition point, such as a position where factors like intensity, volume, and pitch change significantly.

[0112] In other words, in some embodiments, motion data may include: force, pitch, or volume. The rules for displaying pitch and volume can refer to the rules for displaying force, and will not be elaborated here.

[0113] For example, in some embodiments, when there is a certain gap between the first line segment and the second line segment, an arc can be used to connect the two to improve the smoothness of the force display effect.

[0114] For example, in some embodiments, the arc can be a line segment corresponding to the third syllable between the first and second syllables.

[0115] In some embodiments, the display height of a line segment (or target point) on the staff is determined based on the magnitude of the intensity. Specifically, different positions on the staff are pre-marked to represent different intensities (or, different positions are pre-set to represent different intensities), for example, the higher the intensity, the higher the target point is on the staff.

[0116] It is understood that in some embodiments, the force collected can be the force point values ​​collected by the sensor at multiple times. Correspondingly, the present invention uses a continuous curve to smoothly visualize the point value data.

[0117] It is understood that this embodiment can display motion capture data in a relatively intuitive way, such as displaying a display box on a musical score (correspondingly, the method in this embodiment may include steps S101-S103). Alternatively, this embodiment can also directly display motion capture data in a sound wave-like graphic (correspondingly, the method in this embodiment may include steps S101, S102 and S104).

[0118] Furthermore, the present invention employs a three-pronged approach—line display, surface display, and color difference—to highlight the pressure applied by the finger. For example, in some embodiments, S104 further includes:

[0119] Obtain the magnitude of the force corresponding to the target point;

[0120] The display color matching the force is queried according to the preset force display rules, wherein the force display rules preset different display colors for at least two different force values;

[0121] Multiple extended regions with a certain area are formed by extending outward from one side of the target line segment, and the multiple extended regions are connected to each other.

[0122] Obtain the display color of the target point corresponding to the extended region, and fill the extended region with the corresponding display color.

[0123] For example, in some embodiments, as shown in Figure 3, the relationship between the magnitude of the dynamics and the displayed color is schematically shown in the upper right corner of the musical score.

[0124] In some embodiments, S104 further includes the step of:

[0125] When the intensity difference between the first target point and the second target point is greater than a first threshold (and preferably less than or equal to a second threshold), a transition region is provided between the first target point and the second target point; wherein the first target point corresponds to a first display color, the second target point corresponds to a second display color, and the transition region is used to perform a color gradient between the first display color and the second display color. The color gradient is used to improve the smoothness of the sound wave display.

[0126] In this embodiment, a three-dimensional display method (or three-coordinated display method) combining line display, surface display, and color difference display is used to graphically display fingering force. This provides a relatively comprehensive feedback on the key points of fingering changes (such as the magnitude and transition of force) and the trend of change. At the same time, this three-dimensional display method ensures that the graphics are smooth while highlighting the key points, conforming to the user's playing habits, so that the user can more intuitively capture the key information.

[0127] Alternatively, in some embodiments, as shown in the right section of Figure 4, when the force difference between the first target point and the second target point is greater than the second threshold, the first target point and the second target point are represented by two target line segments respectively, and the two target line segments are disconnected.

[0128] In this embodiment, areas of abrupt changes in intensity are differentiated using broken curves. This allows the sound waves to visually highlight these abrupt changes, enabling users to intuitively capture more realistic rhythmic shifts.

[0129] In some embodiments, the method further includes: obtaining the peak position and / or trough position of the target line segment; and providing a display area, such as a data display area and / or a graphic display area, at the peak position and / or the trough position.

[0130] At least two of the parameters are displayed in a predetermined order in the data display area. Alternatively, display icons associated with the motion capture results are shown in the graphics display area.

[0131] In some embodiments, the data display area may be a data display frame. In some embodiments, the data display area and the graphics display area may be combined into a single display area / frame.

[0132] For example, as shown in Figure 4, the intensity, first duration, and second duration can be displayed sequentially in the data display area.

[0133] Of course, in other embodiments, the number or order of parameters displayed in the data display area can be customized by the user.

[0134] It is worth noting that the hierarchical display of fingering data proposed in this invention has at least two implications: 1) The emphasis on displaying fingering force data is greater than its duration, meaning that fingering force is prominently displayed on the interface using graphics (or even graphics overlaid with data); 2) Data of different dimensions can be displayed in layers on the interface. For example, the staff can be the first layer (i.e., the bottom layer), while the graphics (i.e., the force curve) are placed on the second layer above the first layer. Furthermore, the data display area is placed on the third layer above the second layer. Moreover, this invention employs a coordinated and differentiated design in the display format and hierarchy of data of different dimensions. This allows for a more comprehensive display of fingering data while highlighting key trends, enabling users to more intuitively glean important information from the interface.

[0135] Furthermore, the line segment display format adopted in this invention is more in line with the user's abstract understanding of musical scores, thus making it more conducive to presenting key visual information to the user. At the same time, by displaying fingering data in a hierarchical manner, it is possible to highlight key information in the fingering data (such as dynamic values ​​and dynamic change trends), and to display multi-dimensional information smoothly, so as to avoid multi-dimensional information from causing visual confusion and increasing the difficulty of data capture for the user.

[0136] In some embodiments, it also includes:

[0137] Determine whether the length of the target line segment is less than the target length;

[0138] If so, a supplementary line segment is generated based on the target length and the length of the target line segment. The display intensity of the supplementary line segment is less than the display intensity of the target line segment. The display intensity includes one or more of the following attributes: color depth, width, and opacity.

[0139] A new target line segment (or composite line segment) is formed based on the supplementary line segment and the target line segment.

[0140] For example, in some embodiments, the supplementary line segment may have a certain degree of transparency, while the target line segment can be displayed normally. That is, the transparency of the supplementary line segment is greater than that of the target line segment.

[0141] For example, in some embodiments, referring to Figure 6, two vertical lines are used to divide a measure 02, and the target length can refer to the length of a measure 02 in the musical score. Preferably, as shown in Figures 3 and 6, when there is a large difference between the target line segment formed by the syllable and the measure, a supplementary line segment is used to coordinate with the target line segment for simultaneous display of real and virtual elements. This supplementary real and virtual display method can increase the smoothness of the graphics without increasing the complexity of the display, so that the display effect of the graphics conforms to the user's abstract thinking and facilitates the user's intuitive capture of information from the graphics.

[0142] Referring to Figure 6, a composite line segment L2 is formed by the target line segment l4 and the supplementary line segment l5. If the height difference (or strength difference) between two adjacent target points in target line segment l4 and target line segment L3 is greater than a second preset difference, then the two target line segments are disconnected. Furthermore, as shown in Figures 3-4, this disconnection method, combined with the composite line, can mitigate the impact of steep descent areas on the visual smoothness of the graphic.

[0143] For example, in some embodiments, the target line segment that has a sound-producing function is displayed as a solid line, and the supplementary line segment that does not have a sound-producing function is displayed as a transparent line (or dashed line), wherein the transparent line refers to a line segment whose transparency is much lower than that of the target line segment.

[0144] Correspondingly, the display depth (such as color depth or opacity) of at least one point in the supplementary line segment can be gradually reduced along the direction that it moves away from the target line segment in order to distinguish between the real and the virtual parts.

[0145] Furthermore, to further simplify the information capture process, this method also includes the following steps:

[0146] Get the number of peaks (or troughs) under a set length (such as the length of a section);

[0147] When the quantity is less than the first quantity threshold, a second display method is used to display a display area (such as a data display area) at the peak (or trough) position; the second display method allows the display of the first quantity of parameters within the display area; furthermore, the parameter types allowed to be displayed by the second display method include one or more of the following: intensity, first duration, and second duration;

[0148] When the quantity is greater than or equal to the first quantity threshold, a third display method is used to display the data display area at the peak (or trough) position, or the data display area is hidden; wherein, the third display method allows a second quantity of parameters to be displayed within the data display area, and the first quantity is greater than the second quantity. Furthermore, the third display method allows fewer parameter types to be displayed than the second display method.

[0149] For example, in some embodiments, when a measure is relatively easy to play or has a relatively slow rhythm, a second display method is preferably used to display the data display area on the score, such as simultaneously displaying dynamics, first duration, and second duration. As another example, when a measure is difficult to play (e.g., fluctuations in fingering dynamics), a third display method is preferably used to display the data display area on the score, such as displaying only dynamics data. In this embodiment, the data content is preferably displayed using the fluctuation density of line segments.

[0150] It should be noted that different terms are used in this document to refer to the display format of data, such as "the Xth display method" and "the Xth display style." This is mainly to facilitate the distinction between different display systems in different embodiments. In the embodiments of the present invention, the selection of the display method is mainly to facilitate users to clearly view the differences in data. Any display method in the display system that can achieve this purpose can be applied to the present invention and should be within the protection scope of the present invention.

[0151] In some embodiments, the method further includes:

[0152] The motion capture results are displayed through a data display area; this includes the following steps:

[0153] Determine whether the motion capture result matches the reference performance result (or reference performance data);

[0154] Specifically, in some embodiments, the reference performance results can be preset by the user. For example, the reference performance results can be performance data pre-stored by the teacher, or they can be obtained from an existing publicly available music library. For instance, these reference performance results can be performance data from professional pianists. Specifically, in some embodiments, the reference performance results can include one or more of the following: a first duration, a second duration, and motion data.

[0155] If yes, the motion capture result is displayed using the first display style; otherwise, the motion capture result is displayed using the second display style.

[0156] In some embodiments, the first display mode and the second display mode have different display attributes, including one or more of the following: color, brightness, display area, and display order.

[0157] In some embodiments, the steps further include:

[0158] Calculate the degree of difference between the motion capture results and the reference performance results;

[0159] For example, the difference can be the difference between a target line segment (such as dynamic value) formed based on the user's motion capture data and a target line segment formed based on the reference performance result. For example, the difference can be defined by the length of the mismatched area between the two line segments.

[0160] Alternatively, the degree of difference can also be defined by the frequency of performance errors. For example, if the difference between at least one parameter between the motion capture result and the reference performance result is greater than a third set difference, the data can be recorded as a performance error.

[0161] When the difference is less than the first difference, the first sub-display method is used to display the performance result of the score.

[0162] Furthermore, when the difference is greater than or equal to the first difference, a second sub-display method can be used to display the performance result of the score; wherein the first sub-display method and the second sub-display method have different display attributes.

[0163] In some embodiments, the force is the maximum or average force over a second time period.

[0164] In some embodiments, the steps further include:

[0165] Retrieve the performance characteristics of at least one measure; wherein, a performance characteristic may be pre-set with a technique tag, and the technique tag is also associated with the display priority of at least one parameter; wherein the content of the technique tag may be preset by the user based on music theory knowledge;

[0166] The motion capture results are displayed adaptively based on the skill tags.

[0167] For example, in some embodiments, the technique labels may include one or more of the following: Staccato: For scores marked with staccato, the performer needs to master the staccato technique, which involves keeping the fingers close to the keys, applying force with the fingertips, striking the keys quickly, and producing a focused, clear, and resonant sound. The emphasis is on finger dexterity and rebound force. Legato: For sections of score that require slurs or legato, the performer needs to use legato techniques to connect the notes closely. The emphasis is on smooth finger touch and continuous playing. Staccato: For sections of score that require breaking up each note, the performer needs to use staccato techniques, playing each note separately. The emphasis is on the natural lifting and lowering of the arm and the force of the fingertips striking the keys.

[0168] In some embodiments, the display area, display brightness, or display order of the parameter with the highest priority is higher than other parameters.

[0169] From another perspective, this embodiment provides a 3D music score display method, including the following steps:

[0170] A musical score is provided, the score consisting of at least one target voice part, and one target voice part comprising at least one syllable;

[0171] Obtain at least one set of motion capture results that match the syllable; the motion capture results include: a type of index, and the type of index includes: pitch or intensity;

[0172] The motion capture results are displayed on the score according to the display rules.

[0173] The step of displaying the motion capture result on the musical score using display rules includes:

[0174] For at least two syllables, at least two line segments are fitted using the aforementioned index;

[0175] Connect at least two of the line segments to form a target line segment, which is used to demonstrate the performance variation (or rhythmic variation) of the target voice.

[0176] In some embodiments, one type of metric may include pitch, velocity, or volume.

[0177] In some embodiments, the step of fitting at least two line segments to at least two syllables using the one type of index includes:

[0178] Obtain at least one first index value (such as a dynamic value) of the first syllable, and determine a first line segment formed by the first target point on the score based on the at least one first index, wherein the first line segment is used to represent the index change of the first syllable, such as a dynamic change or a pitch change.

[0179] Obtain at least one second index value for the second syllable, and determine a second line segment formed by the second target point on the musical score based on the at least one second index value, wherein the second line segment is used to represent the index value change of the second syllable;

[0180] Connect the first line segment and the second line segment to form the target line segment.

[0181] In some embodiments, the method further includes: color rendering of the target line segment, which includes the steps of:

[0182] Obtain the index value corresponding to the target point;

[0183] The display color matching the indicator value is queried according to the preset indicator display rules, wherein the indicator display rules preset different display colors for at least two indicators of different sizes;

[0184] Multiple extended regions with a certain area are formed by extending outward from one side of the target line segment, and the multiple extended regions are connected to each other.

[0185] Obtain the display color of the target point corresponding to the extended region, and fill the extended region with the corresponding display color.

[0186] It is understood that the intensity in the intensity display rules of the above embodiments (such as the formation of target line segments or the process of color filling) can be replaced by other indicators, such as pitch, volume, etc., which will not be elaborated here.

[0187] Furthermore, in some embodiments, the motion capture result includes: a first type of indicator, which includes pitch or intensity, and a second type of indicator, which includes a first duration, a second duration, a time value, or volume. Correspondingly, any one or more of the above-mentioned indicators can be visualized in the data display box.

[0188] In particular, in some embodiments, the second delay time is the time value. In some embodiments, the time value may be the duration of the note's sound as measured by a sound detection device.

[0189] Example 2

[0190] Referring to Figure 7, the present invention also provides a 3D music score display system, wherein the music score includes: multiple syllables, and each syllable is represented by at least one note; the display system includes:

[0191] The acquisition module 10 is used to acquire at least one set of motion capture results at a first time, and the motion capture results include one or more of the following parameters: a first duration, a second duration, and motion data, and the motion capture results are associated with a timestamp, wherein the first duration is the time when a finger performs a pressing action or generates a pressing tendency on the piano key, the second duration is the duration of the effective sound collected, and the motion data includes: the force exerted when the piano key is pressed, or the displacement generated when the piano key is pressed;

[0192] The association module 20 is used to associate at least one set of the motion capture results with a target voice part using matching rules, wherein the target voice part consists of at least one of the syllables;

[0193] The matching rule requires that the difference between the timestamp of the motion capture result and the target time of the target sound part be less than a first set difference.

[0194] The display module 30 is used to display the motion capture result at the position of the syllable when the motion capture result is associated with the target voice.

[0195] In some embodiments, the display module is further configured to display the motion data using display rules; wherein, the display module includes:

[0196] The first line segment generation unit is used to obtain at least one first dynamic of the first syllable in the target voice part, and determine the first line segment formed by the first target point on the score according to the at least one first dynamic, wherein the first line segment is used to represent the dynamic change of the first syllable.

[0197] The second line segment generation unit is used to obtain at least one second dynamic of the second syllable in the target voice part, and to determine the second line segment formed by the second target point on the score according to the at least one second dynamic. The second line segment is used to represent the dynamic change of the second syllable.

[0198] A target line segment generation unit is used to connect the first line segment and the second line segment with an arc to form a target line segment, which is used to display the dynamic changes of the target sound part.

[0199] In some embodiments, the display module further includes:

[0200] A surface display unit is configured to: acquire the magnitude of the force corresponding to the target point of the target line segment; query the display color matching the force according to a preset force display rule, wherein the force display rule presets different display colors for at least two different magnitudes of force; extend outward from one side of the target line segment to form multiple extension areas with a certain area, and the multiple extension areas are connected; acquire the display color of the target point corresponding to the extension area, and fill the extension area with the corresponding display color.

[0201] In some embodiments, the display module further includes:

[0202] A transition unit is used to set a transition region between two target points when the intensity difference between a first target point and a second target point is greater than a first threshold; wherein the first target point corresponds to a first display color and the second target point corresponds to a second display color, and the transition region is used to perform a color gradient between the first display color and the second display color.

[0203] In some embodiments, the display system further includes: a parameter display unit, configured to acquire the peak position and / or trough position of the target line segment; provide a data display area at the peak position and / or the trough position; and display at least two of the parameters in the data display area in a predetermined order.

[0204] In some embodiments, the display system further includes: a supplementary display unit, configured to determine whether the length of the target line segment is less than the target length; if so, generate a supplementary line segment based on the target length and the length of the target line segment, wherein the display intensity of the supplementary line segment is less than the display intensity of the target line segment, and the display intensity includes one or more of the following attributes: color depth, width, and transparency; and form a composite line segment based on the supplementary line segment and the target line segment.

[0205] In some embodiments, the system further includes: a difference display unit, configured to display the motion capture result through a data display area; specifically, configured to determine whether the motion capture result matches a reference performance result; if so, display the motion capture result using a first display style; if not, display the motion capture result using a second display style.

[0206] In some embodiments, the “voice” used in this invention can be customized by the user, that is, the length of the voice (such as the number of syllables it contains) can be customized by the user. For example, the number of syllables can be increased or decreased according to the difficulty of the musical score, the accuracy of the data analysis, and other requirements.

[0207] From another perspective, this embodiment is equivalent to providing a 3D music score display system, including:

[0208] A music score display module is used to provide a music score, the music score consisting of at least one target voice part, and one target voice part including at least one syllable; an acquisition module is used to acquire at least one set of motion capture results matching the syllable; the motion capture results include: a type of index, and the type of index includes: pitch or dynamics; a motion capture display module is used to display the motion capture results on the music score according to display rules.

[0209] The motion capture display module includes: a fitting unit for fitting at least two line segments for at least two syllables using the aforementioned index; and a connection unit for connecting the at least two line segments to form a target line segment, which is used to display the performance variation effect (or rhythm variation) of the target vocal part.

[0210] Similarly, the intensity in the intensity display rules and color filling rules in this embodiment can be replaced by other types of indicators, which will not be elaborated here.

[0211] The present invention also provides an electronic device, including a processor and a memory; wherein: the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory, causing the processor to perform the method described in any of the embodiments. The present invention also provides a computer-readable storage medium storing computer-executable instructions, wherein when a processor executes the computer-executable instructions, the method described in any of the embodiments is implemented. The present invention also provides a computer product, including a computer program, wherein when the computer program is executed by a processor, it implements the steps of any method as described in any of the embodiments.

[0212] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0213] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a computer terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0214] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims. All of these forms are within the protection scope of the present invention.

Claims

1. A method for displaying 3D musical scores, characterized in that, The musical score includes: multiple syllables, and each syllable is represented by at least one note, including the following steps: S101, acquire at least one set of motion capture results at the first time, and the motion capture results include one or more of the following parameters: first duration, second duration, motion data, and the motion capture results are associated with a timestamp, wherein the first duration is the time when the finger performs a pressing action or generates a pressing tendency on the piano key, the second duration is the duration of the effective sound collected, and the motion data includes: the force exerted when the piano key is pressed, or the displacement generated when the piano key is pressed; S102, using matching rules to associate at least one set of the motion capture results with the target voice part, the target voice part consisting of at least one of the syllables; The matching rule requires that the difference between the timestamp of the motion capture result and the target time of the target sound part be less than a first set difference. S103, when the motion capture result is associated with the target voice, the motion capture result is displayed at the corresponding position of the syllable.

2. The method for displaying 3D sheet music according to claim 1, characterized in that, The method includes the following steps: S104, displaying the motion data according to display rules; wherein, S104 includes: S41, obtain at least one first dynamic of the first syllable in the target voice part, and determine a first line segment formed by the first target point on the score according to the at least one first dynamic, wherein the first line segment is used to represent the dynamic change of the first syllable; S42, obtain at least one second dynamic of the second syllable in the target voice part, and determine the second line segment formed by the second target point on the score according to the at least one second dynamic, the second line segment being used to represent the dynamic change of the second syllable; S43, the first line segment and the second line segment are connected by an arc to form a target line segment, which is used to display the dynamic changes of the target sound part; And / or; the method includes: The motion capture results are displayed through a data display area; this includes the following steps: Determine whether the motion capture results match the reference performance results; If so, the motion capture result is displayed using the first display style; If not, the motion capture results will be displayed using the second display style.

3. The method for displaying 3D sheet music according to claim 2, characterized in that, S104 also includes: Obtain the magnitude of the force corresponding to the target point of the target line segment; The display color matching the force is queried according to the preset force display rules, wherein the force display rules preset different display colors for at least two different force values; Multiple extended regions with a certain area are formed by extending outward from one side of the target line segment, and the multiple extended regions are connected to each other. Obtain the display color of the target point corresponding to the extended region, and fill the extended region with the corresponding display color.

4. The method for displaying 3D musical scores according to claim 3, characterized in that, S104 also includes the following steps: When the intensity difference between the first target point and the second target point is greater than a first threshold, a transition area is set between the two target points; wherein, the first target point corresponds to a first display color, the second target point corresponds to a second display color, and the transition area is used to perform a color gradient between the first display color and the second display color; And / or, the method further includes: Determine whether the length of the target line segment is less than the target length; If so, a supplementary line segment is generated based on the target length and the length of the target line segment. The display intensity of the supplementary line segment is less than the display intensity of the target line segment. The display intensity includes one or more of the following attributes: color depth, width, and transparency. A composite line segment is formed based on the supplementary line segment and the target line segment.

5. The method for displaying 3D musical scores according to claim 4, characterized in that, Also includes: Obtain the peak and / or trough positions of the target line segment; A data display area is provided at the peak and / or trough locations; At least two of the parameters are displayed in the data display area in a set order.

6. A 3D music score display system, characterized in that, The musical score includes multiple syllables, and each syllable is represented by at least one note. The display system includes: The acquisition module is used to acquire at least one set of motion capture results at a first time, and the motion capture results include one or more of the following parameters: a first duration, a second duration, and motion data, and the motion capture results are associated with a timestamp, wherein the first duration is the time when a finger performs a pressing action or generates a pressing tendency on the piano key, the second duration is the duration of the effective sound collected, and the motion data includes: the force exerted when the piano key is pressed, or the displacement generated when the piano key is pressed; The association module is used to associate at least one set of motion capture results with a target voice part using matching rules, wherein the target voice part consists of at least one syllable; The matching rule requires that the difference between the timestamp of the motion capture result and the target time of the target sound part be less than a set difference. The display module is used to display the motion capture result at the position of the syllable when the motion capture result is associated with the target voice.

7. The 3D music score display system according to claim 6, characterized in that, The display module is further configured to display the motion data using display rules; wherein, the display module includes: The first line segment generation unit is used to obtain at least one first dynamic of the first syllable in the target voice part, and determine the first line segment formed by the first target point on the score according to the at least one first dynamic, wherein the first line segment is used to represent the dynamic change of the first syllable. The second line segment generation unit is used to obtain at least one second dynamic of the second syllable in the target voice part, and to determine the second line segment formed by the second target point on the score according to the at least one second dynamic. The second line segment is used to represent the dynamic change of the second syllable. A target line segment generation unit is used to connect the first line segment and the second line segment with an arc to form a target line segment, which is used to display the dynamic changes of the target sound part.

8. A 3D music score display system according to claim 7, characterized in that, The display module further includes: A surface display unit is configured to: acquire the magnitude of the force corresponding to the target point of the target line segment; query the display color matching the force according to a preset force display rule, wherein the force display rule presets different display colors for at least two different magnitudes of force; extend outward from one side of the target line segment to form multiple extension areas with a certain area, and the multiple extension areas are connected; acquire the display color of the target point corresponding to the extension area, and fill the extension area with the corresponding display color.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, wherein when a processor executes the computer-executable instructions, it implements the method as described in any one of claims 1-5.

10. A computer product, characterized in that, Includes a computer program, wherein the computer program, when executed by a processor, implements the method as described in any one of claims 1-5.

Images