Music data processing method and device

Abstract

The invention provides a music data processing method and device, electronic equipment, a non-instantaneous computer readable storage medium and a computer program product, and belongs to the technical field of music data processing. The method comprises the following steps: acquiring music data; marking the music data as a predetermined music structure according to a first preset rule; wherein the preset music structure comprises at least one of music segments, music sentences, music knots and phonetic types; marking the preset music structures as preset attributes according to a second preset rule, and marking the relationship between the preset music structures to form marked music data; according to the method, the annotated music data is stored in the corresponding data table of the database, the problem that annotation of the music data cannot meet training data requirements in the prior art is solved, and the quality and efficiency of annotation of the music data are improved, so that the training data requirements are better met, and data support with annotations is provided for subsequent task research.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a music data processing method, apparatus, electronic device, non-transitory computer-readable storage medium, and computer program product. Background Technology

[0002] In the field of AI, especially machine learning and deep learning, models require a large amount of data for training. Annotated music data provides information about musical attributes such as melody, rhythm, harmony, and style, which can be used to train models to recognize and generate music. The accuracy and diversity of labeled data are crucial for training efficient and accurate AI models.

[0003] Currently, in the music production field, some record companies have launched management software for labeling macro-level attributes such as music genres, styles, composers, and arrangers. This software can export labeled metadata, such as Rebeat Digital and ArmadaMusic. However, this labeling method only labels macro-level attributes. These small-scale attribute descriptions cannot meet the requirements for training data, and therefore cannot achieve effective audio processing by artificial intelligence.

[0004] Therefore, there is an urgent need for an innovative method for music data annotation to overcome the above problems. Summary of the Invention

[0005] In view of this, the technical problem to be solved by the present invention is to provide a music data processing method, apparatus, electronic device, non-transitory computer-readable storage medium and computer program product, so as to solve the problem that the annotation of music data in the prior art cannot meet the requirements of training data.

[0006] To address the aforementioned technical problems, specific embodiments of the present invention provide a music data processing method, the method comprising:

[0007] Acquire music data;

[0008] The music data is labeled as a predetermined music structure according to a first preset rule; wherein the predetermined music structure includes at least one of the following: musical phrase, musical sentence, musical unit, musical pattern;

[0009] According to the second preset rule, the predetermined music structure is labeled as a preset attribute, and the relationship between the predetermined music structures is labeled to form labeled music data;

[0010] The labeled music data is saved to the corresponding data table in the database.

[0011] Optionally, labeling the music data as a predetermined music structure according to a first preset rule includes:

[0012] Select a parent node in the tree structure; wherein the music data contains a music structure that corresponds to the tree structure;

[0013] When the parent node corresponds to a musical phrase, phrase, or section, a sub-region is marked as a leaf node in the musical structure corresponding to the parent node; wherein, when the musical structure corresponding to the parent node is a musical phrase, the leaf node corresponds to a musical phrase; when the musical structure corresponding to the parent node is a musical phrase, the leaf node corresponds to a section; and when the musical structure corresponding to the parent node is a section, the leaf node corresponds to a musical pattern.

[0014] Optionally, labeling the predetermined music structure as a preset attribute according to the second preset rule includes:

[0015] When the predetermined musical structure includes a musical pattern, the attribute name includes a first central note and a chord. The first central note includes elements such as note coordinates, duration, and note name. The chord includes elements such as root note, bass note, and virtuous note. The musical coordinates and note name elements of the first central note are stored in string data format and displayed and labeled via a front-end dropdown control. The duration element of the first central note is stored in floating-point data format and displayed and labeled via a front-end dropdown control. The root note, bass note, and virtuous note elements of the chord are stored in string data format and displayed and labeled via a front-end dropdown control.

[0016] When the predetermined musical structure includes musical phrases, the attribute names include a second central note, a first tonality, and a cadence. The second central note includes elements such as note coordinates, duration, and note name; the first tonality includes elements such as key signature and mode. The musical coordinate and note name elements of the second central note are stored in string data format and displayed and labeled via a front-end dropdown control. The duration element of the second central note is stored in floating-point data format and displayed and labeled via a front-end dropdown control. The key signature and mode elements of the first tonality are stored in string data format and displayed and labeled via a front-end dropdown control. The cadence is stored in integer data format and displayed and labeled via a front-end slider control.

[0017] When the predetermined musical structure includes musical phrases, the attribute names include second tonality, mood, narrative, singability, and fluency; wherein, the second tonality includes elements of key signature and mode, and the key signature and mode elements of the second tonality are stored in string data format and displayed and labeled through a front-end drop-down control; the mood, narrative, singability, and fluency are all stored in integer data format and displayed and labeled through a front-end slider control;

[0018] When the predetermined music structure includes musical segments, the attribute names include dramatic, descriptive, dance-like, and melodic; wherein, the dramatic, descriptive, dance-like, and melodic attributes are all stored in integer data format and displayed and labeled through the front-end slider control.

[0019] Optionally, the relationships between predetermined musical structures can be indicated as follows:

[0020] Select the first and second nodes in the tree structure;

[0021] Relationship data is generated based on the first node and the second node; wherein the relationship data includes variation relationship data and repetition relationship data; wherein the variation relationship data indicates that the notes and rhythms of the musical structures corresponding to the first node and the second node are inconsistent; and the repetition relationship data indicates that the notes and rhythms of the musical structures corresponding to the first node and the second node are consistent.

[0022] Optionally, dividing the music data into a predetermined music structure according to a first preset rule includes:

[0023] Phonetic patterns are labeled using a phonetic pattern segmentation algorithm.

[0024] Optionally, saving the labeled music data to the corresponding data table in the database includes:

[0025] The annotated music data is displayed and stored in the structure and form of a parser graph.

[0026] A specific embodiment of the present invention provides a music data processing device, the device comprising:

[0027] The acquisition module is used to acquire music data;

[0028] A first standard module is used to label the music data as a predetermined music structure according to a first preset rule; wherein the predetermined music structure includes at least one of the following: musical phrase, musical sentence, musical scale, and musical pattern;

[0029] The second annotation module is used to annotate the predetermined music structure as a preset attribute according to the second preset rule, and to annotate the relationship between the predetermined music structures to form annotated music data;

[0030] The saving module is used to save the labeled music data to the corresponding data table in the database.

[0031] Optionally, the first annotation module includes:

[0032] The first selection unit is used to select a parent node in the tree structure; wherein the music data contains a music structure that corresponds to the tree structure;

[0033] The first annotation unit is used to annotate a sub-region as a leaf node in the music structure corresponding to the parent node when the parent node corresponds to a musical phrase, musical sentence, or musical unit; wherein, when the music structure corresponding to the parent node is a musical phrase, the leaf node corresponds to a musical sentence; when the music structure corresponding to the parent node is a musical sentence, the leaf node corresponds to a musical unit; and when the music structure corresponding to the parent node is a musical unit, the leaf node corresponds to a musical pattern.

[0034] Optionally, the second annotation module is used for:

[0035] When the predetermined musical structure includes a musical pattern, the attribute name includes a first central note and a chord. The first central note includes elements such as note coordinates, duration, and note name. The chord includes elements such as root note, bass note, and virtuous note. The musical coordinates and note name elements of the first central note are stored in string data format and displayed and labeled via a front-end dropdown control. The duration element of the first central note is stored in floating-point data format and displayed and labeled via a front-end dropdown control. The root note, bass note, and virtuous note elements of the chord are stored in string data format and displayed and labeled via a front-end dropdown control.

[0036] When the predetermined musical structure includes musical phrases, the attribute names include a second central note, a first tonality, and a cadence. The second central note includes elements such as note coordinates, duration, and note name; the first tonality includes elements such as key signature and mode. The musical coordinate and note name elements of the second central note are stored in string data format and displayed and labeled via a front-end dropdown control. The duration element of the second central note is stored in floating-point data format and displayed and labeled via a front-end dropdown control. The key signature and mode elements of the first tonality are stored in string data format and displayed and labeled via a front-end dropdown control. The cadence is stored in integer data format and displayed and labeled via a front-end slider control.

[0037] When the predetermined musical structure includes musical phrases, the attribute names include second tonality, mood, narrative, singability, and fluency; wherein, the second tonality includes elements of key signature and mode, and the key signature and mode elements of the second tonality are stored in string data format and displayed and labeled through a front-end drop-down control; the mood, narrative, singability, and fluency are all stored in integer data format and displayed and labeled through a front-end slider control;

[0038] When the predetermined music structure includes musical segments, the attribute names include dramatic, descriptive, dance-like, and melodic; wherein, the dramatic, descriptive, dance-like, and melodic attributes are all stored in integer data format and displayed and labeled through the front-end slider control.

[0039] Optionally, the second annotation module is further configured to:

[0040] The second selection unit is used to select the first node and the second node in the tree structure;

[0041] The second annotation unit is used to annotate relationship data based on the first node and the second node; wherein the relationship data includes variation relationship data and repetition relationship data; wherein the variation relationship data indicates that the notes and rhythms of the musical structures corresponding to the first node and the second node are inconsistent; and the repetition relationship data indicates that the notes and rhythms of the musical structures corresponding to the first node and the second node are consistent.

[0042] Optionally, the first annotation module is also used to annotate phonemes using a phoneme segmentation algorithm.

[0043] Optionally, the storage module is also used to display and store the annotated music data in the structure and form of a parser graph.

[0044] A specific embodiment of the present invention also provides an electronic device, comprising:

[0045] Processor; and

[0046] Stored program memory,

[0047] The program includes instructions that, when executed by the processor, cause the processor to perform the method of the embodiments of the present invention.

[0048] A further embodiment of the present invention provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause a computer to execute the method of the embodiments of the present invention.

[0049] A specific embodiment of the present invention also provides a computer program product, the computer program product including instructions, which, when executed, cause a computer to perform the method of the embodiments of the present invention.

[0050] According to the above embodiments of the present invention, music data is acquired, and the music data is labeled as a predetermined music structure according to a first preset rule. The predetermined music structure includes at least one of the following: musical phrase, musical sentence, musical unit, and musical pattern. The predetermined music structure is labeled as a preset attribute according to a second preset rule, and the relationships between the predetermined music structures are labeled to form labeled music data. The labeled music data is then saved to the corresponding data table in the database. This method can at least partially solve the problem in related technologies where the labeling of music data cannot meet the requirements for training data, improves the quality and efficiency of music data labeling, thereby better meeting the requirements of training data and providing labeled data support for subsequent task research.

[0051] It should be understood that the above general description and the following specific embodiments are merely exemplary and illustrative, and do not limit the scope of the invention. Attached Figure Description

[0052] The accompanying drawings, which are part of the specification of this invention, illustrate exemplary embodiments of the invention. The drawings, together with the description in the specification, serve to illustrate the principles of the invention.

[0053] Figure 1 This is a flowchart of a music processing method provided in a specific embodiment of the present invention.

[0054] Figure 2 This is a schematic diagram of a music score data annotation module provided in a specific embodiment of the present invention.

[0055] Figure 3 This is a schematic diagram of a musical score data annotation process provided for a specific embodiment of the present invention.

[0056] Figure 4 This is a schematic block diagram of a music data processing device provided for a specific embodiment of the present invention.

[0057] Figure 5 This is a schematic block diagram of a first annotation module provided for a specific embodiment of the present invention.

[0058] Figure 6 This is a schematic block diagram of a second annotation module provided for a specific embodiment of the present invention.

[0059] Figure 7 A structural block diagram of an exemplary electronic device that can be used to implement embodiments of the present disclosure is shown. Detailed Implementation

[0060] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the spirit of the contents disclosed in the present invention will be clearly explained below with reference to the accompanying drawings and detailed description. After understanding the embodiments of the present invention, any person skilled in the art can make changes and modifications based on the technology taught in the present invention without departing from the spirit and scope of the present invention.

[0061] The illustrative embodiments and descriptions of the present invention are used to explain the invention, but are not intended to limit the invention. Furthermore, elements / components using the same or similar reference numerals in the drawings and embodiments are used to represent the same or similar parts.

[0062] The terms "first," "second," etc., used in this document are not intended to specifically refer to order or sequence, nor are they intended to limit the invention. They are merely used to distinguish elements or operations described using the same technical terms.

[0063] The directional terms used in this article, such as up, down, left, right, front, or back, are for reference only when referring to the accompanying drawings. Therefore, the use of directional terms is for illustrative purposes and not to limit this work.

[0064] The terms “include,” “including,” “have,” “contain,” etc., used in this article are all open-ended terms, meaning that they include but are not limited to.

[0065] The term "and / or" as used herein includes any or all of the things mentioned.

[0066] The term "multiple" in this article includes "two" and "more than two"; the term "multiple groups" in this article includes "two groups" and "more than two groups".

[0067] The terms "approximately," "about," etc., used herein are intended to modify any quantity or error that may vary slightly, but these slight variations or errors do not change the essence of the quantity or error. Generally, the range of slight variations or errors modified by such terms may be 20% in some embodiments, 10% in others, 5% in still others, or other values. Those skilled in the art should understand that the aforementioned values ​​can be adjusted according to actual needs and are not limited thereto.

[0068] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0069] When expressions such as "at least one of A, B, and C" are used, they should generally be interpreted in accordance with the meaning commonly understood by those skilled in the art (e.g., "a system having at least one of A, B, and C" should include, but is not limited to, systems having A alone, having B alone, having C alone, having A and B, having A and C, having B and C, and / or having A, B, and C, etc.). When expressions such as "at least one of A, B, or C" are used, they should generally be interpreted in accordance with the meaning commonly understood by those skilled in the art (e.g., "a system having at least one of A, B, or C" should include, but is not limited to, systems having A alone, having B alone, having C alone, having A and B, having A and C, having B and C, and / or having A, B, and C, etc.). Those skilled in the art should also understand that any conjunction and / or phrase that substantially arbitrarily indicates two or more optional items, whether in the specification, claims, or drawings, should be understood to indicate the possibility of including one of these items, either of these items, or both items. For example, the phrase “A or B” should be understood as including the possibility of “A” or “B”, or “A and B”.

[0070] In the music field, music annotation data is fundamental to AI applications. AI can improve its understanding and creative abilities by learning from and processing large amounts of music annotation data. Specifically, music annotation data helps AI systems perform tasks such as music feature extraction, music style classification, music generation, and recommendation. Without this data, AI models cannot learn how to understand and generate music, nor can they perform well in music information retrieval, recommendation systems, and other music-related tasks. As technology advances, the collection and organization of music annotation data will become even more important, necessitating the development of more efficient and accurate annotation tools and methods.

[0071] This disclosure utilizes the Imageparser concept to provide an innovative music data processing method that can be performed by a terminal, server, and / or other devices with processing capabilities. Imageparser is software for manual and semi-automated manual image annotation, as well as vectorization image processing software. It is also a database containing image annotation databases and And-Or Graph related knowledge, namely the LHI manually annotated image database, which serves as a benchmark platform for researchers in the computer science field. Unlike typical image processing software that focuses only on low-to-medium level editing, conversion, and filtering visual tasks, ImageParser can be used for image parsing, that is, to hierarchically decompose an image into meaningful components, recording internal structural information of objects that is not visible beneath their surface.

[0072] This invention provides a music data processing method, which will be referred to below. Figure 1 The flowchart shown introduces the music data processing method.

[0073] Step S101: Obtain music data.

[0074] In one optional embodiment, the sheet music is obtained through a terminal or server. This embodiment of the invention does not limit the specific method of obtaining music data, and any feasible method is within the protection scope of this embodiment of the invention.

[0075] Step S102: Mark the music data as a predetermined music structure according to the first preset rule; wherein the predetermined music structure includes at least one of the following: musical phrase, musical sentence, musical scale, musical pattern.

[0076] Specifically, a section is a fragment of a piece of music containing one or more relatively complete musical ideas. A section is a larger structural unit in a musical work, usually composed of multiple musical phrases. A section can be further subdivided into different parts, such as a theme, transition, or contrast. The end of a section is usually accompanied by a harmonic resolution or a repetition of musical material.

[0077] A musical phrase is a relatively independent and complete musical segment, but it is not sufficient to express the full musical meaning. A musical phrase is the basic unit that constitutes a musical section, similar to a sentence in language. It consists of a series of notes and has a beginning, development, and end. The length of a musical phrase can vary, but it usually follows a specific rhythmic pattern or harmonic structure.

[0078] Period: A relatively independent segment, but one that conveys a sense of "breath". A period is a specific form of musical phrase, consisting of two balanced parts, each containing a statement and response to a theme. Periods typically follow a specific harmonic pattern, such as TSDT (Tonic-Subdominant-Dominant-Tonic).

[0079] Motif: A motif is a small, recurring unit of musical expression that can appear in multiple pieces of music. It is the smallest structural unit in a musical work, typically composed of a few notes, and possesses a unique melodic or rhythmic character. Motifs can serve as the basis for a theme or be repeated throughout a piece to create a unified musical language.

[0080] Step S103: According to the second preset rule, the predetermined music structure is marked as a preset attribute, and the relationship between the predetermined music structures is marked to form labeled music data.

[0081] In one optional embodiment, preset attributes include the tonality, rhythm, tempo, dynamics, timbre, harmonic complexity, and melodic line of the music. Corresponding attribute annotations are added to the musical structure using music notation software or in an audio workstation. For example, tonality and tempo markers can be added to musical phrases, and dynamic changes can be marked on melodic lines.

[0082] In another alternative embodiment, relationships between musical structures, such as repetition, variation, counterpoint, etc., are determined, and these relationships are annotated in the score or audio editing software. For example, symbols or text can be used to indicate that a certain musical phrase is a repetition of a previous phrase, or how two melodies are counterpointed.

[0083] Step S104: Save the annotated music data to the corresponding data table in the database.

[0084] Existing music analysis platforms at most analyze the structure of music and a limited number of relationships between structures. This invention, through the aforementioned steps, uniformly annotates music information such as levels, structures, attributes, and relationships within the score, unifying the annotation content of the music score information. It enables real-time interactive recording, display, and storage of annotations of music structure, relationships, and attributes, solving the problem that existing technologies cannot meet the requirements for training data. This improves the quality and efficiency of music data annotation, laying the foundation for subsequent tasks such as structured analysis, information retrieval, and controllable generation of music in artificial intelligence, better meeting the requirements of training data, and providing annotated data support for future research tasks.

[0085] Step S102 above involves labeling the music data as a predetermined music structure according to a first preset rule. In a specific embodiment, a parent node in the tree structure is selected. The music structure contained in the music data corresponds to the tree structure. When the parent node corresponds to a musical phrase, musical sentence, or musical section, a sub-region is labeled as a leaf node in the music structure corresponding to the parent node. When the music structure corresponding to the parent node is a musical phrase, the leaf node corresponds to a musical sentence. When the music structure corresponding to the parent node is a musical sentence, the leaf node corresponds to a musical section. When the music structure corresponding to the parent node is a musical section, the leaf node corresponds to a musical pattern.

[0086] Before annotating the music data, in a more specific embodiment, user accounts include two types: administrators and regular annotation users. Administrators have permissions to manage users, the sheet music database, annotation attributes, and relational directories. Regular users can only log in, access the sheet music list, see the sheet music assigned to them by the administrator, and annotate and save it.

[0087] When operating as an administrator, users log in with their username and password to complete the login process. Administrators can view all sheet music in the current database. They can modify sheet music names and assign annotations; after assignment, the corresponding user will only see the sheet music for which the annotation was assigned. Administrators can upload sheet music by clicking the upload button and selecting a local file. Administrators can also delete sheet music by clicking the delete button next to the corresponding sheet music. Administrators can query, add, and modify user accounts.

[0088] When operating as a user, the user logs in by entering their username and password. Annotated users can only see the sheet music assigned to them by the administrator, and can search by sheet music name and annotate the corresponding sheet music.

[0089] The specific annotation process follows the hierarchical structure of the music: musical phrase -> musical section -> musical phrase -> musical pattern. First, a musical phrase A should be annotated. Within musical phrase A, the next level (child node) is annotated, such as musical phrase a. Within musical phrase a, the next level is annotated, such as musical phrase 1. Within musical phrase 1, the next level is annotated, such as musical pattern i. It should be noted that the hierarchy can have a recursive relationship; for any level, its child nodes contain elements of the level it belongs to. For example, the child node of musical phrase A may be musical phrase a; the child node of musical phrase 1 may be musical phrase i. However, the child node of a musical phrase cannot be a musical phrase. Because of the existence of repeated levels, the total number of levels in the tree structure must be greater than or equal to 4.

[0090] The specific annotation process is as follows:

[0091] First, select a parent node in the tree structure. Specifically, the root node is selected by default for the first annotation, and the previously added node is selected by default for any new node. It's important to note that if you forget to click on the parent node before selecting it, the parent node of the newly added node may be in the wrong position. In this case, simply modify the parent node. If you find that the node range is selected incorrectly, simply click on the node in the tree structure and delete it in the properties box. In the sheet music, the area corresponding to the parent node will be marked with a specific colored line below it, and the areas of the existing child nodes of this parent node will be marked with a different specific colored line below them.

[0092] Within the parent node's range, select a new range. Specifically, click the start note, then click the end note. When determining the musical elements contained within the node, select the note elements and ignore rests and other staff elements. These are marked below with a specific colored line, indicating the range being annotated. It should be noted that when two or more notes begin simultaneously, the note order increases sequentially from left to right and bottom to top. Therefore, if you need to include the entire starting note, click the bottommost note.

[0093] In the hierarchy of the attribute information bar, select one of the following: musical phrase, musical section, musical phrase, or musical pattern.

[0094] Step S103 involves labeling the predetermined musical structure as a preset attribute according to a second preset rule. In an optional embodiment, when the predetermined musical structure includes a melody, the attribute name includes a first central note and a chord. The elements of the first central note are note coordinates, duration, and note name. The elements of the chord are root note, bass note, and virtuoso note. The musical coordinates and note name elements of the first central note are stored in string data format and displayed and labeled through a front-end drop-down control. The duration element of the first central note is stored in floating-point data format and displayed and labeled through a front-end drop-down control. The root note, bass note, and virtuoso note elements of the chord are stored in string data format and displayed and labeled through a front-end drop-down control.

[0095] When the predetermined musical structure includes musical phrases, the attribute names include a second central note, a first tonality, and a cadence. The second central note includes elements such as note coordinates, duration, and note name; the first tonality includes elements such as key signature and mode. The musical coordinate and note name elements of the second central note are stored as string data and displayed and labeled via a front-end dropdown control. The duration element of the second central note is stored as floating-point data and displayed and labeled via a front-end dropdown control. The key signature and mode elements of the first tonality are stored as string data and displayed and labeled via a front-end dropdown control. The cadence is stored as an integer data format and displayed and labeled via a front-end slider control.

[0096] When the predetermined musical structure includes musical phrases, the attribute names include second tonality, mood, narrative, singability, and fluency; wherein, the second tonality includes elements of key signature and mode, and the key signature and mode elements of the second tonality are stored in string data format and displayed and labeled through a front-end drop-down control; the mood, narrative, singability, and fluency are all stored in integer data format and displayed and labeled through a front-end slider control;

[0097] When the predetermined musical structure includes musical segments, the attribute names include dramatic, descriptive, dance-like, and melodic; among them, dramatic, descriptive, dance-like, and melodic are all stored in integer data format and displayed and labeled through the front-end slider control.

[0098] See the image below for details:

[0099]

[0100] Step S104 above involves annotating the relationships between predetermined musical structures. In a specific embodiment, a first node and a second node in the tree structure are selected, and relationship data is generated based on the first node and the second node. This relationship data includes variation relationship data and repetition relationship data. The variation relationship data indicates that the notes and rhythms of the musical structures corresponding to the first node and the second node are inconsistent; the repetition relationship data indicates that the notes and rhythms of the musical structures corresponding to the first node and the second node are consistent. More specifically, relationship annotation requires selecting a node in the already constructed tree structure; clicking the add button in the relationship box; selecting another node; automatically generating a relationship data entry in the relationship box; selecting the type of relationship; and completing the addition of a relationship data entry. When any node is clicked in the tree structure, other nodes related to that node will be highlighted in a specific way.

[0101] As a specific implementation method, the aforementioned music data annotation can be accomplished using the Music Parser music data annotation software and system. Music Parser unifies the annotation of music score data, developing a software platform to enable manual annotation of music data represented by scores, including musical structure, melody, harmony, and related attributes and relationships, thus completing the manual analysis of music. Currently, Music Parser is deployed on a server as a web application, providing users with the opportunity to annotate music in web page format.

[0102] Music Parser supports music score files in formats such as xml, musicxml, and mxl. The annotation data is displayed and stored in the structure and form of parsegraph, which preserves the structured and hierarchical information of the music score and reasonably represents the attributes of each level and the relationship between levels. The data structure is clear and the parsed information is explicit.

[0103] Music Parser is a web application deployed on a server, accessible to users as a webpage, for annotating music and storing the annotated data in the cloud. Currently, music data annotated in batches using Music Parser forms datasets containing information such as music structure, relationships, and attributes, which can be used for various tasks in artificial intelligence music processing, including structured analysis, information retrieval, and controlled generation.

[0104] The front-end features innovative functions for marking, displaying, and playing sheet music, with a user-friendly interface and clear data display.

[0105] After selecting the desired interval and its hierarchical structure, the corresponding attributes for that level will appear in the attribute box. Click to annotate the corresponding attribute values ​​one by one, and then click "Add" to complete the process. A new branch will also be added to the corresponding tree structure. It should be noted that if you find any errors in the attribute annotations, you can modify them and submit again.

[0106] Step S102 above involves labeling the music data as a predetermined music structure according to a first preset rule. In an optional embodiment, a phonological segmentation algorithm is used to label the phonological patterns. The phonological segmentation algorithm effectively identifies and segments different phonological patterns (such as melody, rhythm, harmony, etc.) in the audio signal. The backend embeds an automatic phonological segmentation algorithm, which intelligently provides phonological segmentation and supports user modification, thereby improving labeling efficiency.

[0107] Step S104 above involves saving the annotated music data to the corresponding data table in the database. In an optional embodiment, the annotated music data is displayed and stored in the structure and form of a parser graph. A parser graph provides a structured representation method that can clearly represent the various components of music data and their interrelationships, such as melody lines, rhythmic patterns, and harmonic structures. Parser graphs can also serve as input for machine learning models, used to train models for tasks such as identifying music features, style classification, and sentiment analysis. Displaying and storing annotated music data in the structure and form of a parser graph organizes music information in a graphical data structure, facilitating analysis, retrieval, and further processing. Displaying and storing annotated data in the structure and form of a parser graph preserves the structured and hierarchical information of the music score and reasonably represents the attributes of each level and the relationships between levels, laying the foundation for subsequent tasks such as structured parsing, information retrieval, and controllable generation of AI-generated music. Through this optional embodiment, music data can not only exist in a more intuitive and easily processed form but also bring higher efficiency and broader application possibilities to fields such as music creation, analysis, and research. With the continuous advancement of technology, analytical graphs will be used more widely and become one of the important tools for music information processing.

[0108] Music Parser primarily focuses on user-interactive data annotation, post-annotation data reading, and storage. It also incorporates related tasks such as music data playback, reading / writing, visualization, and displaying annotated data in the form of a Parser Graph to complete the annotation and parsing of various types of music data. It implements an annotation mode for sheet music data and includes five functional modules: data display and playback, structural annotation, attribute annotation, relationship annotation, and parser graph display. The relationships between these functional modules are as follows: Figure 2 As shown. Figure 2The relationships between the modules are illustrated. After data is read in, it is collaboratively displayed through the display and playback module and the analytical graph display module. The annotation function module includes a structure annotation module, an attribute annotation module, and a relationship annotation module. Structure annotation is the first operation performed, providing structured objects for the music data. Subsequent attributes and relationships are annotated between these structured objects. The structured data is collaboratively displayed in the display and playback module and the analytical graph display module, while the relationship annotations are displayed in the analytical graph display module. After annotation is completed, the annotated data is stored. The overall annotation process is as follows: Figure 3 As shown, after the data is read in, it is displayed through the display and playback module. This module can adjust the basic playback state according to the user's settings. Annotation operations include three types: structural annotation, attribute annotation, and relationship annotation.

[0109] The data is divided according to structural hierarchy. For each labeled hierarchical structure, a corresponding node is generated in the analytical diagram display module. When the node is selected in the analytical diagram display module, the corresponding music segment is highlighted in the data display module.

[0110] Each hierarchical object is labeled with its attributes, and different objects have different attribute definitions.

[0111] When annotating hierarchical relationships, the relationship between two objects within the same hierarchy is labeled. Each time a relationship is added to the relationship list, a corresponding node relationship is generated in the analytical graph display module. Selecting that node relationship in the analytical graph display module highlights the corresponding data entry in the relationship list.

[0112] Each annotation step requires storing the annotation data in the corresponding data table in the database, which must have basic functions such as adding, deleting, modifying, and querying.

[0113] In one specific embodiment, a B / S (Browser / Server) software development architecture is adopted. The annotation software application is deployed on a web server. Each user sends an HTTP request to the web server through a client browser. The corresponding web application running on the web server processes the client request, responds after processing, and stores and performs CRUD operations on the user-annotated data in the database system on the web server. Therefore, the entire software system and the annotated data are stored on the server, while the annotation interface is provided to the annotator on the client in the form of a webpage.

[0114] Once the entire score structure is annotated, each node's attributes are labeled, the tree structure is successfully built, and node relationships are added, the entire annotation process is complete. In the score list, scores without any operations will show as unannotated with no modification time. Once a user performs an operation, the annotation status will show as annotated, recording the modification time and the person who made the modification. To continue editing, click "Edit" to see the current annotation status and make additions or modifications.

[0115] This invention provides a music data processing apparatus for implementing the aforementioned music data processing method. Figure 4 The schematic block diagram of the music data processing device shown includes:

[0116] Module 401 is used to acquire music data;

[0117] The first standard module 402 is used to label the music data as a predetermined music structure according to a first preset rule; wherein the predetermined music structure includes at least one of the following: musical phrase, musical sentence, musical scale, and musical pattern;

[0118] The second annotation module 403 is used to annotate the predetermined music structure as a preset attribute according to the second preset rule, and to annotate the relationship between the predetermined music structures to form annotated music data;

[0119] The save module 404 is used to save the annotated music data to the corresponding data table in the database.

[0120] Figure 5 This is a schematic block diagram of the first annotation module 402, as shown below. Figure 5 As shown, the first annotation module 402 also includes:

[0121] The first selection unit 4021 is used to select a parent node in the tree structure; wherein the music data contains a music structure that corresponds to the tree structure.

[0122] The first annotation unit 4022 is used to annotate a sub-region as a leaf node in the music structure corresponding to the parent node when the parent node corresponds to a musical phrase, musical sentence, or musical section; wherein, when the music structure corresponding to the parent node is a musical phrase, the leaf node corresponds to a musical sentence; when the music structure corresponding to the parent node is a musical sentence, the leaf node corresponds to a musical section; and when the music structure corresponding to the parent node is a musical section, the leaf node corresponds to a musical pattern.

[0123] Optionally, the second annotation module 403 is also used for:

[0124] When the predetermined musical structure includes a musical pattern, the attribute names include a first central note and a chord. The first central note includes elements such as note coordinates, duration, and note name. The chord includes elements such as root note, bass note, and chord. The musical coordinates and note name of the first central note are stored in string data format and displayed and labeled through a front-end dropdown control. The duration of the first central note is stored in floating-point data format and displayed and labeled through a front-end dropdown control. The root note, bass note, and chord elements are stored in string data format and displayed and labeled through a front-end dropdown control.

[0125] When the predetermined musical structure includes musical phrases, the attribute names include a second central note, a first tonality, and a cadence. The second central note includes elements such as note coordinates, duration, and note name; the first tonality includes elements such as key signature and mode. The musical coordinate and note name elements of the second central note are stored as string data and displayed and labeled via a front-end dropdown control. The duration element of the second central note is stored as floating-point data and displayed and labeled via a front-end dropdown control. The key signature and mode elements of the first tonality are stored as string data and displayed and labeled via a front-end dropdown control. The cadence is stored as an integer data format and displayed and labeled via a front-end slider control.

[0126] When the predetermined musical structure includes musical phrases, the attribute names include second tonality, mood, narrative, singability, and fluency; wherein, the second tonality includes elements of key signature and mode, and the key signature and mode elements of the second tonality are stored in string data format and displayed and labeled through a front-end drop-down control; the mood, narrative, singability, and fluency are all stored in integer data format and displayed and labeled through a front-end slider control;

[0127] When the predetermined musical structure includes musical segments, the attribute names include dramatic, descriptive, dance-like, and melodic; among them, dramatic, descriptive, dance-like, and melodic are all stored in integer data format and displayed and labeled through the front-end slider control.

[0128] Figure 6 This is a schematic block diagram of the second annotation module 403, as shown below. Figure 6 As shown, the second annotation module 403 includes:

[0129] The second selection unit 4031 is used to select the first node and the second node in the tree structure;

[0130] The second annotation unit 4032 is used to annotate relationship data based on the first node and the second node; wherein the relationship data includes variation relationship data and repetition relationship data; wherein the variation relationship data indicates that the notes and rhythms of the musical structures corresponding to the first node and the second node are inconsistent; and the repetition relationship data indicates that the notes and rhythms of the musical structures corresponding to the first node and the second node are consistent.

[0131] Optionally, the first annotation module 403 is also used to annotate phonemes using a phoneme segmentation algorithm.

[0132] Optionally, the storage module 404 is also used to display and store the annotated music data in the structure and form of a parser graph.

[0133] An exemplary embodiment of the present invention also provides an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor. The memory stores a computer program executable by the at least one processor, the computer program being executed by the at least one processor to cause the electronic device to perform a method according to an embodiment of the present invention.

[0134] An exemplary embodiment of the present invention also provides a non-transitory computer-readable storage medium storing a computer program, wherein the computer program, when executed by a computer's processor, is used to cause the computer to perform a method according to an embodiment of the present invention.

[0135] An exemplary embodiment of the present invention also provides a computer program product, including a computer program, wherein, when executed by a computer's processor, the computer program is used to cause the computer to perform a method according to an embodiment of the present invention.

[0136] refer to Figure 7 The present invention will now describe a structural block diagram of an electronic device that can serve as a server or client of the present invention, which is an example of a hardware device that can be applied to various aspects of the present invention. The electronic device is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0137] like Figure 7As shown, the electronic device 700 includes a computing unit 701, which can perform various appropriate actions and processes based on a computer program stored in a read-only memory (ROM) 702 or a computer program loaded from a storage unit 708 into a random access memory (RAM) 703. The RAM 703 may also store various programs and data required for the operation of the device 700. The computing unit 701, ROM 702, and RAM 703 are interconnected via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0138] Multiple components in electronic device 700 are connected to I / O interface 705, including: input unit 706, output unit 707, storage unit 708, and communication unit 709. Input unit 706 can be any type of device capable of inputting information to electronic device 700. Input unit 706 can receive input digital or character information and generate key signal inputs related to user settings and / or function control of electronic device. Output unit 707 can be any type of device capable of presenting information and may include, but is not limited to, a display, speaker, video / audio output terminal, vibrator, and / or printer. Storage unit 704 may include, but is not limited to, disk and optical disk. Communication unit 709 allows electronic device 700 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks, and may include, but is not limited to, modems, network cards, infrared communication devices, wireless communication transceivers, and / or chipsets, such as Bluetooth™ devices, WiFi devices, WiMax devices, cellular communication devices, and / or the like.

[0139] The computing unit 701 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 701 performs the various methods and processes described above. For example, in some embodiments, the music data processing method can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 708. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 700 via ROM 702 and / or communication unit 709. In some embodiments, the computing unit 701 can be configured to perform the music data processing method by any other suitable means (e.g., by means of firmware).

[0140] The program code used to implement the methods of the present invention can be written in any combination of one or more programming languages. This program code can be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code can be executed entirely on the machine, partially on the machine, as a standalone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0141] In the context of this invention, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0142] As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, device, and / or apparatus (e.g., disk, optical disk, memory, programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including machine-readable media that receive machine instructions as machine-readable signals. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0143] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0144] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0145] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other.

Claims

1. A music data processing method, characterized in that, The method includes: Acquire music data; The music data is labeled as a predetermined music structure according to a first preset rule; wherein the predetermined music structure includes at least one of the following: musical phrase, musical sentence, musical unit, musical pattern; According to the second preset rule, the predetermined music structure is labeled as a preset attribute, and the relationship between the predetermined music structures is labeled to form labeled music data; The labeled music data is saved to the corresponding data table in the database.

2. The music data processing method according to claim 1, characterized in that, According to the first preset rule, the music data is labeled as a predetermined music structure, including: Select a parent node in the tree structure; wherein the music data contains a music structure that corresponds to the tree structure; When the parent node corresponds to a musical phrase, phrase, or section, a sub-region is marked as a leaf node in the musical structure corresponding to the parent node; wherein, when the musical structure corresponding to the parent node is a musical phrase, the leaf node corresponds to a musical phrase; when the musical structure corresponding to the parent node is a musical phrase, the leaf node corresponds to a section; and when the musical structure corresponding to the parent node is a section, the leaf node corresponds to a musical pattern.

3. The music data processing method according to claim 1, characterized in that, According to the second preset rule, the predetermined music structure is marked as a preset attribute, including: When the predetermined musical structure includes a musical pattern, the attribute name includes a first central note and a chord. The first central note includes elements such as note coordinates, duration, and note name. The chord includes elements such as root note, bass note, and virtuous note. The musical coordinates and note name elements of the first central note are stored in string data format and displayed and labeled via a front-end dropdown control. The duration element of the first central note is stored in floating-point data format and displayed and labeled via a front-end dropdown control. The root note, bass note, and virtuous note elements of the chord are stored in string data format and displayed and labeled via a front-end dropdown control. When the predetermined musical structure includes musical phrases, the attribute names include a second central note, a first tonality, and a cadence. The second central note includes elements such as note coordinates, duration, and note name; the first tonality includes elements such as key signature and mode. The musical coordinate and note name elements of the second central note are stored in string data format and displayed and labeled via a front-end dropdown control. The duration element of the second central note is stored in floating-point data format and displayed and labeled via a front-end dropdown control. The key signature and mode elements of the first tonality are stored in string data format and displayed and labeled via a front-end dropdown control. The cadence is stored in integer data format and displayed and labeled via a front-end slider control. When the predetermined musical structure includes musical phrases, the attribute names include second tonality, mood, narrative, singability, and fluency; wherein, the second tonality includes elements of key signature and mode, and the key signature and mode elements of the second tonality are stored in string data format and displayed and labeled through a front-end drop-down control; the mood, narrative, singability, and fluency are all stored in integer data format and displayed and labeled through a front-end slider control; When the predetermined music structure includes musical segments, the attribute names include dramatic, descriptive, dance-like, and melodic; wherein, the dramatic, descriptive, dance-like, and melodic attributes are all stored in integer data format and displayed and labeled through the front-end slider control.

4. The music data processing method according to claim 1, characterized in that, The relationships between the pre-defined musical structures are indicated as follows: Select the first and second nodes in the tree structure; Relationship data is generated based on the first node and the second node; wherein the relationship data includes variation relationship data and repetition relationship data; wherein the variation relationship data indicates that the notes and rhythms of the musical structures corresponding to the first node and the second node are inconsistent; and the repetition relationship data indicates that the notes and rhythms of the musical structures corresponding to the first node and the second node are consistent.

5. The music data processing method according to claim 1, characterized in that, According to the first preset rule, the music data is divided into a predetermined music structure, including: Phonetic patterns are labeled using a phonetic pattern segmentation algorithm.

6. The music data processing method according to any one of claims 1 to 5, characterized in that, Saving the labeled music data to the corresponding data table in the database includes: The annotated music data is displayed and stored in the structure and form of a parser graph.

7. A music data processing device, characterized in that, The device includes: The acquisition module is used to acquire music data; A first standard module is used to label the music data as a predetermined music structure according to a first preset rule; wherein the predetermined music structure includes at least one of the following: musical phrase, musical sentence, musical scale, and musical pattern; The second annotation module is used to annotate the predetermined music structure as a preset attribute according to the second preset rule, and to annotate the relationship between the predetermined music structures to form annotated music data; The saving module is used to save the labeled music data to the corresponding data table in the database.

8. The music data processing apparatus according to claim 7, characterized in that, The first annotation module includes: The first selection unit is used to select a parent node in the tree structure; wherein the music data contains a music structure that corresponds to the tree structure; The first annotation unit is used to annotate a sub-region as a leaf node in the music structure corresponding to the parent node when the parent node corresponds to a musical phrase, musical sentence, or musical unit; wherein, when the music structure corresponding to the parent node is a musical phrase, the leaf node corresponds to a musical sentence; when the music structure corresponding to the parent node is a musical sentence, the leaf node corresponds to a musical unit; and when the music structure corresponding to the parent node is a musical unit, the leaf node corresponds to a musical pattern.

9. The music data processing apparatus according to claim 7, characterized in that, The second annotation module is used for: When the predetermined musical structure includes a musical pattern, the attribute name includes a first central note and a chord. The first central note includes elements such as note coordinates, duration, and note name. The chord includes elements such as root note, bass note, and virtuous note. The musical coordinates and note name elements of the first central note are stored in string data format and displayed and labeled via a front-end dropdown control. The duration element of the first central note is stored in floating-point data format and displayed and labeled via a front-end dropdown control. The root note, bass note, and virtuous note elements of the chord are stored in string data format and displayed and labeled via a front-end dropdown control. When the predetermined musical structure includes musical phrases, the attribute names include a second central note, a first tonality, and a cadence. The second central note includes elements such as note coordinates, duration, and note name; the first tonality includes elements such as key signature and mode. The musical coordinate and note name elements of the second central note are stored in string data format and displayed and labeled via a front-end dropdown control. The duration element of the second central note is stored in floating-point data format and displayed and labeled via a front-end dropdown control. The key signature and mode elements of the first tonality are stored in string data format and displayed and labeled via a front-end dropdown control. The cadence is stored in integer data format and displayed and labeled via a front-end slider control. When the predetermined musical structure includes musical phrases, the attribute names include second tonality, mood, narrative, singability, and fluency; wherein, the second tonality includes elements of key signature and mode, and the key signature and mode elements of the second tonality are stored in string data format and displayed and labeled through a front-end drop-down control; the mood, narrative, singability, and fluency are all stored in integer data format and displayed and labeled through a front-end slider control; When the predetermined music structure includes musical segments, the attribute names include dramatic, descriptive, dance-like, and melodic; wherein, the dramatic, descriptive, dance-like, and melodic attributes are all stored in integer data format and displayed and labeled through the front-end slider control.

10. The music data processing apparatus according to claim 7, characterized in that, The second annotation module is also used for: The second selection unit is used to select the first node and the second node in the tree structure; The second annotation unit is used to annotate relationship data based on the first node and the second node; The relationship data includes variation relationship data and repetition relationship data; wherein, the variation relationship data indicates that the notes and rhythms of the musical structures corresponding to the first node and the second node are inconsistent; the repetition relationship data indicates that the notes and rhythms of the musical structures corresponding to the first node and the second node are consistent.

11. The music data processing apparatus according to claim 7, characterized in that, The first annotation module is also used to annotate phonemes using a phoneme segmentation algorithm.

12. The music data processing apparatus according to any one of claims 7 to 11, characterized in that, The storage module is also used to display and store the annotated music data in the structure and form of a parser graph.

13. An electronic device, comprising: processor; as well as Stored program memory, The program includes instructions that, when executed by the processor, cause the processor to perform the method according to any one of claims 1-6.

14. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-6.

15. A computer program product, characterized in that, The computer program product includes instructions that, when executed, cause a computer to perform the method of any one of claims 1 to 6.

Images