AI-based piano and orchestral instrument total score chord identification method

Abstract

The invention provides an AI-based piano and orchestral instrument total score and chord identification method, and relates to the technical field of music information retrieval and processing. The method realizes accurate chord identification through the following steps: firstly, carrying out data preprocessing on the total score of the piano or the orchestral instrument, and extracting note information of each sound part; key feature vectors such as pitch contour are extracted; then, constructing and training a deep learning model, updating parameters by adopting an optimization algorithm, and preventing overfitting by utilizing regularization; inputting a music score to be recognized into the model to obtain chord label probability distribution, and selecting a label with the maximum probability as a preliminary recognition result; and finally, performing post-processing optimization on the result. According to the method, a sound part priority rule is innovatively introduced, the problem of core harmony information interference caused by multi-sound part note conflicts is effectively solved, the model is endowed with strong cross-style adaptability through specific style data training and addition of style auxiliary features, and then transfer learning, model integration and data enhancement are combined.

Description

Technical Field

[0001] This invention relates to the field of music information retrieval and processing technology, specifically to an AI-based method for chord recognition in piano and orchestral scores. Background Technology

[0002] Automatic chord recognition technology for piano and orchestral scores essentially analyzes the harmonic function of multi-part note combinations through computational models. Existing methods typically rely on a combination of pitch profile (PCP) feature extraction and deep learning models. However, due to the spatiotemporal overlap of notes in multiple parts such as melody, harmony, and ornamentation in orchestral scores, core harmonic information is easily interfered with. In particular, when bass parts (such as cello C3) and treble ornamentation parts (such as flute E5) coexist, traditional weighted average feature extraction strategies can blur the root note location, leading to misjudgment of chord attributes (such as misidentifying a C major triad as CMaj7). The root cause of this problem lies in the lack of a mechanism for prioritizing voice functions in existing technologies.

[0003] Current mainstream solutions share common limitations: A two-stage piano chord recognition method based on deep learning (patent number CN114898744B) uses a framework that connects single-note recognition and chord recognition models, but it lacks rules for assigning part weights, resulting in decorative parts (such as strong percussion beats) being weighted equally in the feature vector; at the same time, its static model structure is difficult to adapt to non-standard progression logic such as jazz (e.g., dominant seventh chord followed by a half-diminished seventh chord). A chord recognition method, device, equipment, and storage medium (patent number CN112927667A) corrects the results through an attention mechanism, but it relies on a preset classical harmony rule library (e.g., forcibly prohibiting parallel fifths), making it incompatible with styles that allow exceptions to the rules, such as folk music, and it lacks an algorithm for resolving part conflicts. Both types of technologies face the problems of insufficient resolution of multi-voice conflicts and rigidity of cross-style adaptability. Specifically, they are manifested as follows: 1) Due to the lack of quantification of voice priority, when the melodic voice conflicts with the bass pitch, the existing model may incorrectly increase the weight of ornaments, causing root tone shift; 2) Fixed rule bases or static network structures are difficult to fully accommodate the chord logic variations of different genres (such as parallel fifths, which are prohibited in classical music, are acceptable in folk music). These limitations make the existing methods have a generalization bottleneck in complex score recognition scenarios. Summary of the Invention

[0004] (a) Technical problems to be solved To address the shortcomings of existing technologies, this invention provides an AI-based method for chord recognition in piano and orchestral scores, solving the problems mentioned in the background section.

[0005] (II) Technical Solution To achieve the above objectives, the present invention provides the following technical solution: an AI-based method for chord recognition in piano and orchestral scores, comprising the following steps: Data preprocessing: Input the piano or orchestral score into the system in digital form, and analyze the score to extract the note information of each voice. Feature extraction: Based on the parsed musical score data, at least pitch-class contour feature vectors should be extracted; Chord recognition model construction: Constructing a sequence modeling model based on deep learning; Model training: The model was trained using piano and orchestral scores with chord annotations, including adjusting hyperparameters and structure using a validation set and employing regularization techniques to prevent overfitting. Chord recognition: Input the musical score data to be recognized into the trained model, output the probability distribution of chord labels at each time position, and select the label with the highest probability as the preliminary recognition result; Post-processing: Post-processing the preliminary recognition results, including at least one of the following: chord label smoothing, transposition processing, and chord optimization.

[0006] Preferably, in the model training step, at least one of the following optimization methods is adopted: (a) initializing model parameters using a pre-trained model through transfer learning; (b) fusing the output results of multiple deep learning models using model ensemble techniques.

[0007] Preferably, the data preprocessing step further includes: performing format conversion and / or data normalization on the input digitized musical score to ensure data consistency and compatibility.

[0008] Preferably, in the data preprocessing step, a part priority rule is established for multiple instrumental parts in the orchestral score; when notes from different parts conflict at the same time position, the core part notes are retained according to the priority rule; the priority rule is: harmonic basic part > melody part > ornamentation part.

[0009] Preferably, the feature extraction step further includes extracting at least one of the following auxiliary features: note duration, note dynamics, voice spacing, or interval distribution.

[0010] Preferably, in the model building and / or training steps, at least one of the following is performed for a specific music style: (a) Use labeled datasets of this style to train or fine-tune the model; (b) Add auxiliary features that identify the music style to the input features.

[0011] Preferably, the model training step employs data augmentation techniques, including performing at least one of the following operations on the training data: transposing, changing time values ​​or rhythmic patterns, or adding ornamental notes or performance techniques.

[0012] Preferably, the post-processing step further includes verifying and correcting the recognition results based on preset chord formation rules and harmony logic.

[0013] (III) Beneficial Effects This invention provides an AI-based method for chord recognition in piano and orchestral scores, which has the following advantages: This invention introduces a voice priority rule, which can intelligently retain core voice notes and filter out ornamental voice interference during data preprocessing. This solves the problems of root note shift and chord attribute misjudgment caused by the lack of quantification of voice priority in existing methods, significantly improving chord recognition accuracy, especially when dealing with complex scores where bass and treble ornaments coexist.

[0014] By training with specific style data and incorporating music style-assisted features, the chord recognition system of this invention can flexibly adapt to different music styles. Whether it's works that follow classical music harmony rules or jazz and folk music that use unconventional chords, it can achieve high-precision recognition, greatly expanding the system's application scenarios. By employing transfer learning, model ensemble, and data augmentation techniques, this invention significantly improves model training efficiency and recognition performance. Transfer learning accelerates model convergence, model ensemble reduces the risk of overfitting, and data augmentation enhances model adaptability, enabling the system to maintain robust recognition capabilities in the face of varied musical data and ensuring that the output chord sequences are both accurate and musically plausible. Attached Figure Description

[0015] Figure 1 This is a schematic diagram of the process of the present invention. Detailed Implementation

[0016] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0017] Example

[0018] like Figure 1 As shown, this embodiment of the invention provides an AI-based method for chord recognition in piano and orchestral scores, including the following steps: S1. Data Preprocessing: Input the piano or orchestral score into the data preprocessing device in digital form (such as MusicXML or MIDI format); The data preprocessing device performs unified format conversion and standardization processing on the score files of different formats to ensure the consistency and compatibility of the input data; Extract the note information of each voice, including key elements such as pitch, duration, and rhythm; S2. Extracting Pitch Class Contour Features: The feature extraction device converts the parsed musical score data into pitch class contour feature vectors. By analyzing the pitch position of each note and statistically analyzing the energy distribution across different pitch classes, a feature vector capable of representing chord characteristics is formed. This feature vector is further enriched by combining information such as note duration, volume intensity, and pitch intervals between different voices. S3. Model training data preparation: Collect a large amount of piano and orchestral scores with chord annotations from different musical styles, periods and composers. Remove noise data and annotation errors from the collected data. Perform feature extraction and vectorization on the cleaned data. Divide the processed data into training set, validation set and test set. S4. Model Training: The model training device uses optimization algorithms such as stochastic gradient descent (SGD) and Adam to train the deep learning model. It iteratively updates the model's parameters to minimize the error between the predicted results and the actual labels. Dropout and L2 regularization are applied to prevent overfitting. Various metrics during the training process are dynamically monitored and promptly fed back to the user. Users can adjust the training parameters according to their specific needs. S5. Chord Recognition: Input the piano or orchestral score data to be recognized into the trained deep learning model. The model infers from the input data and outputs the probability distribution of chord labels corresponding to each time position or note combination. Based on the probability distribution, the chord label with the highest probability is selected as the recognition result. S6. Post-processing: Remove isolated and unreasonable chord jumps that occasionally occur in the recognition results to make the chord sequence smoother and more natural; convert the recognized chords into root position chords or common inversions so that users can intuitively understand their harmonic function; combine the context information of the music to identify common chord progression patterns and make detailed annotations in the recognition results.

[0019] The AI-based chord recognition method for piano and orchestral scores requires the joint participation of a data preprocessing module, a feature extraction module, a deep learning model building module, a model training data preparation module, a model training module, and a chord recognition and post-processing module.

[0020] The data preprocessing module is used to input piano or orchestral scores into the system in digital form. It supports a variety of common score formats, such as MusicXML and MIDI. Its core function is to parse the score and accurately extract the note information of each voice, covering key elements such as pitch, duration, and rhythm. In terms of design, this module is equipped with a format conversion module, which can perform unified format conversion and standardization processing for score files from different sources and in different formats. This effectively solves the problems of consistency and compatibility of input data and lays a solid foundation for subsequent processing.

[0021] The feature extraction module is closely linked to the data preprocessing module. It is responsible for extracting features closely related to chord recognition from the parsed musical score data. Based on the rules of chord formation in music theory, this module converts the note sequence in the score into pitch class profile (PCP) feature vectors. Specifically, it analyzes the pitch position of each note and statistically analyzes the energy distribution in different pitch classes to form feature vectors that can characterize chord characteristics. At the same time, the module also has an extension function, which can combine multi-dimensional information such as note duration, volume intensity, and pitch intervals between different voices to further enrich the connotation of the feature vectors. This provides more comprehensive and detailed input data for the chord recognition model, significantly improving the model's recognition ability and accuracy.

[0022] The deep learning model building module is one of the core components of this invention. This module provides a variety of deep learning architecture options, including Long Short-Term Memory (LSTM), Convolutional Neural Network (CNN), and Transformer, to meet the chord recognition needs under different complexities and application scenarios. The LSTM architecture, with its unique gating mechanism, can effectively capture long-term dependencies in music sequence data, and has a significant advantage in modeling the variation patterns of chord sequences. The CNN architecture is good at using convolutional layers to automatically extract features of local chord patterns, achieving efficient learning and recognition of chord features. The Transformer architecture, with its self-attention mechanism, can dynamically focus on the relationship between notes at different positions in the sequence, thereby accurately modeling harmonic context information globally. Users can flexibly choose a single architecture or build a hybrid model that integrates the advantages of multiple architectures according to actual task requirements and computing resources, giving full play to the characteristics of each architecture and improving the performance of chord recognition.

[0023] The model training data preparation module plays a crucial role in providing high-quality training data for deep learning models. This module includes a professional data collection module capable of collecting a large amount of chord-annotated piano and orchestral scores from various musical styles, periods, and composers. The data collection covers multiple genres, including classical, romantic, and modern music, ensuring the model is exposed to a rich variety of chord usage examples during training. The collected data is first transferred to the data cleaning submodule to remove noise and labeling errors. It then enters the feature extraction submodule, where features are extracted and vectorized according to unified standards. Finally, the processed data is scientifically divided into three parts: a training set, a validation set, and a test set. The training set is used for model parameter learning, the validation set for hyperparameter tuning and optimization, and the test set for objective and accurate performance evaluation of the trained model, ensuring strong generalization ability and practical application value.

[0024] The model training module works in tandem with the model training data preparation module, employing advanced optimization algorithms to efficiently train deep learning models. This module supports various optimization algorithms, such as stochastic gradient descent (SGD) and Adam, and can automatically select the optimal algorithm based on the model architecture and data characteristics. During training, the model training module fully utilizes the training set data, continuously updating the model parameters through iterative calculations to minimize the error between the prediction results and the true labels. To avoid overfitting, the module incorporates various regularization techniques, such as dropout and L2 regularization, which can be appropriately applied during training to enhance the model's generalization performance. Furthermore, the model training module features real-time monitoring, dynamically monitoring various metrics during training, such as loss function values ​​and accuracy, and promptly providing feedback to the user. In addition, the module allows users to flexibly adjust training parameters, such as learning rate and batch size, to achieve fine-grained control over the training process, ensuring optimal model training results and efficiently learning chord recognition knowledge and patterns from the data.

[0025] The chord recognition and post-processing module is the key execution component for realizing the chord recognition function. After the preprocessing and model training processes described above, the piano or orchestral score data to be recognized is input into this module. The module first uses a trained deep learning model to efficiently infer the input data. The model outputs the probability distribution of chord labels corresponding to each time position or note combination. Based on this probability distribution, the module adopts a scientific decision-making strategy to select the chord label with the highest probability as the final recognition result. To ensure high-quality output of the recognition results, the module also integrates advanced post-processing functions, one of which is chord label smoothing. First, it can remove isolated and unreasonable chord jumps that occasionally appear in the recognition results based on the harmonic logic of the music, making the chord sequence smoother and more natural. Second, the chord inversion processing function can flexibly convert the recognized chords into root position chords or common inversion forms according to the chord inversion rules in music theory, so that users can understand their harmonic function more intuitively. Third, the chord progression analysis function can accurately identify common chord progression patterns, such as authentic progressions and plural progressions, by combining the contextual information of the music, and make detailed annotations in the recognition results, bringing users a more in-depth and comprehensive harmonic analysis experience.

[0026] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. An AI-based piano and orchestral score and chord recognition method, characterized by, The method comprises the following steps: Data preprocessing: inputting piano or orchestral score in digital form into the system, and parsing the score to extract note information of each part; Feature extraction: based on the parsed score data, at least extracting a pitch class profile feature vector; Chord recognition model construction: constructing a sequence modeling model based on deep learning; Model training: training the model using piano and orchestral score data with chord annotations, including adjusting hyperparameters and structure using a validation set, and using regularization techniques to prevent overfitting; Chord recognition: inputting the to-be-recognized score data into the trained model, outputting chord label probability distribution at each time position, and selecting the probability maximum label as the preliminary recognition result; Post-processing: post-processing the preliminary recognition result, including at least one of chord label smoothing, transposition processing, and chord optimization.

2. The AI-based piano and orchestral score and chord recognition method of claim 1, wherein: In the model training step, at least one of the following optimization methods is used: (a) using a pre-trained model to initialize model parameters through transfer learning; (b) using model ensemble technology to fuse the output results of multiple deep learning models.

3. The AI-based piano and orchestral score and chord recognition method of claim 1, wherein: The data preprocessing step further comprises format conversion and / or data normalization processing on the input digital score to ensure data consistency and compatibility.

4. The AI-based piano and orchestral score and chord recognition method of claim 1, wherein: The data preprocessing step comprises establishing a part priority rule for multiple instrument parts of the orchestral score; when notes of different parts conflict at the same time position, the core part notes are retained according to the priority rule; the priority rule is: harmonic bass part > melodic part > decorative part.

5. The AI-based piano and orchestral score and chord recognition method of claim 1, wherein: The feature extraction step further comprises extracting at least one of the following auxiliary features: note duration, note dynamics, part spacing, or interval distribution.

6. The AI-based piano and orchestral score and chord recognition method of claim 1, wherein: In the model construction and / or training step, at least one of the following is performed for a specific music style: (a) training or fine-tuning the model using the annotated data set of the style; (b) adding auxiliary features that identify the music style to the input features.

7. The AI-based piano and orchestral score and chord recognition method of claim 1, wherein: In the model training step, data augmentation techniques are used, including performing at least one of the following operations on the training data: transposition, changing time value or rhythm pattern, or adding decorative notes or playing techniques.

8. The AI-based piano and orchestral score and chord recognition method of claim 1, wherein: The post-processing step further comprises verifying and correcting the recognition result based on preset chord composition rules and harmony logic.

Images