Transplanting method of Guqin music
Through deep learning technology, the ancient piano scores are data -based processing and feature extraction, and the automated transplantation of Guqin songs is realized, the problem of high complexity and high threshold of the piano song transplantation is solved, and the accuracy and naturalness of the performance are improved.
Patent Information
- Application Number
- CN202411484648.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-23
- Publication Date
- 2025-01-24
AI Technical Summary
When playing traditional music, guqin instruments need to arrange reasonable fingering to achieve smooth performance, but this process is complex and demanding, which limits the popularity of guqin.
Adopt a method based on deep learning, data -based processing of Guqin scores, extracting the characteristics of the note duration, sound height, fingerprint and other characteristics, and use the LSTM or Transformer model training prediction sequence and rhythm information to realize the automated transplantation of the guqin song.
Through automated fingerprint prediction and rhythm analysis, the complexity and threshold of Guqin song transplantation have been reduced, the accuracy and natural nature of the performance are improved, and the Guqin culture is easier to promote.
Smart Images
Figure CN119360811A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to the technical field of music information processing, in particular to a method for transplanting guqin music. Background Art
[0002] Guqin is an ancient Chinese musical instrument. Jianzi notation is a score that describes the fingering of the guqin. In the past, the characteristics of guqin Jianzi notation were: it was an action score for playing fingerings rather than a pitch sequence, because for the same pitch, there are many fingerings to play, so recording the fingerings makes it easier for others to learn guqin music; the second characteristic is that the ancient Jianzi notation did not record rhythm information. Without rhythm information, a process of "marking the score" is required when playing, which is to add rhythm to the fingering score and play it.
[0003] With the development of Guqin, multi-line notation has been developed, such as Guqin notation with simplified notation and reduced notation. It not only uses reduced notation to mark fingering, but also uses simplified notation to mark rhythm information. This is considered a complete and playable Guqin notation. In addition to the creation of Guqin music, modern people have two special tasks. The first is to play the music: playing the music is to add rhythm information to the ancient reduced notation and play it out; the second is to transplant Guqin music: in traditional instruments such as piano, the music score only records the pitch sequence, not the fingering, because there is only one corresponding key position for one pitch, and there are no different playing positions. In Guqin, the same note is played at different positions of different strings, so the fingering and position are recorded. If traditional music such as pop music is played using Guqin, it is necessary to arrange reasonable fingering to play. The rationality of fingering arrangement is very high, which determines whether it can be played smoothly using Guqin.
[0004] Guqin transplantation requires strong professional knowledge, which limits the transplantation process. For the current massive pop songs, using the guqin to perform requires complex fingering arrangements. With the development of deep learning and the emergence of large language models, AI transplantation has become possible, which greatly reduces the entry threshold of the guqin and will be of great help to the promotion of guqin culture.
[0005] There are many existing guqin AI-related works that are about notation work, but there are few works that use AI to predict fingerings to complete transplantation. Here, a deep learning method based on guqin score fingering actions is proposed. The core of this method is to train and predict guqin scores, that is, playing actions. The actions contain a series of information: the strings used by the guqin, the right hand fingerings used, the right hand fingering direction, the left and right fingerings, and the position information of the left hand frets. Through training, a fingering sequence that is convenient for playing is generated to carry out the transplantation of guqin music. Summary of the invention
[0006] Technical issues solved
[0007] In view of the shortcomings of the prior art, the present invention provides a method for transplanting guqin music, which solves the following problems:
[0008] 1. In the Guqin instrument, the same note is produced by different positions of different strings, so the fingering and position are recorded. If you want to play traditional pop music with the Guqin instrument, you need to arrange reasonable fingering to play it, and the rationality of the fingering arrangement is very high;
[0009] 2. Learning traditional guqin requires a high investment of time and energy.
[0010] Technical Solution
[0011] To achieve the above object, the present invention is implemented by the following technical scheme: a method for transplanting guqin music, the transplanting method comprising the following steps:
[0012] Sp1. Collecting original data: The collecting of original data includes digitizing the Guqin score and collecting and forming a Guqin score data set;
[0013] Sp2, feature extraction: the feature extraction includes various action sequences, string numbers, fret positions, fingering and rhythm details;
[0014] Sp3, model training:
[0015] Sp3.1, the model training includes selecting a suitable large model, the large model is either LSTM or Transformer;
[0016] Sp3.2, use the guqin score dataset collected in Sp2 to train the model, and the training model is used to predict the fingering sequence and rhythm information;
[0017] Sp4, transplantation algorithm: the transplantation algorithm includes a given music sequence lacking fingering and predicting the fingering action;
[0018] Sp5. Optimization and verification: The optimization and verification include performance evaluation and iterative improvement.
[0019] Preferably, the feature extraction step of Sp2 further includes note duration analysis: for each note duration T n The calculation formula is:
[0020] T n =t end -t start
[0021] t end is the note ending time, t startis the note start time. This step is used to obtain the duration of each note and provide basic data for subsequent rhythm analysis. Pitch analysis: the pitch of the note P n By the score position (x n ,y n ) is determined, and the pitch mapping function is defined as:
[0022] P n =f(x n ,y n )
[0023] f is a function that maps the score coordinates to pitches, which is implemented through a lookup table or linear interpolation so that the position of each note in the score is converted into a specific pitch value.
[0024] Preferably, the model training step includes: selecting a suitable large model, the large model is LSTM, and its basic unit update formula is:
[0025] h t =o t ⊙tanh(C t )
[0026] h t is the hidden state of the current time step, o t is the activation value of the output gate, controlling the flow of information into the output, C t is the current cell state. The update of the cell state is determined by the following formula:
[0027] C t =f(C t-1 ,i t ,g t )
[0028] C t-1 is the cell state at the previous moment, i t is the input gate, which controls the introduction of new information, g t is a candidate value, representing a new information flow. The model retains or forgets information through these gating mechanisms. The training loss function formula is:
[0029]
[0030] y i is the real fingering sequence, is the model prediction, h refers to the method sequence, N is the number of samples, and the loss function is used to measure the gap between the prediction and the real data to optimize the model parameters.
[0031] Preferably, the transplantation algorithm comprises: inputting a music sequence lacking fingering, and using a trained model for prediction, the prediction formula is:
[0032]
[0033] X is the input sequence, f model is a trained model function that outputs the predicted fingering sequence Dynamically adjust fingering: Combined with the music context information, the prediction results are smoothed to optimize the fluency of the fingering. The smoothing algorithm used is:
[0034]
[0035] α is the smoothing factor, Predicts the fingering at the previous moment, aiming to reduce sudden and unnatural fingering switches.
[0036] Preferably, the optimization and verification steps include: Performance evaluation: evaluating the model performance by calculating the accuracy and F1 value, defined as:
[0037]
[0038] TP is true positive, TN is true negative, FP is false positive, FN is false negative, Precision is accuracy, Recall is recall. This step ensures the reliability of the model in a real environment. The optimization and verification steps also include iterative improvements based on the evaluation results, by adjusting model parameters or increasing training data to improve the model effect. The optimization algorithm uses an adaptive learning rate:
[0039]
[0040] η t is the current learning rate, η0 is the initial learning rate, β is the decay factor, and t is the number of iterations.
[0041] Preferably, the model training step includes: using a transfer learning method to adapt to different styles of guqin music, and the transfer learning model output is:
[0042]
[0043] f new For the new model, f pretrained For pre-trained models, the output is combined to leverage existing knowledge to accelerate learning of new tasks. To transfer learning models;
[0044] The model training step also includes: fixing some layers during the training process to avoid overfitting, and setting them as:
[0045] Freeze={L1,L2,…,L k}
[0046] Lk The network layer to be fixed, freeze some layers to improve the stability of the model in new tasks, Freeze is the fixed layer function.
[0047] Preferably, the feature extraction step comprises: clustering the playing styles of different performers using a K-means clustering algorithm:
[0048]
[0049] C i is the data point of cluster i, μ i is the mean of the cluster. The style characteristics of different performers are analyzed through clustering to achieve personalized transplantation. Cost is the clustering algorithm function.
[0050] The feature extraction step also includes outputting clustering results and performing feature analysis to determine the note distribution characteristics of each style.
[0051] Preferably, the transplantation algorithm further includes:
[0052] Adjust the predicted fingering according to the music context and calculate the final fingering sequence P final The formula is:
[0053]
[0054] P final : The final output fingering sequence, Fingering sequence predicted by the model, α: context adjustment coefficient, which controls the degree of influence of context information on the final result, C: context information, including the relationship between the notes and the rhythm information. Through this combination, the model can adjust the fingering to better adapt to the overall emotional expression of the music. The context information C is extracted by the following formula:
[0055]
[0056] C: context information, used to adjust the fingering sequence, j: time offset, indicating the influence of k notes before and after the current note, β j : Weight coefficient, which indicates the influence of the previous and next notes on the current fingering during the adjustment process. It is determined through training. : The predicted fingering at time t+j, reflecting the impact on the current fingering at a specific moment.
[0057] Preferably, the optimization and verification step further includes: constructing a user feedback mechanism, and the formula for evaluating model performance is:
[0058]
[0059] Performance: the overall performance score of the model, N: the total number of feedback, indicating the number of user feedback samples collected, Feedback i : The feedback score of the i-th user is scored based on the accuracy or fluency of the performance. By calculating the average of all feedbacks, the performance of the model in practical applications is obtained. In the evaluation process, the indicator accuracy is introduced, and the calculation formula is:
[0060]
[0061] TP: True Positives, i.e. the number of correctly predicted fingering movements, FP: False Positives, i.e. the number of incorrectly predicted correct fingering movements. This indicator is used to quantify the accuracy of the model in fingering prediction.
[0062] Preferably, the transplantation method further comprises model optimization: the model optimization uses an adaptive learning rate to update the model, and the formula is:
[0063]
[0064] θ t+1 : Updated model parameters, θ t : Current model parameters, η t : Adaptive learning rate, dynamically adjust the learning rate according to the gradient information, The gradient of the loss function with respect to the parameter θ represents the error of the model under the current parameters. The adaptive learning rate can accelerate the convergence process and improve the training efficiency of the model.
[0065] The model optimization also includes introducing a regularization term to prevent overfitting, and the adjustment formula of the loss function is:
[0066] J ′ (θ)=J(θ)+λR(θ)
[0067] J ′ (θ): adjusted loss function, J(θ): original loss function, which measures the error between the model prediction and the true result, λ: regularization coefficient, which controls the influence of the regularization term, R(θ): regularization term, using L2 norm.
[0068] Beneficial Effects
[0069] The present invention provides a method for transplanting guqin music, which has the following beneficial effects:
[0070] 1. The present invention processes the guqin score into data, and the scheme can extract key features (such as note duration, pitch, fingering, etc.) from a large amount of score data. Compared with traditional manual analysis, this data-driven method is more accurate and efficient, reduces human error, and uses deep learning models such as LSTM or Transformer to effectively capture complex patterns in time series data. These models perform well in processing the rhythm and fingering prediction of guqin music, thereby improving the accuracy and naturalness of automatic performance.
[0071] 2. The present invention realizes the automatic performance of guqin music by converting guqin music scores into executable fingering sequences. This advantage allows music lovers and learners to experience guqin performance more easily, especially for those who cannot accept traditional teaching, it provides a new learning path, and can provide instant feedback for beginners, helping them to master fingering and rhythm more quickly. In addition, automatic performance can also provide a reference for performers to enhance their learning effect. BRIEF DESCRIPTION OF THE DRAWINGS
[0072] Figure 1 This is an example of ancient notation for the guqin.
[0073] Figure 2 This is the guqin score with simple line notation added to mark the rhythm;
[0074] Figure 3 A minus sign is used to indicate the use of digital methods;
[0075] Figure 4 The pitch of each position of the seven strings of the guqin. The same note can have multiple positions.
[0076] Figure 5 It is a system step diagram of the present invention. DETAILED DESCRIPTION
[0077] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. Specific embodiment one:
[0079] As shown in the figure, a method for transplanting guqin music includes the following steps:
[0080] Sp1. Collecting original data: Collecting original data includes digitizing the Guqin score and collecting it to form a Guqin score dataset;
[0081] Sp2, feature extraction: Feature extraction includes various action sequences, string numbers, fret positions, fingering and rhythm details;
[0082] Sp3, model training:
[0083] Sp3.1, model training includes selecting a suitable large model, which can be either LSTM or Transformer;
[0084] Sp3.2, use the guqin score dataset collected in Sp2 to train the model, and the training model is used to predict the fingering sequence and rhythm information;
[0085] Sp4, transplantation algorithm: The transplantation algorithm consists of given a music sequence lacking fingerings and predicting the fingering movements;
[0086] Sp5, Optimization and verification: Optimization and verification include performance evaluation and iterative improvement.
[0087] The feature extraction step of Sp2 further includes note duration analysis: for each note duration T n The calculation formula is:
[0088] T n =t end -t start
[0089] t end is the note ending time, t start is the note start time. This step is used to obtain the duration of each note and provide basic data for subsequent rhythm analysis. Pitch analysis: the pitch of the note P n By the score position (x n ,y n ) is determined, and the pitch mapping function is defined as:
[0090] P n =f(x n ,y n )
[0091] f is a function that maps the score coordinates to pitches, which is implemented through a lookup table or linear interpolation so that the position of each note in the score is converted into a specific pitch value.
[0092] The model training steps include: selecting a suitable large model, the large model is LSTM, and its basic unit update formula is:
[0093] h t =o t ⊙tanh(C t )
[0094] h t is the hidden state of the current time step, ot is the activation value of the output gate, controlling the flow of information into the output, C t is the current cell state. The update of the cell state is determined by the following formula:
[0095] C t =f(C t-1 ,i t ,g t )
[0096] C t-1 is the cell state at the previous moment, i t is the input gate, which controls the introduction of new information, g t is a candidate value, representing a new information flow. The model retains or forgets information through these gating mechanisms. The training loss function formula is:
[0097]
[0098] y i is the real fingering sequence, is the model prediction, h refers to the method sequence, N is the number of samples, and the loss function is used to measure the gap between the prediction and the real data to optimize the model parameters.
[0099] The transplantation algorithm includes: inputting a music sequence lacking fingering, using the trained model for prediction, and the prediction formula is:
[0100]
[0101] X is the input sequence, f model is a trained model function that outputs the predicted fingering sequence Dynamically adjust fingering: Combined with the music context information, the prediction results are smoothed to optimize the fluency of the fingering. The smoothing algorithm used is:
[0102]
[0103] α is the smoothing factor, Predicts the fingering at the previous moment, aiming to reduce sudden and unnatural fingering switches.
[0104] The optimization and validation steps include: Performance evaluation: The model performance is evaluated by calculating the accuracy and F1 value, which are defined as:
[0105]
[0106] TP is true positive, TN is true negative, FP is false positive, FN is false negative, Precision is accuracy, and Recall is recall. This step ensures the reliability of the model in a real environment. The optimization and verification steps also include iterative improvements based on the evaluation results, by adjusting model parameters or increasing training data to improve the model effect. The optimization algorithm uses an adaptive learning rate:
[0107]
[0108] η t is the current learning rate, η0 is the initial learning rate, β is the decay factor, and t is the number of iterations.
[0109] The model training steps include: using the transfer learning method to adapt to different styles of guqin music, and the transfer learning model output is:
[0110]
[0111] f new For the new model, f pretrained For pre-trained models, the output is combined to leverage existing knowledge to accelerate learning of new tasks. To transfer learning models;
[0112] The model training step also includes: During the training process, some layers are fixed to avoid overfitting, which is set as:
[0113] Freeze={L1,L2,…,L k}
[0114] L k The network layer to be fixed, freeze some layers to improve the stability of the model in new tasks, Freeze is the fixed layer function.
[0115] The feature extraction step includes: clustering the playing styles of different players using the K-means clustering algorithm:
[0116]
[0117] C i is the data point of cluster i, μ i is the mean of the cluster. The style characteristics of different performers are analyzed through clustering to achieve personalized transplantation. Cost is the clustering algorithm function.
[0118] The feature extraction step also includes outputting the clustering results and performing feature analysis to determine the note distribution characteristics of each style.
[0119] The migration algorithm also includes:
[0120] Adjust the predicted fingering according to the music context and calculate the final fingering sequence P final The formula is:
[0121]
[0122] P final : The final output fingering sequence, Fingering sequence predicted by the model, α: context adjustment coefficient, which controls the degree of influence of context information on the final result, C: context information, including the relationship between the notes and the rhythm information. Through this combination, the model can adjust the fingering to better adapt to the overall emotional expression of the music. The context information C is extracted by the following formula:
[0123]
[0124] C: context information, used to adjust the fingering sequence, j: time offset, indicating the influence of k notes before and after the current note, β j : Weight coefficient, which indicates the influence of the previous and next notes on the current fingering during the adjustment process. It is determined through training. : The predicted fingering at time t+j, reflecting the impact on the current fingering at a specific moment.
[0125] The optimization and verification steps also include: building a user feedback mechanism, and the formula for evaluating model performance is:
[0126]
[0127] Performance: the overall performance score of the model, N: the total number of feedback, indicating the number of user feedback samples collected, Feedback i : The feedback score of the i-th user is scored based on the accuracy or fluency of the performance. By calculating the average of all feedbacks, the performance of the model in practical applications is obtained. In the evaluation process, the indicator accuracy is introduced, and the calculation formula is:
[0128]
[0129] TP: True Positives, i.e. the number of correctly predicted fingering movements, FP: False Positives, i.e. the number of incorrectly predicted correct fingering movements. This indicator is used to quantify the accuracy of the model in fingering prediction.
[0130] Model optimization also includes: using adaptive learning rate to update the model, the formula is:
[0131]
[0132] θ t+1 : Updated model parameters, θt : Current model parameters, η t : Adaptive learning rate, dynamically adjust the learning rate according to the gradient information, The gradient of the loss function with respect to the parameter θ represents the error of the model under the current parameters. The adaptive learning rate can accelerate the convergence process and improve the training efficiency of the model.
[0133] Model optimization also includes the introduction of regularization terms to prevent overfitting. The adjustment formula of the loss function is:
[0134] J ′ (θ)=J(θ)+λR(θ)
[0135] J ′ (θ): adjusted loss function, J(θ): original loss function, which measures the error between the model prediction and the true result, λ: regularization coefficient, which controls the influence of the regularization term, R(θ): regularization term, using L2 norm.
[0136] Sp1. Collecting raw data: Data conversion process:
[0137] Use a high-resolution scanner to scan the paper guqin score to generate a digital image file, ensuring that the image is clear and readable for subsequent processing.
[0138] The scanned image is fed into an optical character recognition (OCR) system to extract the notes, rhythms, and fingering markings on the score. This process requires ensuring that the OCR model is fully trained to improve recognition accuracy.
[0139] After OCR recognition, manual review is performed to correct recognition errors, ensure the integrity and accuracy of the data, and avoid errors affecting subsequent steps.
[0140] Dataset construction:
[0141] Collect guqin music of different styles and types, including versions by multiple performers, to form a diverse dataset. This dataset should include pieces of varying difficulty to improve the adaptability and generalization ability of the model.
[0142] All collected score data and audio recordings are organized into a structured format (such as CSV or database) to facilitate subsequent analysis and processing.
[0143] Sp2. Feature extraction: note duration analysis:
[0144] Each identified note is time-stamped to record when it starts and ends, thereby determining the duration of the note. This information is crucial for analyzing rhythmic structure and helping to understand the role of each note in the flow of the music.
[0145] Ensure the accuracy of the analysis to provide reliable data for subsequent rhythm analysis.
[0146] Pitch Analysis:
[0147] The pitch of the note is determined by its position on the score, and the score coordinates are converted into specific pitch values through a lookup table or interpolation method. This step ensures that the high and low changes of the melody can be accurately captured, reflecting the unique timbre of the guqin.
[0148] Other feature extraction:
[0149] In addition to note duration and pitch, other important features are extracted, such as fingering, string number, fret position, etc. These features help the model learn the style and technique of playing, ensuring that the final output can truly reproduce the way the guqin is played.
[0150] Sp3. Model training: Choose a large model:
[0151] Choose a suitable deep learning model, such as LSTM (Long Short-Term Memory Network) or Transformer, and train it based on the note sequence and extracted features.
[0152] When choosing a model, you need to consider the complexity of the data and the training time. LSTM is suitable for processing time series data, while Transformer is better at capturing long-range dependencies.
[0153] Training process:
[0154] The model is trained using the extracted feature dataset and will learn how to predict fingering and rhythm information based on the input note sequence. During the training process, the model parameters are updated through optimization algorithms (such as Adam or SGD) to improve the prediction accuracy.
[0155] During the training process, it is necessary to monitor the performance of the model and adjust the hyperparameters to achieve the best results.
[0156] Sp4. Transplantation algorithm: fingering prediction:
[0157] Input a music sequence that lacks fingerings, and use the trained model to predict the corresponding fingering movements. The model will make predictions based on context information, combined with previous note and rhythm data, to ensure that the output fingerings are natural and smooth.
[0158] Utilization of contextual information:
[0159] In the prediction process, contextual information such as the relationship between notes and rhythm changes is considered to improve the accuracy and fluency of the prediction. The prediction results are optimized by combining contextual information through methods such as weighted average.
[0160] Sp5. Optimization and Verification: Performance Evaluation:
[0161] Evaluate the model's output through user feedback and automatic evaluation metrics such as accuracy, recall, etc. Collect user feedback on the model's prediction results to understand how the model performs in real-world applications.
[0162] Iterative improvements:
[0163] Based on the evaluation results, iterative optimization is performed on the shortcomings of the model, and the training data set, model architecture or hyperparameters are adjusted to continuously improve the performance of the model. Specific embodiment 2:
[0165] Transplantation of the Guqin piece "Three Variations on Plum Blossoms":
[0166] In this case, the goal is to convert the score information of the ancient guqin piece "Three Variations on Plum Blossoms" into a fingering sequence for automatic performance, using modern technology and machine learning models to achieve this goal.
[0167] Step 1: Collect raw data:
[0168] First, the paper score of "Three Variations on Plum Blossoms" was obtained and digitized. The score was scanned into an image file using a high-resolution scanner to ensure that every detail was clearly visible. Next, these images were input into the optical character recognition (OCR) system to automatically extract the notes, rhythm symbols, and fingering marks on the score. To improve recognition accuracy, the OCR system was specially trained to adapt to the characteristics of guqin score. After the recognition is completed, music professionals will conduct manual review to correct recognition errors and ensure the accuracy and completeness of the data.
[0169] Step 2: Build the dataset:
[0170] The identified notes, rhythms, and fingering information are organized into a structured data format (such as CSV), and multiple versions of "Three Variations on Plum Blossoms" are collected to analyze the styles of different performers. Each note record includes its start and end time, pitch, and corresponding fingering information, which provides basic data for subsequent feature extraction and model training.
[0171] Step 3: Feature Extraction:
[0172] In this phase, the duration of each note is first analyzed, and the start and end times of the notes are marked to ensure that rhythmic changes can be accurately captured. At the same time, the pitch of the notes is determined according to their position on the score to ensure that the high and low changes of the melody are effectively represented. In addition, other important features such as string numbers and fret positions are extracted to fully describe how the notes are played.
[0173] Step 4: Model training:
[0174] The Long Short-Term Memory (LSTM) network was chosen as the model because it performs well in processing time series data. Using the Guqin score dataset obtained from the feature extraction step, the model was trained to predict the fingering sequence and rhythm information. The cross-validation method was used during the training process to ensure the accuracy and robustness of the model. The model takes the time series of notes as input and learns the temporal dependencies between the notes.
[0175] Step 5: Fingering prediction:
[0176] After the training is completed, input a music sequence that lacks fingerings and use the trained LSTM model to predict fingerings. The model will generate a suitable fingering sequence based on context information and historical note data. At the same time, the prediction results are adjusted based on context information to enhance the expressiveness and coherence of the fingerings.
[0177] Step 6: Optimization and verification:
[0178] Finally, the model is optimized through user feedback mechanism and performance evaluation. In actual performance, feedback from performers is collected and the model is iteratively improved based on the feedback data. Performance evaluation includes checking the fluency and accuracy of fingering to ensure that the final fingering sequence can truly reflect the performance effect of "Three Variations on Plum Blossoms". Specific embodiment three:
[0180] The transplantation steps of the Guqin piece "Yangchun Baixue":
[0181] 1. Collect raw data
[0182] We first obtained the paper score of "Yang Chun Bai Xue" and digitized it using a high-resolution scanner to generate a clear image file. Then, the image was input into the optical character recognition (OCR) system to extract the notes, rhythms and fingering marks. The OCR system was specially trained for guqin scores to improve the accuracy of recognition. After the recognition was completed, the music experts reviewed the recognition results to ensure that they were correct and made necessary corrections. All data was organized into CSV format.
[0183] 2. Build a dataset
[0184] When constructing the dataset, we collected different versions of "Yang Chun Bai Xue" performed by multiple performers, including their recordings and scores. We ensured that the dataset covered diversity to facilitate the analysis of different playing styles. We recorded the detailed information of each note in a unified database, including its duration, pitch, and corresponding fingering.
[0185] 3. Feature extraction
[0186] When extracting features from the data, we first analyze the duration and pitch of each note. We record the start and end times of the notes to ensure that rhythmic changes are captured accurately. At the same time, we use the score coordinates to determine the pitch of each note to ensure that the high and low changes of the melody are effectively represented. In addition, other features such as string number and fingering are extracted to fully describe the performance.
[0187] 4. Model Training
[0188] Select a suitable model (such as LSTM or Transformer) and train it using the dataset obtained from feature extraction. The model learns to predict fingering sequences and rhythm information by analyzing the timing relationship of notes. During the training process, methods such as cross-validation are used to optimize model parameters and improve model accuracy and stability.
[0189] 5. Fingering prediction and verification
[0190] In the transplant algorithm, given a music sequence with missing fingerings, the model can predict the corresponding fingering movements. The prediction results are optimized to ensure that the predicted fingerings are consistent with the music context. During the verification process, the performance of the model is evaluated by comparing it with the real performance, and necessary adjustments and improvements are made.
[0191] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "comprising a reference structure" do not exclude the existence of other identical elements in the process, method, article or device including the elements.
[0192] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and alterations may be made to the embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.
Claims
1. A method for transplanting guqin music, characterized by: The transplantation method comprises the following steps: Sp1. Collecting original data: The collecting of original data includes digitizing the Guqin score and collecting and forming a Guqin score data set; Sp2, feature extraction: the feature extraction includes various action sequences, string numbers, fret positions, fingering and rhythm details; Sp3, model training: Sp3.1, the model training includes selecting a suitable large model, the large model is either LSTM or Transformer; Sp3.2, use the guqin score dataset collected in Sp2 to train the model, and the training model is used to predict the fingering sequence and rhythm information; Sp4, transplantation algorithm: the transplantation algorithm includes a given music sequence lacking fingering and predicting the fingering action; Sp5. Optimization and verification: The optimization and verification include performance evaluation and iterative improvement.
2. The method for transplanting guqin music according to claim 1, characterized in that: The feature extraction step of Sp2 further includes note duration analysis: for each note duration T n The calculation formula is: T n =t end -t start t end is the note ending time, t start is the note start time. This step is used to obtain the duration of each note and provide basic data for subsequent rhythm analysis. Pitch analysis: the pitch of the note P n By the score position (x n ,y n ) is determined, and the pitch mapping function is defined as: P n =f(x n ,y n ) f is a function that maps the score coordinates to pitches, which is implemented through a lookup table or linear interpolation so that the position of each note in the score is converted into a specific pitch value.
3. The method for transplanting guqin music according to claim 1, characterized in that: The model training step includes: selecting a suitable large model, the large model is LSTM, and its basic unit update formula is: h t =o t ⊙tanh(C t ) h t is the hidden state of the current time step, o t is the activation value of the output gate, controlling the flow of information into the output, C t is the current cell state. The update of the cell state is determined by the following formula: C t =f(C t-1 ,i t ,g t ) C t-1 is the cell state at the previous moment, i t is the input gate, which controls the introduction of new information, g t is a candidate value, representing a new information flow. The model retains or forgets information through these gating mechanisms. The training loss function formula is: y i is the real fingering sequence, is the model prediction, h refers to the method sequence, N is the number of samples, and the loss function is used to measure the gap between the prediction and the real data to optimize the model parameters.
4. The method for transplanting guqin music according to claim 1, characterized in that: The transplantation algorithm includes: inputting a music sequence lacking fingering, using a trained model to make predictions, and the prediction formula is: X is the input sequence, f model is a trained model function that outputs the predicted fingering sequence Dynamically adjust fingering: Combined with the music context information, the prediction results are smoothed to optimize the fluency of the fingering. The smoothing algorithm used is: α is the smoothing factor, Predicts the fingering at the previous moment, aiming to reduce sudden and unnatural fingering switches.
5. The method for transplanting guqin music according to claim 1, characterized in that: The optimization and verification steps include: Performance evaluation: The model performance is evaluated by calculating the accuracy and F1 value, which are defined as: TP is true positive, TN is true negative, FP is false positive, FN is false negative, Precision is accuracy, Recall is recall. This step ensures the reliability of the model in a real environment. The optimization and verification steps also include iterative improvements based on the evaluation results, by adjusting model parameters or increasing training data to improve the model effect. The optimization algorithm uses an adaptive learning rate: η t is the current learning rate, η0 is the initial learning rate, β is the decay factor, and t is the number of iterations.
6. The method for transplanting guqin music according to claim 1, characterized in that: The model training step includes: using the transfer learning method to adapt to different styles of guqin music, and the transfer learning model output is: f new For the new model, f pretrained For pre-trained models, the output is combined to leverage existing knowledge to accelerate learning of new tasks. To transfer learning models; The model training step also includes: fixing some layers during the training process to avoid overfitting, and setting them as: Freeze={L1,L2,…,L k } L k The network layer to be fixed, freeze some layers to improve the stability of the model in new tasks, Freeze is the fixed layer function.
7. The method for transplanting guqin music according to claim 1, characterized in that: The feature extraction step includes: clustering the playing styles of different performers using a K-means clustering algorithm: C i is the data point of cluster i, μ i is the mean of the cluster. The style characteristics of different performers are analyzed through clustering to achieve personalized transplantation. Cost is the clustering algorithm function. The feature extraction step also includes outputting clustering results and performing feature analysis to determine the note distribution characteristics of each style.
8. The method for transplanting guqin music according to claim 1, characterized in that: The transplantation algorithm also includes: Adjust the predicted fingering according to the music context and calculate the final fingering sequence P final The formula is: P final : The final output fingering sequence, Fingering sequence predicted by the model, α: context adjustment coefficient, which controls the degree of influence of context information on the final result, C: context information, including the relationship between the notes and the rhythm information. Through this combination, the model can adjust the fingering to better adapt to the overall emotional expression of the music. The context information C is extracted by the following formula: C: context information, used to adjust the fingering sequence, j: time offset, indicating the influence of k notes before and after the current note, β j : Weight coefficient, which indicates the influence of the previous and next notes on the current fingering during the adjustment process. It is determined through training. The predicted fingering at time t+j reflects the impact on the current fingering at a specific moment.
9. The method for transplanting guqin music according to claim 1, characterized in that: The optimization and verification steps also include: building a user feedback mechanism, and the formula for evaluating model performance is: Performance: the overall performance score of the model, N: the total number of feedbacks, which indicates the number of user feedback samples collected, Feedback i: the feedback score of the i-th user, which is scored based on the accuracy or fluency of the performance. By calculating the average of all feedbacks, the performance of the model in actual application is obtained. In the evaluation process, the indicator accuracy is introduced, and the calculation formula is: TP: True Positives, i.e. the number of correctly predicted fingering movements, FP: False Positives, i.e. the number of incorrectly predicted correct fingering movements. This indicator is used to quantify the accuracy of the model in fingering prediction.
10. The method for transplanting guqin music according to claim 1, characterized in that: The transplantation method also includes model optimization: the model optimization uses an adaptive learning rate to update the model, and the formula is: θt +1 : Updated model parameters, θt: current model parameters, ηt: adaptive learning rate, dynamically adjust the learning rate according to the gradient information, The gradient of the loss function with respect to the parameter θ represents the error of the model under the current parameters. The adaptive learning rate can accelerate the convergence process and improve the training efficiency of the model. The model optimization also includes introducing a regularization term to prevent overfitting, and the adjustment formula of the loss function is: J′(θ)=J(θ)+λR(θ) J′(θ): adjusted loss function, J(θ): original loss function, which measures the error between the model prediction and the true result, λ: regularization coefficient, which controls the influence of the regularization term, R(θ): regularization term, using L2 norm.