Quantitative evaluation method for mental load of piano hand music score playing

Abstract

The invention discloses a mental workload quantitative evaluation method for piano music score playing. The method comprises the following steps: collecting pulse wave data during static sitting, and pulse wave data and playing data during sound hand spectrum playing; performing preprocessing, feature extraction and baseline removal processing on the pulse wave data to obtain baseline removal features; performing sequence comparison processing on the playing data to obtain playing accuracy characteristics; performing feature screening on all features consisting of the baseline-removed features and the playing accuracy features, and constructing a mental load level classification model by using the screened optimal feature combination to obtain a classification posterior probability; and in combination with the classification posterior probability, establishing a mental load quantification formula, and calculating a mental load score when playing the sound-hand music score. According to the method, objective quantification can be realized on the mental load real state of piano hand music score playing in a lossless and noninvasive manner, and a new thought and technical support are provided for improvement and optimization of music education.

Description

Technical Field

[0001] The present invention relates to the field of signal processing technology, and in particular to a method for quantitatively evaluating the mental workload of piano sound and hand-written music playing. Background Art

[0002] Playing the piano is a complex cognitive and motor coordination task that involves multiple processes such as visual information processing and finger movement control, and requires a large amount of brain resources. Specifically, mental workload refers to the degree to which an individual's brain consumes mental resources when performing a task. In recent years, research on mental workload has received widespread attention in fields such as driving, aviation, and human-computer interaction, but has been less applied in the field of music performance, especially the lack of quantitative evaluation of mental workload during piano playing. Existing research on piano playing mostly focuses on technical performance, such as accuracy and speed, but lacks in-depth discussion of the mental workload state during playing, making it difficult to provide a scientific basis for piano teaching and practice. Therefore, how to objectively and quantitatively evaluate the level of mental workload during playing is of great research significance for optimizing piano teaching methods, improving learning efficiency, and improving music score design.

[0003] Staff notation is a traditional musical notation system that contains a large number of abstract note symbols and musical notation concepts. Playing requires rapid identification of note positions, durations, key changes, and various markings. This highly specialized system makes it difficult for non-professional piano players or beginners to identify, read, and recognize keys. This can lead to excessively high mental workload during playing, making it difficult to assess changes in mental workload. However, hand notation, as a new type of musical notation, draws on underlying musical logic to provide intuitive and easy-to-understand representations of note durations and fingering cues, simplifying the complex information found in staff notation. It combines the visual pitch of staff notation with the ease of note recognition and transposition of simplified notation, allowing non-professional piano players to quickly complete the process of reading, recognizing, identifying keys, and playing, maintaining a moderate mental workload and making it easier to assess changes in mental workload. Therefore, choosing hand notation for piano playing facilitates quantitative assessment of mental workload levels during playing.

[0004] Currently, methods for assessing mental workload primarily include subjective rating scales, task performance, and physiological indicator measurements. Subjective rating scales, primarily through questionnaires, scales, or interviews, are simple to use but suffer from subjectivity and poor real-time performance. Task performance measures primarily rely on performance indicators such as accuracy, completion time, and error rate, but they struggle to directly reflect mental workload levels. Physiological indicator measurements are objective, with commonly used physiological indicators such as EEG and ECG. However, these measurement methods have limitations in terms of sensitivity and portability. Studies have shown that pulse wave signals, as a physiological indicator, can reflect changes in mental workload and are simple and convenient to measure.

[0005] Currently, there is no objective method for evaluating mental workload during piano playing that combines performance and physiological indicators. Therefore, combining task performance with pulse wave-based physiological indicator measurement is valuable and feasible for studying mental workload during piano playing. Summary of the Invention

[0006] This invention aims to fill a gap in current research on mental workload in music performance by disclosing a method for quantitatively assessing the mental workload of piano finger-tab performance. This method objectively measures and analyzes individual mental workload levels during performance, accurately assessing the player's mental workload under finger-tab performances of varying difficulty levels. This method can provide a scientific basis for music teaching and training, help optimize teaching strategies, improve learning efficiency, and provide data support for the development of personalized teaching plans.

[0007] The purpose of the present invention can be achieved by taking the following technical solutions:

[0008] A method for quantitatively evaluating the mental workload of piano music playing comprises the following steps:

[0009] S1, collecting pulse wave data of the subject when sitting quietly, and collecting pulse wave data and playing data of the subject when playing piano music;

[0010] S2. Preprocessing the collected pulse wave data, extracting features, and performing baseline removal processing to obtain baseline-removed features;

[0011] S3, performing sequence comparison processing on the collected playing data to obtain playing accuracy characteristics;

[0012] S4, performing feature screening on all features consisting of the baseline-removed features and the playing accuracy features, constructing a mental workload level classification model with the best feature combination screened out, and obtaining a posterior probability of classification;

[0013] S5. Based on the classification posterior probability, a mental workload quantification formula is established to calculate the mental workload score of the subject when playing the piano sheet music.

[0014] Furthermore, the step S1 includes the following steps:

[0015] S101, establishing a piano music playing system equipped with a pulse wave acquisition module, having the subject play with their right hand and use their left hand to collect pulse wave data;

[0016] S102. Collect pulse wave data when sitting quietly, and pulse wave data and playing data when playing with low difficulty, medium difficulty and high difficulty respectively.

[0017] Furthermore, step S2 includes the following steps:

[0018] S201, performing low-pass filtering on the pulse wave data using a Blackman window;

[0019] S202, performing peak and trough detection on the pulse wave data after low-pass filtering using a dual-window search method to obtain a peak point sequence and a trough point sequence;

[0020] S203, performing baseline correction on the pulse wave data after low-pass filtering using a piecewise cubic Hermite interpolation method based on the trough point sequence obtained in S202;

[0021] S204. Segment all original cycles in the pulse wave data after baseline correction based on the trough point sequence obtained in S202. Interpolate and resample all segmented original cycles to obtain standardized cycles of the same length. Use the K-means clustering algorithm to divide all standardized cycles into three categories, calculate the average cycle template in the category with the largest number of cycles, calculate the correlation coefficient between this average cycle template and all standardized cycles one by one, eliminate standardized cycles with correlation coefficients lower than 0.9, and complete the abnormal cycle filtering process.

[0022] S205. Extract a pulse wave feature vector from the pulse wave data after abnormal cycle filtering; wherein the pulse wave feature vector includes 21 features: heart rate, minimum pulse rate, maximum pulse rate, pulse interval sequence mean, pulse interval sequence median, pulse interval sequence standard deviation, mean square value of differences between adjacent intervals of the pulse interval sequence, continuous difference variation coefficient, normalized coefficient of variation, pulse interval sequence skewness, pulse interval sequence kurtosis, very low frequency band power, low frequency band power, high frequency band power, total power, ratio of low frequency band power to high frequency band power, Shannon entropy, slope entropy, permutation entropy, sample entropy, and multi-scale entropy; subtract the pulse wave feature vector during meditation from the pulse wave feature vector during playing to obtain a true feature vector.

[0023] Furthermore, step S3 includes the following steps:

[0024] S301, performing sequence alignment processing on the subject's playing data using the longest common subsequence algorithm to obtain a subsequence with the correct position order;

[0025] S302. Compare the actual duration of each note in the subsequence with the standard duration, eliminate notes with a duration error greater than 30%, and divide the number of retained notes by the maximum length of the actual playing sequence and the standard playing sequence to obtain the playing accuracy.

[0026] Furthermore, step S4 includes the following steps:

[0027] S401, combining a set of real feature vectors including the 21 features and the playing accuracy feature to obtain a feature vector including 22 features;

[0028] S402, adding a mental load label to the feature vector according to the difficulty of playing, setting the label of playing the low-difficulty vocal tablature to 0, the medium-difficulty tablature to 1, and the high-difficulty tablature to 2, to obtain a playing feature vector sample;

[0029] S403, having multiple subjects play the three levels of difficulty of the vocal notation in sequence, obtaining multiple playing feature vector samples, and constructing a playing feature set;

[0030] S404, dividing the playing feature set into a training set and a test set in a ratio of 8:2;

[0031] S405, performing recursive feature elimination combined with cross-validation processing on the playing feature set to screen out the best feature combination;

[0032] S406. Input the obtained optimal feature combination into the deep forest model, train the deep forest model, and construct a mental workload level classification model;

[0033] S407. Output the posterior probabilities of the three categories of low load, medium load and high load through the mental workload level classification model, and set the posterior probability of low load as P0, the posterior probability of medium load as P1, and the posterior probability of high load as P2.

[0034] Furthermore, step S5 includes the following steps:

[0035] S501, combining the three posterior probabilities in step S407 and establishing a mental workload quantification formula in an exponentially weighted manner;

[0036] S502, collecting pulse wave data of a subject while sitting in meditation, and pulse wave data and playing data of a subject while playing piano music, to obtain a set of sitting pulse wave data, a set of playing pulse wave data and a corresponding set of playing data;

[0037] S503, processing the three data of the subject collected in step S502 through steps S2, S3, and S4 to obtain the posterior probabilities P0, P1, and P2 of the three classifications;

[0038] S504 , using the mental workload quantification formula established in S501 on the posterior probabilities of the three categories obtained in step S503 , to calculate the mental workload score of the subject when playing the piano sheet music.

[0039] The present invention has the following advantages and effects compared to the prior art:

[0040] 1) The present invention innovatively combines objective pulse wave physiological indicators with playing accuracy performance indicators, thereby improving the accuracy of quantitative evaluation of mental workload. Compared with traditional subjective scale evaluation methods, the present invention effectively avoids the subjective bias and individual differences that may be caused by self-evaluation of subjects, and provides more objective and repeatable evaluation results. At the same time, compared with the physiological indicator evaluation method that relies on a single electrocardiogram signal or electroencephalogram signal, the present invention reduces the complexity of data collection, does not require relatively expensive professional equipment, and has higher portability, providing an efficient, economical and easy-to-promote innovative solution for the quantitative evaluation of mental workload.

[0041] 2) The present invention is not only limited to the simple classification of mental workload levels, but also further quantifies the actual state of mental workload more accurately, thereby achieving a deeper study of mental workload. Through quantitative evaluation, the dynamic changes of mental workload under different playing difficulty conditions can be more intuitively reflected, thereby revealing the subtle effects of playing tasks of different difficulty levels on mental workload. In addition, the present invention has individualized analysis capabilities, can adapt to the physiological characteristics and playing styles of different players, and provide a scientific basis for personalized evaluation and precise intervention. At the same time, the method is suitable for long-term tracking and monitoring, and can sustainably analyze the changing trends of the player's mental workload, providing a reliable evaluation tool for music learning, cognitive training and intelligent education, which helps to optimize training programs and improve learning efficiency.

[0042] 3) This invention quantitatively assesses mental workload during piano playing, a relatively novel research direction with considerable application value in music cognition research and music education. It can help researchers gain a deeper understanding of the changing patterns of mental workload during piano playing and provide a scientific assessment tool for music teaching. Furthermore, this method is not only applicable to piano playing but can also be further extended to other types of instrumental performance and music training tasks, providing a broader space for applied research in mental workload assessment and helping to advance intelligent music education, personalized training systems, and interdisciplinary cognitive science research. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0044] Figure 1 This is a flow chart of a method for quantitatively evaluating the mental workload of piano music playing disclosed in an embodiment of the present invention;

[0045] Figure 2 This is a schematic diagram of a scenario for collecting subject data in an embodiment of the present invention;

[0046] Figure 3 Schematic diagram of playing a low-difficulty vocal score according to an embodiment of the present invention;

[0047] Figure 4 is a flow chart of pulse wave data preprocessing in an embodiment of the present invention;

[0048] Figure 5 is a schematic diagram of a pulse wave before preprocessing in an embodiment of the present invention;

[0049] Figure 6 is a schematic diagram of a pulse wave after preprocessing in an embodiment of the present invention;

[0050] Figure 7 is a schematic diagram of feature extraction from a preprocessed pulse wave in an embodiment of the present invention;

[0051] Figure 8 1 is a schematic diagram of sequence comparison processing of playing data according to an embodiment of the present invention;

[0052] Figure 9 This is a flow chart of constructing a mental workload classification model in an embodiment of the present invention. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention. A method for quantitatively evaluating the mental workload of playing piano sound scores, the evaluation method comprising the following steps:

[0054] S1, collecting pulse wave data when sitting quietly, as well as pulse wave data and playing data when playing vocal music;

[0055] S2. Preprocess the pulse wave data, extract features, and perform baseline removal processing to obtain baseline-removed features;

[0056] S3, performing sequence comparison processing on the playing data to obtain playing accuracy characteristics;

[0057] S4. Perform feature screening on all features consisting of baseline features and playing accuracy features, and use the best feature combination screened out to construct a mental workload level classification model to obtain the classification posterior probability;

[0058] S5. Combine the classification posterior probability to establish a mental workload quantification formula and calculate the mental workload score when playing the vocal notation.

[0059] Furthermore, the step S1 includes the following steps:

[0060] S101: Establish a piano music playing system equipped with a pulse wave acquisition module, and ask the subject to play with the right hand, keeping the left hand still to collect pulse wave data. In this step, the right hand is used to play mainly because most subjects are right-handed.

[0061] S102. Collect pulse wave data while sitting quietly, as well as pulse wave data and playing data when playing piano hand-piano scores of low, medium, and high difficulty levels. Therefore, by having a subject perform a piano hand-piano score playing experiment, four sets of pulse wave data and three sets of playing data can be obtained. The purpose of this step is to induce three levels of mental workload, namely, low workload, medium workload, and high workload, by playing piano hand-piano scores of three difficulty levels, thereby determining the number of categories in the subsequent mental workload classification model.

[0062] Furthermore, step S2 includes the following steps:

[0063] S201, low-pass filtering the pulse wave data collected in S102; the purpose of this step is to remove external environmental interference and high-frequency noise in the original pulse wave data and retain the main pulse wave components;

[0064] S202: Using a dual-window search method with two different lengths to perform peak and trough detection on the low-pass filtered pulse wave data, obtaining a peak point sequence and a trough point sequence. This step is intended to provide a relatively accurate periodic reference for subsequent baseline correction and in the early stages of signal preprocessing.

[0065] S203. Based on the trough point sequence obtained in S202, a piecewise cubic Hermite interpolation method is used to obtain a baseline drift curve. The baseline drift curve is subtracted from the pulse wave data after low-pass filtering to complete the baseline correction process. The purpose of this step is to remove ultra-low frequency signals caused by physiological respiratory movement, changes in human posture, or hardware circuit instability, namely, baseline drift noise interference, to ensure that subsequent feature extraction work has a unified benchmark.

[0066] S204: Segment all original cycles in the pulse wave data after baseline correction based on the trough point sequence obtained in S202. First, interpolation and resampling are performed on all segmented original cycles to make them have the same length, thereby achieving cycle standardization. Then, a K-means clustering algorithm is used to divide all standardized cycles into three categories of cycle clusters. The calculation formula of the K-means algorithm is as follows:

[0067]

[0068] Where k is the number of clusters, k value is 3, S i is the i-th cluster, x i is the ith normalized period among all normalized periods, μ i is the center of the i-th cluster;

[0069] Then, the average cycle template of the cycle cluster with the largest number of cycles is calculated; the calculation formula of the average cycle template is as follows:

[0070]

[0071] n is the number of cycles in the largest cluster, cycle i represents the i-th normalized period in the largest cluster, It is the average cycle template, which better represents the true normal cycle pattern;

[0072] Next, the Pearson correlation coefficient is calculated between this average cycle template and all standardized cycles one by one, and standardized cycles with correlation coefficients below 0.9 are eliminated. Finally, the original cycles corresponding to the retained standardized cycles are connected in their original order to complete signal reconstruction and achieve the purpose of abnormal cycle filtering. The purpose of this step is to eliminate motion artifact noise introduced by slight shaking of the human body during the pulse wave data acquisition process, thereby improving the accuracy of subsequent feature extraction.

[0073] S205. Extract 21 features from the pulse wave data after abnormal cycle filtering from three levels: physiological indicators, pulse rate variability, and nonlinear characteristics: heart rate (HR), minimum pulse rate (MinPR), maximum pulse rate (MaxPR), mean pulse interval sequence (MeanPP), median pulse interval sequence (MedianPR), standard deviation of pulse interval sequence (SDPP), mean square value of difference between adjacent intervals of pulse interval sequence (RMSSD), coefficient of variation of continuous differences (CVSD), normalized coefficient of variation (CVPP), skewness of pulse interval sequence (SkewPP), kurtosis of pulse interval sequence (KurtPP), very low frequency power (VLF), low frequency power (LF), high frequency power (HF), total power (TP), ratio of low frequency power to high frequency power (LF / HF), Shannon entropy, slope entropy, permutation entropy, sample entropy, and multi-scale entropy. Then combine the features to obtain a pulse wave feature vector. Taking into account the fact that physiological conditions of individual subjects, such as skin color, skin thickness, and cardiovascular system, are not completely consistent, the pulse waves collected from different subjects will differ in morphology, peak amplitude, and other aspects, resulting in individual differences between the pulse wave features extracted therefrom, thereby affecting the accuracy of the subsequent mental workload classification model; considering that the mental workload level of the subjects is lowest when sitting still, the collected pulse wave data can be used as baseline data, so this step also includes subtracting the sitting still pulse wave feature vector from the playing pulse wave feature vector to obtain the true feature vector, thereby achieving the purpose of removing individual differences in features.

[0074] Furthermore, the step S3 includes the following steps:

[0075] S301, performing sequence alignment processing on the subject's actual playing data using a longest common subsequence algorithm; comparing the actual playing data sequence with the standard playing data sequence, assuming that the lengths of the standard playing sequence and the actual playing sequence are M and N, respectively; using the standard playing sequence as the vertical axis and the actual playing sequence as the horizontal axis, constructing a two-dimensional dynamic programming table according to a recursive relationship, generating a backtracking path, and obtaining a specific longest common subsequence, i.e., a subsequence with the correct position order;

[0076] S302: Set the duration error to 30%, compare the actual duration of each note in the subsequence obtained in S301 with the standard duration, remove the notes with a duration error greater than 30%, and the remaining notes are correctly played. Divide the number of correctly played notes (num) by the maximum of M and N to obtain the playing accuracy rate. The playing accuracy rate is calculated as follows:

[0077]

[0078] Among them, ACC represents the playing accuracy, num refers to the number of correctly played notes, M and N are the lengths of the standard playing sequence and the actual playing sequence, respectively.

[0079] Furthermore, the step S4 includes the following steps:

[0080] S401, combining the true feature vector containing twenty-one features obtained in S205 and the playing feature playing accuracy rate obtained in S302 to obtain a feature vector containing twenty-two features;

[0081] S402: Add a mental workload label to the feature vector obtained in S401 based on the difficulty of playing. Set the label of playing easy-difficulty tablatures to 0, medium-difficulty tablatures to 1, and high-difficulty tablatures to 2, to obtain a playing feature vector. The purpose of this step is to label the feature vector and provide it for training and testing of the mental workload classification model constructed in subsequent steps.

[0082] S403, extracting the playing feature vectors of all subjects and constructing a playing feature set;

[0083] S404, dividing the playing feature set into a training set and a test set in a ratio of 8:2;

[0084] S405, using recursive feature elimination combined with a cross-validation algorithm to optimize and screen the playing feature set, through a cyclic process of iterative modeling and feature elimination, using classification accuracy as a feature importance evaluation index, comparing the classification accuracy of each feature combination through cross-validation, and gradually screening out the best feature combination with the highest classification accuracy; wherein, classification accuracy is a common index for evaluating feature importance and can reflect the impact of features on model performance to a certain extent; this step eliminates redundant features, screens out features that are significantly related to the mental workload level, eliminates interference of irrelevant features on the model, prevents the dimensionality curse, improves the training efficiency of the classification model, and ensures that the mental workload classification model obtained by subsequent training has high generalization performance;

[0085] S406. Input the obtained optimal feature combination into the Deep Forest model, train the Deep Forest model, establish a mapping relationship between the playing feature vector and the mental load level, and build a mental load level classification model. Since Deep Forest does not rely on large-scale data sets for training and adopts a multi-level random forest structure, it can adaptively learn features, requires less computing resources, and has strong interpretability. Therefore, Deep Forest is selected to build the mental load classification model. The purpose of this step is to comprehensively evaluate the performance of the classification model and reduce its overfitting risk, thereby building a robust cognitive load level classification model.

[0086] S407. The mental workload classification model constructed in S406 can output the posterior probabilities of the three categories of low load, medium load and high load, and the probability of being judged as low load is set as P0, the probability of being judged as medium load is set as P1, and the probability of being judged as high load is set as P2.

[0087] Furthermore, the step S5 includes the following steps:

[0088] S501. Combine the three posterior probabilities in S407 and establish a mental workload quantification formula in an exponentially weighted manner, as shown below:

[0089]

[0090] Among them, 10, 50, and 100 correspond to the basic weights P2 of low load probability P0, medium load probability P1, and high load probability, respectively, mapping the quantitative range of mental load during playing to [10, 100], and setting the low load interval to [10, 40], the medium load interval to [40, 70], and the high load interval to [70, 100]. x This is used to emphasize the contribution of high-probability workloads, amplifying the impact of these probabilities to enhance discrimination, consistent with the nonlinear variation of mental workload. The purpose of this step is to integrate probabilistic information representing different mental workload levels to establish a mental workload quantification formula, thereby quantitatively reflecting the actual state of mental workload and achieving objective quantification of mental workload levels.

[0091] S502, collecting pulse wave data of a subject while meditating, and pulse wave data and playing data of a subject while playing a finger music score, to obtain a set of meditating pulse wave data, a set of playing pulse wave data and a corresponding set of playing data;

[0092] S503, the three data of the subject are processed through steps S2, S3, and S4 to obtain three classification posterior probabilities P0, P1, and P2;

[0093] S504. The three classification posterior probabilities of S503 are used to calculate the mental workload quantification formula established in S501 to calculate the mental workload score of the subject when playing the tablature. A lower score indicates a lower mental workload, and the difficulty can be increased. Conversely, a higher score indicates a higher mental workload, and the difficulty needs to be decreased. The purpose of this step is to understand the actual mental workload during playing based on the quantified score, so as to make further improvements to learning and education.

[0094] Example 1

[0095] Figure 1 The present invention provides a flowchart of a method for quantitatively evaluating the mental workload of piano music playing.

[0096] S1. The process of collecting pulse wave data during meditation, and pulse wave data and playing data during playing the music score is as follows:

[0097] S101. Establish a piano music score playing system equipped with a pulse wave acquisition module. Ask the subject to play with the right hand and keep the left hand still for collecting pulse wave data.

[0098] S102: Collect pulse wave data when sitting quietly, and pulse wave data and playing data when playing low-difficulty, medium-difficulty and high-difficulty hand-written music scores.

[0099] Among them, this embodiment is carried out in a quiet and comfortable indoor place. Before the start of the experiment of playing the sound hand score, there is about five minutes of preparation time to allow the subjects to familiarize themselves with the experimental environment and experimental tasks, and relax by sitting quietly. The pulse wave data collected at this time can be used as pulse wave baseline data later. After the experiment starts, the subjects are asked to play the low-difficulty sound hand score to induce a low mental load level, play the medium-difficulty sound hand score to induce a medium mental load level, and play the high-difficulty sound hand score to induce a high mental load level. The collected pulse wave data and playing data correspond to different mental load levels respectively. The scene diagram of collecting subject data in this embodiment is shown as follows Figure 2 As shown, a display is used to display the hand-written music score, a pulse wave acquisition module is used to collect pulse wave data, and a digital piano is used to collect playing data. Figure 3 As shown, the music score is presented in a dynamic and visual form, with intuitive note length expressions and finger prompts.

[0100] S2. Preprocess the pulse wave data, extract features, and perform baseline removal processing to obtain baseline-removed features.

[0101] In this example, since there are various interferences and noises in the two pulse wave data collected, including external environment interference, high frequency noise, baseline drift noise and motion artifacts, it is necessary to preprocess the pulse wave data. The flow chart of the pulse wave data preprocessing in this embodiment is as follows: Figure 4 As shown in Figure 1, it includes low-pass filtering, peak and trough detection, baseline correction, and abnormal cycle filtering. Figure 5 The specific process of preprocessing is as follows:

[0102] S201. The original pulse wave data collected by S102 is low-pass filtered using the Blackman window function. In terms of filter design, the sampling rate, passband cutoff frequency, stopband cutoff frequency, filter order and stopband minimum attenuation are set to 100Hz, 10Hz, 15Hz, 111 and -74dB respectively. Original.

[0103] S202 : Using a dual-window search method with two different lengths to perform peak and trough detection on the pulse wave data after low-pass filtering, obtain a peak point sequence and a trough point sequence.

[0104] The two search windows are 0.15 seconds and 0.65 seconds long, respectively. Considering the possibility of duplicate detection of the same peak, a minimum peak distance limit is added, with the minimum distance between adjacent peaks defined as no less than 0.3 seconds. This ensures that distinct peaks are detected and avoids over-detection.

[0105] S203 , using the segmented cubic Hermite interpolation method to obtain a baseline drift curve based on the trough point sequence obtained in S202 , and subtracting the baseline drift curve from the pulse wave data after low-pass filtering to complete the baseline correction process.

[0106] S204. For the pulse wave data after baseline correction, all the original cycles in the pulse wave data are segmented according to the trough point sequence obtained in S202; first, all the segmented original cycles are interpolated and resampled so that all the original cycles have the same length to achieve cycle standardization; then, the K-means clustering algorithm is used to divide all the standardized cycles into three categories of cycle clusters; then, the average cycle template in the cycle cluster with the largest number of cycles is calculated; then, the Pearson correlation coefficient of this average cycle template is calculated with all the standardized cycles one by one, and the standardized cycles with correlation coefficients lower than 0.9 are eliminated. Finally, the original cycles corresponding to the retained standardized cycles are connected in order in the original order to complete the signal reconstruction and achieve the purpose of abnormal cycle filtering. At this step, the preprocessing of the pulse wave is completed. The schematic diagram of the pulse wave after preprocessing in this embodiment is shown as follows. Figure 6 shown.

[0107] S205. Extract 21 features from the pulse wave data after abnormal cycle filtering from three levels: physiological indicators, pulse rate variability, and nonlinear characteristics: heart rate (HR), minimum pulse rate (MinPR), maximum pulse rate (MaxPR), mean pulse interval sequence (MeanPP), median pulse interval sequence (MedianPR), standard deviation of pulse interval sequence (SDPP), mean square value of difference between adjacent intervals of pulse interval sequence (RMSSD), coefficient of variation of continuous difference (CVSD), normalized coefficient of variation (CVPP), skewness of pulse interval sequence (SkewPP), kurtosis of pulse interval sequence (KurtPP), very low frequency power (VLF), low frequency power (LF), high frequency power (HF), total power (TP), ratio of low frequency power to high frequency power (LF / HF), Shannon entropy, slope entropy, permutation entropy, sample entropy, and multi-scale entropy. The schematic diagram of feature extraction from the pre-processed pulse wave in this embodiment is shown in FIG. Figure 7 shown.

[0108] Among them, heart rate (HR) belongs to the physiological indicator feature; minimum pulse rate (MinPR), maximum pulse rate (MaxPR), mean pulse interval sequence (MeanPP), median pulse interval sequence (MedianPR), standard deviation of pulse interval sequence (SDPP), mean square value of difference between adjacent intervals of pulse interval sequence (RMSSD), coefficient of variation of continuous difference (CVSD), normalized coefficient of variation (CVPP), skewness of pulse interval sequence (SkewPP), and kurtosis of pulse interval sequence (KurtPP) belong to the time domain features of pulse rate variability; very low frequency band power (VLF), low frequency band power (LF), high frequency band power (HF), total power (TP), and the ratio of low frequency band power to high frequency band power (LF / HF) belong to the frequency domain features of pulse rate variability; Shannon entropy, slope entropy, permutation entropy, sample entropy, and multi-scale entropy belong to nonlinear features; then the above features are combined to obtain the pulse wave feature vector. Subsequently, the pulse wave feature vector during playing is subtracted from the pulse wave feature vector during sitting meditation to obtain the true feature vector, thereby achieving the purpose of removing individual differences in features.

[0109] S3. Perform sequence comparison processing on the playing data to obtain the playing accuracy feature as follows:

[0110] S301, use the longest common subsequence algorithm to perform sequence alignment processing on the actual playing data of the subject; compare the actual playing data sequence with the standard playing data sequence, assuming that the lengths of the standard playing sequence and the actual playing sequence are M and N respectively; use the standard playing sequence as the vertical axis and the actual playing sequence as the horizontal axis, construct a two-dimensional dynamic programming table according to the recursive relationship, generate a backtracking path, and obtain a specific longest common subsequence, that is, a subsequence with the correct position order. The sequence alignment processing diagram of the playing data in this embodiment is shown in FIG. Figure 8 As shown, the actual playing sequence is

[1324] , the standard sequence is

[1234] , M and N are both 4, the backtracking path is the squares connected by the curve with arrows, and the elements corresponding to the dark squares in the backtracking path are the subsequence

[124] with the correct position order.

[0111] S302: Set the duration error to 30%. Compare the actual duration of each note in the subsequence obtained in S301 with the standard duration. Remove notes with duration errors greater than 30%. The remaining notes are considered correctly played notes. Divide the number of correctly played notes by the maximum of M and N to obtain the playing accuracy rate. In this example, the duration errors of notes 1 and 2 in the subsequence are less than 30%, while the duration error of note 4 is greater than 30%. Therefore, notes 1 and 2 are retained, and note 4 is removed. The number of correctly played notes is 2, and M and N are both 4. According to the formula, the playing accuracy rate is 0.5.

[0112] S4, perform feature screening on all features consisting of baseline features and playing accuracy features, and construct a mental workload classification model with the best feature combination screened out to obtain the classification posterior probability. Figure 9 The process is as follows:

[0113] S401. Combine the true feature vector containing 21 features obtained in S205 and the playing accuracy of the playing features obtained in S302 to obtain a feature vector containing 22 features.

[0114] S402. Add a mental load label to the feature vector obtained in S401 according to the difficulty of playing. Set the label of playing low-difficulty vocal tablature to 0, medium-difficulty tablature to 1, and high-difficulty tablature to 2 to obtain a playing feature vector.

[0115] S403: Extract the playing feature vectors of all subjects and construct a playing feature set.

[0116] S404: Divide the playing feature set into a training set and a test set in a ratio of 8:2.

[0117] S405. Recursive feature elimination combined with cross-validation algorithm is used to optimize and screen the playing feature set. Through the cyclic process of iterative modeling and feature elimination, classification accuracy is used as the feature importance evaluation indicator. The classification accuracy scores of each feature combination are compared through cross-validation, and the best feature combination with the highest classification accuracy is gradually screened out.

[0118] S406. Input the obtained optimal feature combination into the deep forest model, train the deep forest model, establish a mapping relationship between the playing feature vector and the mental workload level, and construct a mental workload level classification model.

[0119] The Deep Forest model's hyperparameters set the number of cascade forest layers to 2, the number of decision trees per forest to 90, and multi-granularity scanning to disabled. The Deep Forest classification model's classification accuracy on the test set reflects the model's overall assessment accuracy for all samples. In this example, the Deep Forest classification model achieved an accuracy of 88.9%.

[0120] S407. The mental workload classification model constructed in S406 can output the posterior probabilities of the three categories of low load, medium load and high load, and the probability of being judged as low load is set as P0, the probability of being judged as medium load is set as P1, and the probability of being judged as high load is set as P2.

[0121] S5. Combine the classification posterior probability to establish a mental workload quantification formula. The process of calculating the mental workload score when playing the vocal tablature is as follows:

[0122] S501. Combine the three posterior probabilities in S407 and establish a mental workload quantification formula in an exponentially weighted manner, mapping the mental workload score to an interval range of [10, 100], where the low workload interval is [10, 40], the medium workload interval is [40, 70], and the high workload interval is [70, 100].

[0123] S502, collecting pulse wave data of a subject while meditating, and pulse wave data and playing data of a subject while playing a finger music score, to obtain a set of meditating pulse wave data, a set of playing pulse wave data and a corresponding set of playing data;

[0124] S503, the three data of the subject are processed through steps S2, S3, and S4 to obtain a low load probability P0 = 0.7, a medium load probability P1 = 0.2, and a high load probability P2 = 0.1;

[0125] S504. Using the mental workload quantification formula established in S501 for the three classification posterior probabilities of S503, the subject's mental workload score when playing the tablature is calculated to be 21.2, which is within the low-load range of [10, 40]. Therefore, the learning method can be adjusted, such as increasing the difficulty of playing.

[0126] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be considered as equivalent replacement methods and are included in the scope of protection of the present invention.

Claims

1. A method for quantitatively evaluating the mental workload of piano music performance, characterized in that: The following steps are involved: S1, collecting pulse wave data of the subject when sitting quietly, and collecting pulse wave data and playing data of the subject when playing piano music; S2. Preprocessing the collected pulse wave data, extracting features, and performing baseline removal processing to obtain baseline-removed features; S3, performing sequence comparison processing on the collected playing data to obtain playing accuracy characteristics; S4, performing feature screening on all features consisting of the baseline-removed features and the playing accuracy features, constructing a mental workload level classification model with the best feature combination screened out, and obtaining a posterior probability of classification; S5. Based on the classification posterior probability, a mental workload quantification formula is established to calculate the mental workload score of the subject when playing the piano sheet music.

2. The method for quantitatively evaluating the mental workload of piano music sheet playing according to claim 1, characterized in that: The step S1 comprises the following steps: S101, establishing a piano music playing system equipped with a pulse wave acquisition module, having the subject play with their right hand and use their left hand to collect pulse wave data; S102. Collect pulse wave data when sitting quietly, and pulse wave data and playing data when playing with low difficulty, medium difficulty and high difficulty respectively.

3. The method for quantitatively evaluating the mental workload of piano music playing according to claim 1, characterized in that: The step S2 comprises the following steps: S201, performing low-pass filtering on the pulse wave data using a Blackman window; S202, performing peak and trough detection on the pulse wave data after low-pass filtering using a dual-window search method to obtain a peak point sequence and a trough point sequence; S203, performing baseline correction on the pulse wave data after low-pass filtering using a piecewise cubic Hermite interpolation method based on the trough point sequence obtained in S202; S204. Segment all original cycles in the pulse wave data after baseline correction based on the trough point sequence obtained in S202. Interpolate and resample all segmented original cycles to obtain standardized cycles of the same length. Use the K-means clustering algorithm to divide all standardized cycles into three categories, calculate the average cycle template in the category with the largest number of cycles, calculate the correlation coefficient between this average cycle template and all standardized cycles one by one, eliminate standardized cycles with correlation coefficients lower than 0.9, and complete the abnormal cycle filtering process. S205. Extract a pulse wave feature vector from the pulse wave data after abnormal cycle filtering; wherein the pulse wave feature vector includes 21 features: heart rate, minimum pulse rate, maximum pulse rate, pulse interval sequence mean, pulse interval sequence median, pulse interval sequence standard deviation, mean square value of differences between adjacent intervals of the pulse interval sequence, continuous difference variation coefficient, normalized coefficient of variation, pulse interval sequence skewness, pulse interval sequence kurtosis, very low frequency band power, low frequency band power, high frequency band power, total power, ratio of low frequency band power to high frequency band power, Shannon entropy, slope entropy, permutation entropy, sample entropy, and multi-scale entropy; subtract the pulse wave feature vector during meditation from the pulse wave feature vector during playing to obtain a true feature vector.

4. The method for quantitatively evaluating the mental workload of piano music sheet playing according to claim 3, characterized in that: The step S3 comprises the following steps: S301, performing sequence alignment processing on the subject's playing data using the longest common subsequence algorithm to obtain a subsequence with the correct position order; S302. Compare the actual duration of each note in the subsequence with the standard duration, eliminate notes with a duration error greater than 30%, and divide the number of retained notes by the maximum length of the actual playing sequence and the standard playing sequence to obtain the playing accuracy.

5. The method for quantitatively evaluating the mental workload of piano music playing according to claim 4, characterized in that: The step S4 comprises the following steps: S401, combining a set of real feature vectors including the 21 features and the playing accuracy feature to obtain a feature vector including 22 features; S402, adding a mental load label to the feature vector according to the difficulty of playing, setting the label of playing the low-difficulty vocal tablature to 0, the medium-difficulty tablature to 1, and the high-difficulty tablature to 2, to obtain a playing feature vector sample; S403, having multiple subjects play the three levels of difficulty of the vocal notation in sequence, obtaining multiple playing feature vector samples, and constructing a playing feature set; S404, dividing the playing feature set into a training set and a test set in a ratio of 8:2; S405, performing recursive feature elimination combined with cross-validation processing on the playing feature set to screen out the best feature combination; S406. Input the obtained optimal feature combination into the deep forest model, train the deep forest model, and construct a mental workload level classification model; S407. Output the posterior probabilities of the three categories of low load, medium load and high load through the mental workload level classification model, and set the posterior probability of low load as P0, the posterior probability of medium load as P1, and the posterior probability of high load as P2.

6. The method for quantitatively evaluating the mental workload of piano music performance according to claim 5, characterized in that: The step S5 comprises the following steps: S501, combining the three posterior probabilities in step S407 and establishing a mental workload quantification formula in an exponentially weighted manner; S502, collecting pulse wave data of a subject while sitting in meditation, and pulse wave data and playing data of a subject while playing piano music, to obtain a set of sitting pulse wave data, a set of playing pulse wave data and a corresponding set of playing data; S503, processing the three data of the subject collected in step S502 through steps S2, S3, and S4 to obtain the posterior probabilities P0, P1, and P2 of the three classifications; S504 , using the mental workload quantification formula established in S501 on the posterior probabilities of the three categories obtained in step S503 , to calculate the mental workload score of the subject when playing the piano sheet music.

Images