Chinese zither playing skill intelligent identification method and system based on acoustic feature fusion
By employing methods such as data acquisition, filtering, framing, and multi-domain feature fusion, combined with a KNN classifier, the problem of insufficient accuracy in recognizing guzheng playing techniques was solved, achieving efficient and accurate recognition and automated analysis of guzheng playing techniques.
Patent Information
- Application Number
- CN202511601441.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-04
- Publication Date
- 2026-01-09
AI Technical Summary
Existing technologies struggle to effectively integrate multi-domain acoustic features, resulting in limited accuracy in recognizing guzheng playing techniques, particularly in terms of generalization ability and classification accuracy when faced with a wide variety of playing techniques.
After collecting audio data and performing moving average filtering for noise reduction, the data is processed frame by frame. Combined with time domain, frequency domain, and time-frequency domain feature extraction, a comprehensive feature vector is constructed, and a KNN classifier is used for intelligent recognition of guzheng playing techniques.
It achieves precise and intelligent recognition of guzheng playing techniques, improving recognition accuracy and efficiency, and enabling automated analysis and teaching assistance.
Smart Images
Figure CN121306191A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of audio analysis technology, specifically to a method and system for intelligent recognition of guzheng playing techniques based on acoustic feature fusion. Background Technology
[0002] In guzheng performance, accurate identification of playing techniques is of great significance for guzheng teaching, music analysis, and the development of intelligent auxiliary systems. Traditional guzheng playing technique identification mainly relies on manual auditory discrimination. This method is not only time-consuming and laborious, but also greatly affected by subjective factors, making it difficult to guarantee accuracy and efficiency.
[0003] With the development of digital signal processing and artificial intelligence technologies, the recognition of musical instrument playing techniques based on audio features has become a research hotspot. Currently, research on the recognition of guzheng playing techniques has made some progress, with some methods classifying and recognizing techniques by extracting time-domain, frequency-domain, or single time-frequency domain features from the audio. For example, some studies use only time-domain features such as short-time energy and short-time zero-crossing rate, or only frequency-domain energy distribution features obtained from Fourier transform, to distinguish different playing techniques. However, these single-domain features often only reflect part of the characteristics of the guzheng sound, failing to capture subtle acoustic differences between techniques, resulting in limited recognition accuracy.
[0004] Some studies have attempted to combine features from both domains, but these studies still fall short in terms of the comprehensiveness and effectiveness of feature fusion. Furthermore, existing classification models need improvement in generalization ability and classification accuracy when faced with the rich and diverse playing techniques of the guzheng.
[0005] Furthermore, the playing techniques of the guzheng are inherently complex, including frequency fluctuations in vibrato, pitch variations in glissando, and timbre enhancement through string vibrato. The differences in acoustic characteristics between different techniques are subtle, requiring more comprehensive feature extraction and more efficient classification methods for accurate identification. Therefore, how to integrate multi-domain acoustic features and construct an efficient classification and recognition system to achieve accurate and rapid identification of various guzheng playing techniques has become a pressing technical problem to be solved. Summary of the Invention
[0006] (a) Technical problems to be solved
[0007] To address the shortcomings of existing technologies, this invention provides a method and system for intelligent recognition of guzheng playing techniques based on acoustic feature fusion, which solves the problems mentioned in the background.
[0008] (II) Technical Solution
[0009] To achieve the above objectives, the present invention provides the following technical solution: an intelligent recognition method for Guzheng playing techniques based on acoustic feature fusion, comprising the following steps:
[0010] Step 1: Data Acquisition and Preprocessing
[0011] Audio data is collected using a microphone, then subjected to moving average filtering for noise reduction, and finally divided into frames according to a pre-defined frame length and frame shift to obtain multi-frame audio signals.
[0012] Step 2: Temporal Feature Extraction
[0013] For the segmented audio signal, calculate the short-time energy and short-time zero-crossing rate of each frame;
[0014] Step 3: Frequency Domain Feature Extraction
[0015] The audio signal after being framed is subjected to a fast Fourier transform to obtain the frequency domain signal. Then, the frequency intervals are divided, and the energy of each interval and the proportion of the corresponding interval energy to the total energy are calculated.
[0016] Step 4: Joint Time-Frequency Domain Feature Extraction
[0017] Time-frequency analysis of audio signals is performed using wavelet transform to calculate wavelet energy and wavelet energy ratio at each scale.
[0018] Step 5: Feature Fusion
[0019] The time-domain features corresponding to short-time energy and zero-crossing rate, the frequency-domain features corresponding to the energy proportion of each frequency interval, and the time-frequency features corresponding to the energy proportion of each scale wavelet are fused into a comprehensive vector.
[0020] Step 6: Classification and Recognition
[0021] A sample library containing various guzheng playing techniques was constructed and manually labeled. Then, the feature vectors were associated with the labels, and a KNN classifier was trained. Guzheng playing techniques were identified through nearest neighbor search and voting, and the accuracy was verified by samples.
[0022] As a further aspect of the present invention, the moving average filtering noise reduction method is as follows:
[0023] The audio data is labeled as X(n), where n = 0, 1, 2, ..., N−1, and N is the total number of sampling points of the audio data; at the same time, the length of the filtering window is defined as M;
[0024] pass: The signal value X at the nth sampling point after filtering is obtained. f (n);
[0025] The principle is to use the averaging effect of local area data to reduce the interference of random noise on the original signal.
[0026] As a further aspect of the present invention: for framed audio signals, the frame length is set to L=1024 sampling points, and the frame shift is set to S=512 sampling points.
[0027] As a further aspect of the present invention: for the segmented audio signal, the audio signal of the i-th frame is marked as X. i (m), where m = 0, 1, ..., L−1, i = 0, 1, ..., U−1, and U is the total number of frames.
[0028] As a further aspect of the present invention, the temporal feature extraction method is as follows:
[0029] Short-term energy extraction:
[0030] pass: Calculate the short-time energy E of the i-th frame. i ;
[0031] Among them, X i (m) is the audio signal value of the m-th sample point in the i-th frame, and L is the frame length;
[0032] The principle is to quantify the energy intensity of the signal over a short period of time to reflect the changes in the intensity of the sound during that period.
[0033] Short-time zero-crossing rate extraction:
[0034] pass: Calculate the short-time zero-crossing rate Z of the i-th frame. i ;
[0035] Where sgn[] is the sign function, which outputs 1 when the input is greater than 0, 0 when the input is equal to 0, and -1 when the input is less than 0;
[0036] The principle is to count the number of times the signal crosses zero in a short period of time to distinguish the signal characteristics of different timbres.
[0037] As a further aspect of the present invention, the frequency domain feature extraction method is as follows:
[0038] For the framed signal X i (m) Perform a Fast Fourier Transform (FFT) to obtain the frequency domain signal X. i (a);
[0039] The calculation formula is as follows:
[0040] Where a = 0, 1, ..., L−1, j is the imaginary unit, L is the frame length, and a is the frequency point index;
[0041] That is, by decomposing a discrete signal in the time domain into sine and cosine components of different frequencies according to the Fourier transform principle, the signal is transformed from the time domain to the frequency domain, thereby enabling the analysis of the frequency characteristics of the signal.
[0042] Spectral energy distribution characteristics:
[0043] First, the frequency range is divided into A intervals. The frequency range of the p-th interval is [f p−1 ,f p The corresponding frequency point index range is [a] p−1 ,a p ];
[0044] pass: Calculate the energy E of the p-th frequency interval in the i-th frame. i,p ;
[0045] The principle is to quantify the energy of a signal within a specific frequency range, reflecting the signal strength within that frequency range;
[0046] pass: Calculate the proportion R of the energy in this interval to the total energy. i,p ;
[0047] Among them, |X i (a) | is a frequency domain signal X i (a) is the amplitude, where A is the number of frequency intervals;
[0048] The principle is to calculate the proportion of energy in a single frequency range in the total energy, thereby clarifying the contribution of different frequency ranges to the overall signal.
[0049] As a further aspect of the present invention: the time-frequency analysis selects the db4 wavelet as the mother wavelet and performs 5-level wavelet decomposition;
[0050] The coefficients of the i-th frame at different scales after wavelet decomposition are denoted as W. i,j (m), where j = 1, 2, ..., 5, j represents the scale, and m represents the time point.
[0051] As a further aspect of the present invention, the joint time-frequency domain feature extraction method is as follows:
[0052] Wavelet energy characteristics:
[0053] pass: Calculate the wavelet energy WE of the i-th frame at the j-th scale. i,j ;
[0054] Among them, M j W is the number of time points at the j-th scale. i,j (m) are the wavelet coefficients of the i-th frame at the j-th scale;
[0055] The principle is to decompose the signal by wavelet transform, quantize the energy of the signal at different scales, and capture the local time-frequency features of the signal.
[0056] Wavelet energy proportion characteristics:
[0057] pass: Calculate the wavelet energy ratio WER at scale j in frame i. i,j ;
[0058] The principle is to determine the proportion of wavelet energy at a single scale in the total wavelet energy, highlighting the differences in the contribution of different scales to the time-frequency characteristics of the signal.
[0059] As a further aspect of the present invention: wherein the comprehensive feature vector after multi-dimensional features is obtained through F i =[E i Z i R i,1 ...R i,K WER i,1 ...WER i,5 ]mark;
[0060] E i For short-time energy, Z i For short-time zero crossing rate, R i,p For the energy percentage of each frequency range, WER i,j The proportion of wavelet energy at each scale;
[0061] The feature fusion methods are as follows:
[0062] First, each feature is normalized to eliminate dimensional differences. Normalization transforms the original physical quantity into a dimensionless relative value and maps it to a specific interval, such as [0,1] or [-1,1]. This eliminates the influence of units while preserving the relative magnitude of the physical quantities.
[0063] The formula is derived from software simulation using a large amount of collected data to obtain the most recent real-world situation. The preset parameters and threshold selections in the formula are set by those skilled in the art according to the actual situation.
[0064] The normalized features are then arranged in a fixed order to form a comprehensive feature vector.
[0065] As a further aspect of the present invention: Guzheng playing techniques encompass vibrato, glissando, string vibrato, plucking, and pressing;
[0066] For each sample in the sample library, the manual annotations are as follows: vibrato is labeled with "1", glissando with "2", vibrato with "3", plucked notes with "4", and pressed notes with "5".
[0067] As a further aspect of the present invention: the feature vector is associated with the label in the following way:
[0068] For each labeled sample in the sample library, the comprehensive feature vector F is extracted by following the steps of data acquisition and preprocessing, time domain feature extraction, frequency domain feature extraction, time-frequency joint feature extraction, and feature fusion. A one-to-one correspondence is established with the corresponding skill category label V to form a training sample pair (F,V).
[0069] Where V∈{1, 2, ..., NV}, and NV is the total number of skill categories.
[0070] As a further aspect of the present invention, the steps for training using a K-Nearest Neighbor (KNN) classifier are as follows:
[0071] Step 1: Determine the value of K:
[0072] Choose K=5;
[0073] Step 2: Definition of distance metric:
[0074] The similarity between feature vectors is calculated using Euclidean distance. For two feature vectors Fa = [fa1, fa2, ..., fa...] D ] and Fb=[fb1, fb2, ..., fb D ];
[0075] Where D is the feature dimension;
[0076] The Euclidean distance d(Fa,Fb) is calculated using the following formula: ;
[0077] The principle is to measure the similarity between two feature vectors by quantifying the straight-line distance between two points in the vector space; the smaller the distance, the higher the similarity.
[0078] Step 3: Model Training
[0079] Store all training sample pairs (F,V) into the sample library to complete the training of the KNN classifier.
[0080] As a further aspect of the present invention, the nearest neighbor search and voting methods are as follows:
[0081] For the input guzheng audio data to be identified, the comprehensive feature vector F is extracted following the same process as for the training samples. unknown ;
[0082] Calculate distance F unknown The Euclidean distance d(F) between the feature vector F of all training samples in the sample library and the feature vector F of all training samples in the sample library. unknown ,F);
[0083] Select K nearest neighbors from the sample database, with a distance F. unknown Record the skill category labels of the K most recent training samples;
[0084] The voting decision involves voting on these K labels, and the category with the most votes is F. unknown The corresponding guzheng playing technique category is determined by the following method: in the K nearest neighbor samples, the number of votes obtained for each category V is counted, and the category V with the most votes is selected as the final predicted category VY.
[0085] The formula is expressed as:
[0086] Where VY represents the final predicted category; argmax V This indicates finding the class V that maximizes the following expression; K represents the number of nearest neighbor samples selected; c = 1, 2, ..., K, H(V) c =V) means: if the class of the c-th nearest neighbor sample is V c If the value is equal to category V, return 1; otherwise, return 0. c Let c be the category label of the c-th nearest neighbor.
[0087] The principle is to use the majority voting principle to determine the category of the sample to be identified based on the category labels of the nearest samples, thus ensuring the reliability of the classification results.
[0088] As a further aspect of the present invention, the accuracy verification method is as follows:
[0089] 1200 samples of guzheng playing techniques were selected, including 6 different techniques, with 200 samples of each technique, and divided into training set and test set in an 8:2 ratio;
[0090] After training the KNN classifier using the training set, perform recognition tests on the test set and calculate the recognition accuracy:
[0091]
[0092] The principle is to evaluate the recognition performance of the classification model by quantifying the proportion of correctly identified samples; the higher the ratio, the better the model's recognition effect.
[0093] A Guzheng playing technique intelligent recognition system based on acoustic feature fusion is provided. This system implements a method and system approach for intelligent recognition of Guzheng playing techniques based on acoustic feature fusion. The system includes:
[0094] Acquisition and processing unit: Acquires audio data using a microphone, performs moving average filtering for noise reduction, and then divides the data into frames according to a pre-defined frame length and frame shift to obtain multiple frames of audio signal;
[0095] The feature extraction unit is used for extracting time-domain features, frequency-domain features, and joint time-frequency-domain features.
[0096] The feature fusion unit is used to fuse the time-domain features corresponding to short-time energy and zero-crossing rate, the frequency-domain features corresponding to the energy proportion of each frequency interval, and the time-frequency features corresponding to the energy proportion of each scale wavelet into a comprehensive vector.
[0097] The classification and recognition unit is used to build a sample library containing various guzheng playing techniques and manually label it. Then, the feature vector is associated with the label, and a classifier is trained using KNN. The guzheng playing techniques are identified through nearest neighbor search and voting, and the accuracy is verified by samples.
[0098] (III) Beneficial Effects
[0099] This invention provides a method and system for intelligent recognition of guzheng playing techniques based on acoustic feature fusion. Compared with existing technologies, it has the following advantages:
[0100] This invention achieves accurate and intelligent recognition of guzheng playing techniques through multi-domain acoustic feature fusion. In the data acquisition and preprocessing stage, a high-sensitivity microphone is used to collect audio, combined with moving average filtering for noise reduction, ensuring the quality of the original audio. Simultaneously, frame-by-frame processing lays the foundation for subsequent feature extraction. Short-time energy and zero-crossing rate are extracted in the time domain, effectively distinguishing between loud and soft playing, and plucking and vibrato techniques. In the frequency domain, Fourier transform analysis of the spectral energy distribution captures the frequency energy differences of different techniques. Wavelet transform is used in the time-frequency domain to extract multi-scale energy features, further enriching the feature dimensions. The fusion of multi-domain features enables the comprehensive feature vector to fully characterize the acoustic properties of guzheng playing techniques, providing rich and effective feature support for accurate recognition.
[0101] The classification and recognition system constructed in this invention is highly efficient and reliable. The sample library covers a variety of typical guzheng playing techniques, with sufficient samples in each category. Manual annotation ensures the accuracy of the labels, providing high-quality data for model training. The KNN classifier, with its parameterless model characteristics, simplifies the training process; training can be completed simply by storing the sample library. Simultaneously, the Euclidean distance metric and voting decision mechanism effectively determine the category of unknown samples. Model validation employs reasonable sample partitioning and multiple technique categories. Test results show that the method has a high recognition accuracy and can meet the practical needs of intelligent recognition of guzheng playing techniques.
[0102] This invention has significant application value and promotional implications. In the field of music education, it can help learners quickly identify deficiencies in their playing techniques, enabling personalized learning and precise improvement. In the field of music appreciation and analysis, it can provide objective and quantitative evidence for the technical analysis of guzheng works, facilitating a deeper understanding of the performance art of the pieces. From a technical perspective, its approach of combining multi-domain acoustic feature fusion with KNN classification can provide a reference for the intelligent recognition research of other ethnic musical instruments or musical elements, promoting the development of intelligent music analysis technology, and also expanding new directions for the application of acoustic signal processing in the arts. Attached Figure Description
[0103] Figure 1 This is a system block diagram of the intelligent recognition method and system for guzheng playing techniques based on acoustic feature fusion, which is based on the present invention.
[0104] Figure 2 This is a flowchart illustrating the intelligent recognition method and system for guzheng playing techniques based on acoustic feature fusion, as described in this invention. Detailed Implementation
[0105] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0106] Please see Figure 1 and Figure 2 As shown, the embodiments of the present invention provide the following technical solutions:
[0107] As an embodiment of the present invention:
[0108] This invention relates to an intelligent recognition method and system for guzheng playing techniques based on acoustic feature fusion, comprising steps of data acquisition and preprocessing, as follows:
[0109] First, a high-sensitivity microphone is used to collect audio data during the guzheng performance, resulting in audio data X(n), where n = 0, 1, 2, ..., N−1, and N is the total number of audio data sampling points.
[0110] In this embodiment, the sampling frequency is set to fs = 44100 Hz and the sampling bit depth is 16 bits;
[0111] Subsequently, a moving average filtering method was used to filter and reduce noise in the acquired audio signal, as follows:
[0112] Define the filter window length as M;
[0113] pass: The signal value X at the nth sampling point after filtering is obtained. f (n);
[0114] In this embodiment, the value of M can be adjusted according to the actual noise situation, and its range is 50≤M≤200;
[0115] Next, the denoised audio signal is processed by frame segmentation:
[0116] The frame length is L=1024 sampling points, and the frame shift is S=512 sampling points;
[0117] Label the audio signal of the i-th frame as X. i (m), where m = 0, 1, ..., L−1, i = 0, 1, ..., U−1, and U is the total number of frames.
[0118] This embodiment employs a high-sensitivity microphone to capture guzheng performance audio and combines it with moving average filtering for noise reduction, effectively minimizing noise interference with the audio signal. Simultaneously, by appropriately setting the sampling frequency, sampling bit depth, frame length, and frame shift for frame-by-frame processing, a high-quality audio data foundation is laid for subsequent feature extraction and recognition. Its preprocessing process balances data integrity and accuracy, adapts to different noise environments, and provides reliable raw data support for guzheng performance technique recognition.
[0119] As a second embodiment of the present invention:
[0120] In specific implementation, compared with Embodiment 1, the technical solution of this embodiment differs from Embodiment 1 only in that this embodiment also includes feature extraction and feature fusion steps, as follows:
[0121] Feature extraction includes: time-domain feature extraction, frequency-domain feature extraction, and joint time-frequency-domain feature extraction.
[0122] The temporal feature extraction method is as follows:
[0123] Short-time energy extraction: Short-time energy describes the energy changes of a signal over a short period of time, and is of great significance for distinguishing different playing techniques of the guzheng (such as loud and soft playing).
[0124] pass: Calculate the short-time energy E of the i-th frame. i ;
[0125] Among them, X i (m) is the audio signal value of the m-th sample point in the i-th frame, and L is the frame length;
[0126] Short-time zero-crossing rate extraction: The short-time zero-crossing rate reflects the number of times a signal crosses zero in a short period of time, and can be used to distinguish between plucking techniques and vibrato techniques on the guzheng.
[0127] pass: Calculate the short-time zero-crossing rate Z of the i-th frame. i ;
[0128] Where sgn[] is the sign function, which outputs 1 when the input is greater than 0, 0 when the input is equal to 0, and -1 when the input is less than 0;
[0129] The frequency domain feature extraction method is as follows:
[0130] Fourier transform: for the framed signal X i (m) Perform a Fast Fourier Transform (FFT) to obtain the frequency domain signal X. i (a);
[0131] The calculation formula is as follows:
[0132] Where a = 0, 1, ..., L−1, j is the imaginary unit, L is the frame length, and a is the frequency point index;
[0133] Spectral energy distribution characteristics: Calculate the energy proportion of each frequency range to distinguish the differences in energy distribution in the frequency domain for different playing techniques;
[0134] First, the frequency range is divided into A intervals. The frequency range of the p-th interval is [f p−1 ,f p The corresponding frequency point index range is [a] p−1 ,a p ];
[0135] pass: Calculate the energy E of the p-th frequency interval in the i-th frame. i,p ;
[0136] pass: Calculate the proportion R of the energy in this interval to the total energy. i,p ;
[0137] Among them, |X i (a) | is a frequency domain signal X i (a) is the amplitude, where A is the number of frequency intervals;
[0138] The joint time-frequency domain feature extraction method is as follows: wavelet transform is used to perform time-frequency analysis on the Guzheng audio signal, and the db4 wavelet is selected as the mother wavelet for 5-level wavelet decomposition.
[0139] After wavelet decomposition, the coefficients W at different scales are obtained for the i-th frame. i,j(m), where j = 1, 2, ..., 5, j represents the scale, and m represents the time point;
[0140] Wavelet energy characteristics:
[0141] Calculate the wavelet energy at each scale;
[0142] pass: Calculate the wavelet energy WE of the i-th frame at the j-th scale. i,j ;
[0143] Among them, M j W is the number of time points at the j-th scale. i,j (m) are the wavelet coefficients of the i-th frame at the j-th scale;
[0144] Wavelet energy proportion characteristics:
[0145] Calculate the proportion of wavelet energy to total wavelet energy at each scale;
[0146] pass: Calculate the wavelet energy ratio WER at scale j in frame i. i,j ;
[0147] The feature fusion method is as follows: short-time energy E i Short-time zero-crossing rate Z i Corresponding time-domain characteristics, energy proportion R in each frequency range i,p The corresponding frequency domain characteristics, and the wavelet energy proportion (WER) at each scale. i,j The corresponding time-frequency domain features are fused to form a comprehensive feature vector F. i =[E i Z i R i,1 ...R i,K WER i,1 ...WER i,5 ].
[0148] The feature fusion methods are as follows:
[0149] First, each feature is normalized to eliminate dimensional differences; in this embodiment, the normalization process uses linear normalization to the [0,1] interval.
[0150] The normalized features are then arranged in a fixed order to form a comprehensive feature vector.
[0151] Building upon Example 1, this example comprehensively captures the acoustic characteristics of different guzheng playing techniques by extracting time-domain, frequency-domain, and time-frequency domain combined features. Specifically, short-time energy and zero-crossing rate effectively distinguish the time-domain differences between techniques such as dynamics, plucking, and vibrato; the frequency-domain spectral energy distribution characteristics reflect the differences in the frequency energy proportions of different techniques; and the time-frequency domain features obtained through wavelet transform accurately characterize the joint changes of the signal in time and frequency. The comprehensive feature vector formed through feature fusion integrates multi-dimensional acoustic information, significantly improving the comprehensiveness and accuracy of the description of guzheng playing techniques.
[0152] As an embodiment of the present invention:
[0153] In its specific implementation, compared to Embodiment 1 and Embodiment 2, the technical solution of this embodiment is to combine the solutions of Embodiment 1 and Embodiment 2. The only difference between this embodiment and Embodiment 1 and Embodiment 2 is that this embodiment also includes a classification and identification step, and the classification and identification method is as follows:
[0154] Sample library construction:
[0155] Collect audio samples covering typical guzheng playing techniques such as vibrato, glissando, string vibrato, plucking, and pressing, with no less than 300 valid samples collected for each technique category;
[0156] Each sample was manually labeled:
[0157] Clearly identify the skill category label to which it belongs, for example: label "1" represents vibrato, label "2" represents glissando, label "3" represents vibrato, label "4" represents plucking, and label "5" represents stopped notes;
[0158] Associating feature vectors with labels:
[0159] For each labeled sample, following the steps of data acquisition and preprocessing, time-domain feature extraction, frequency-domain feature extraction, time-frequency joint feature extraction, and feature fusion, its comprehensive feature vector F is extracted and a one-to-one correspondence is established with the corresponding skill category label V to form a training sample pair (F,V).
[0160] Where V∈{1, 2, ..., NV}, and NV is the total number of skill categories;
[0161] Classification model training:
[0162] The K-Nearest Neighbors (KNN) classifier is used for training, and the steps are as follows:
[0163] Step 1: Determine the value of K:
[0164] Choose K=5;
[0165] The value of K can be adjusted according to the experiment, and is generally taken as an odd number between 3 and 10 to avoid a tie in voting;
[0166] Step 2: Definition of distance metric:
[0167] The similarity between feature vectors is calculated using Euclidean distance. For two feature vectors Fa = [fa1, fa2, ..., fa...] D ] and Fb=[fb1, fb2, ..., fb D ];
[0168] Where D is the feature dimension;
[0169] The Euclidean distance d(Fa,Fb) is calculated using the following formula: ;
[0170] Step 3: Model Training
[0171] Store all training sample pairs (F,V) into the sample library to complete the training of the KNN classifier;
[0172] In this embodiment, KNN is a parameter-free model, and the training process is actually a process of storing the sample database.
[0173] Feature extraction of the sample to be identified:
[0174] For the input guzheng audio data to be identified, the comprehensive feature vector F is extracted following the same process as for the training samples. unknown ;
[0175] Nearest neighbor search and category determination:
[0176] Step 1: Calculate the distance F unknown The Euclidean distance d(F) between the feature vector F of all training samples in the sample library and the feature vector F of all training samples in the sample library. unknown ,F).
[0177] Step 2: Select K nearest neighbors from the sample database, choosing those with a distance F. unknown Record the skill class labels of the K most recent training samples.
[0178] Step 3: Voting Decision. Vote on these K labels; the category with the most votes is F. unknown Corresponding categories of guzheng playing techniques;
[0179] The method is as follows: among the K nearest neighbor samples, count the number of votes obtained by each category V, and select the category V with the most votes as the final predicted category VY;
[0180] The formula is expressed as:
[0181] Where VY represents the final predicted category; argmax V This indicates finding the class V that maximizes the following expression; K represents the number of nearest neighbor samples selected; c = 1, 2, ..., K, H(V) c =V) means: if the class of the c-th nearest neighbor sample is V c If the value is equal to category V, return 1; otherwise, return 0. c Let c be the category label of the c-th nearest neighbor.
[0182] Model validation methods:
[0183] 1200 samples of guzheng playing techniques were selected, including 6 different techniques, with 200 samples of each technique, and divided into training set and test set in an 8:2 ratio;
[0184] After training the KNN classifier using the training set, perform recognition tests on the test set and calculate the recognition accuracy:
[0185] The formula is: .
[0186] This embodiment integrates the solutions of the previous two embodiments, introducing a classification and recognition step. By constructing a sample library covering various typical guzheng playing techniques, and combining it with a KNN classifier, intelligent recognition of playing techniques is achieved. The KNN classifier uses Euclidean distance metric and voting decision mechanism, which is simple, efficient, and easy to implement. Furthermore, the reliability of the model's recognition is ensured through the reasonable division of training and test sets for validation. This solution can automatically classify guzheng playing techniques, improving recognition efficiency and accuracy, and providing effective support for the automated analysis and teaching application of guzheng playing techniques.
[0187] As an embodiment of the present invention:
[0188] In specific implementation, compared with Embodiment 1, Embodiment 2 and Embodiment 3, the technical solution of this embodiment is to combine the solutions of Embodiment 1, Embodiment 2, Embodiment 3 and Embodiment 4.
[0189] This embodiment combines the technical solutions of the first three embodiments to form a complete process from audio data acquisition, preprocessing, feature extraction and fusion to classification and recognition, realizing a full-chain solution for intelligent recognition of guzheng playing techniques. This solution not only ensures high-quality data processing but also further improves the stability and accuracy of the recognition system through multi-dimensional feature fusion and efficient classification algorithms. It can more comprehensively and accurately identify various guzheng playing techniques, providing more comprehensive technical support for the digital analysis of guzheng performance, teaching assistance, and artistic research.
[0190] Experimental results show that this method can achieve an accuracy rate of over 90% in recognizing guzheng playing techniques, and its robustness in small sample scenarios is superior to traditional single-feature recognition methods, demonstrating the inventiveness and practicality of this invention.
[0191] This invention also provides an intelligent recognition system for guzheng playing techniques based on acoustic feature fusion. This system is used to execute a method and system for intelligent recognition of guzheng playing techniques based on acoustic feature fusion. The system includes:
[0192] Acquisition and processing unit: Acquires audio data using a microphone, performs moving average filtering for noise reduction, and then divides the data into frames according to a pre-defined frame length and frame shift to obtain multiple frames of audio signal;
[0193] The feature extraction unit is used for extracting time-domain features, frequency-domain features, and joint time-frequency-domain features.
[0194] The feature fusion unit is used to fuse the time-domain features corresponding to short-time energy and zero-crossing rate, the frequency-domain features corresponding to the energy proportion of each frequency interval, and the time-frequency features corresponding to the energy proportion of each scale wavelet into a comprehensive vector.
[0195] The classification and recognition unit is used to build a sample library containing various guzheng playing techniques and manually label it. Then, the feature vector is associated with the label, and a classifier is trained using KNN. The guzheng playing techniques are identified through nearest neighbor search and voting, and the accuracy is verified by samples.
[0196] It should be stated that all user data collected in this application was collected with the user's consent and authorization, and the use of user data is legal and compliant, and the use and processing of user data comply with the relevant laws, regulations and standards of the relevant regions.
[0197] Furthermore, any content not described in detail in this specification is existing technology known to those skilled in the art.
[0198] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.
[0199] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.
[0200] The various embodiments of this application have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or improvement of the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.
Claims
1. A method for intelligent recognition of guzheng playing techniques based on acoustic feature fusion, characterized in that, Includes the following steps: Data acquisition and preprocessing: Audio data is acquired using a microphone, followed by moving average filtering for noise reduction, and then divided into frames according to the pre-defined frame length and frame shift to obtain multi-frame audio signals. Temporal feature extraction: For the segmented audio signal, calculate the short-time energy and short-time zero-crossing rate of each frame; Frequency domain feature extraction: Perform a fast Fourier transform on the framed audio signal to obtain the frequency domain signal, then divide the frequency intervals, calculate the energy of each interval and the proportion of the corresponding interval energy to the total energy; Joint time-frequency domain feature extraction: Wavelet transform is used to perform time-frequency analysis on audio signals, and wavelet energy and wavelet energy ratio at each scale are calculated. Feature fusion: The time-domain features corresponding to short-time energy and zero-crossing rate, the frequency-domain features corresponding to the energy proportion of each frequency interval, and the time-frequency features corresponding to the energy proportion of each scale wavelet are fused into a comprehensive vector. Classification and Recognition: A sample library containing various guzheng playing techniques was constructed and manually labeled. Then, the feature vectors were associated with the labels, and a classifier was trained using KNN. Guzheng playing techniques were identified through nearest neighbor search and voting, and the accuracy was verified using samples.
2. The intelligent recognition method for guzheng playing techniques based on acoustic feature fusion according to claim 1, characterized in that: The moving average filtering noise reduction method is as follows: The audio data is labeled as X(n), where n = 0, 1, 2, ..., N−1, and N is the total number of sampling points of the audio data; at the same time, the length of the filtering window is defined as M; The signal value X at the nth sampling point after filtering f (n) is obtained by calculating the average value of the audio data of M sampling points before and after the nth sampling point.
3. The intelligent recognition method for guzheng playing techniques based on acoustic feature fusion according to claim 1, characterized in that: The temporal feature extraction method is as follows: Short-term energy extraction: Let X be the audio signal value of the m-th sampling point in the i-th frame of the segmented audio signal. i (m), then X i (m) The short-time energy E of the i-th frame is obtained by squaring the audio signal values at each sampling point and then summing all the squared values. i ; Where m = 0, 1, ..., L−1, L is the frame length; i = 0, 1, ..., U−1, U is the total number of frames; Short-time zero-crossing rate extraction: First, calculate the absolute value of the difference between the sign function results of the audio signal values of two adjacent sampling points in the frame. Then, add all these absolute values together and finally divide by twice the frame length to obtain the short-time zero-crossing rate Z of the i-th frame. i ; The sign function rule is: output 1 if the input is greater than 0, output 0 if the input is equal to 0, and output -1 if the input is less than 0.
4. The intelligent recognition method for guzheng playing techniques based on acoustic feature fusion according to claim 3, characterized in that: The frequency domain feature extraction method is as follows: For the framed signal X i (m) Perform a Fast Fourier Transform (FFT) to obtain the frequency domain signal X. i (a); By decomposing a discrete signal in the time domain into sine and cosine components of different frequencies according to the Fourier transform principle, the signal is transformed from the time domain to the frequency domain. Spectral energy distribution characteristics: First, the frequency range is divided into A intervals. The frequency range of the p-th interval is [f p−1 ,f p The corresponding frequency point index range is [a] p−1 ,a p ]; All frequency domain signals X within this frequency range i (a) The absolute value of the amplitude is squared, and then all squared values are summed to obtain the energy E of the p-th frequency interval in the i-th frame. i,p ; Use the energy E in this interval i,p Divide by the total energy of all frequency ranges in the frame to obtain the proportion R of the energy in that frequency range to the total energy. i,p .
5. The intelligent recognition method for guzheng playing techniques based on acoustic feature fusion according to claim 4, characterized in that: The joint time-frequency domain feature extraction method is as follows: Wavelet energy characteristics: The wavelet coefficients W at all time points under the relevant scale i,j (m) Squaring each component and summing them together, we obtain the wavelet energy WE of the i-th frame at the j-th scale. i,j ; Where j represents the scale, j=1, 2, ..., 5, and m represents the time point, m=0, 1, 2, ..., M. j -1, M j W is the number of time points at the j-th scale. i,j (m) are the wavelet coefficients of the i-th frame at the j-th scale; Wavelet energy proportion characteristics: Through wavelet energy WE at relevant scales i,j Divide by the sum of wavelet energies at all scales in the frame to obtain the wavelet energy percentage (WER) at scale j in the i-th frame. i,j .
6. The intelligent recognition method for guzheng playing techniques based on acoustic feature fusion according to claim 1, characterized in that: The feature vector is associated with the label in the following way: For each labeled sample in the sample library, its comprehensive feature vector is extracted according to the steps of data acquisition and preprocessing, time domain feature extraction, frequency domain feature extraction, time-frequency domain joint feature extraction, and feature fusion. It is denoted as F and a one-to-one correspondence is established with the corresponding skill category label to form a training sample pair (F,V). Where V is the skill category label, V∈{1、2、……NV}, and NV is the total number of skill categories.
7. The intelligent recognition method for guzheng playing techniques based on acoustic feature fusion according to claim 6, characterized in that: The steps for training a K-Nearest Neighbor (KNN) classifier are as follows: Step 1: Determine the value of K: Choose K=5; Step 2: Definition of distance metric: The similarity between feature vectors is calculated using Euclidean distance. For two feature vectors Fa = [fa1, fa2, ..., fa...] D ] and Fb=[fb1, fb2, ..., fb D ]; Where D is the feature dimension; The Euclidean distance d(Fa,Fb) is calculated as follows: First, calculate the square of the difference between the corresponding elements of the two feature vectors, then add all the squared values together, and finally take the square root of the sum. Step 3: Model Training Store all training sample pairs (F,V) into the sample library to complete the training of the KNN classifier.
8. The intelligent recognition method for guzheng playing techniques based on acoustic feature fusion according to claim 7, characterized in that: The nearest neighbor search and voting methods are as follows: For the input guzheng audio data to be identified, the comprehensive feature vector F is extracted following the same process as for the training samples. unknown ; Calculate distance F unknown The Euclidean distance d(F) between the feature vector F of all training samples in the sample library and the feature vector F of all training samples in the sample library. unknown ,F); Select K nearest neighbors from the sample database, with a distance F. unknown Record the skill category labels of the K most recent training samples; The voting decision involves voting on these K labels, and the category with the most votes is F. unknown The corresponding guzheng playing technique category is determined by the following method: among the K nearest neighbor samples, the number of votes received by each category V is counted, and the category V with the most votes is selected as the final predicted category VY.
9. The intelligent recognition method for guzheng playing techniques based on acoustic feature fusion according to claim 8, characterized in that: The accuracy verification method is as follows: 1200 samples of guzheng playing techniques were selected, including 6 different techniques, with 200 samples of each technique, and divided into training set and test set in an 8:2 ratio; After training the KNN classifier using the training set, a recognition test is performed on the test set. The recognition accuracy is obtained by dividing the number of correctly recognized samples in the test set by the total number of samples in the test set.
10. A Guzheng playing technique intelligent recognition system based on acoustic feature fusion, the system being used to execute the Guzheng playing technique intelligent recognition method and system method based on acoustic feature fusion as described in any one of claims 1-9, characterized in that, The system includes: Acquisition and processing unit: Acquires audio data using a microphone, performs moving average filtering for noise reduction, and then divides the data into frames according to a pre-defined frame length and frame shift to obtain multiple frames of audio signal; The feature extraction unit is used for extracting time-domain features, frequency-domain features, and joint time-frequency-domain features. The feature fusion unit is used to fuse the time-domain features corresponding to short-time energy and zero-crossing rate, the frequency-domain features corresponding to the energy proportion of each frequency interval, and the time-frequency features corresponding to the energy proportion of each scale wavelet into a comprehensive vector. The classification and recognition unit is used to build a sample library containing various guzheng playing techniques and manually label it. Then, the feature vector is associated with the label, and a classifier is trained using KNN. The guzheng playing techniques are identified through nearest neighbor search and voting, and the accuracy is verified by samples.