A vocal music teaching feedback optimization method based on intelligent voice
By using speech signal processing and intelligent algorithms, the system precisely locates deviations in pitch, rhythm, and stability in vocal music teaching, enabling adaptive error correction and continuous optimization. This solves the problems of inconsistent feedback and lack of adaptability in existing vocal music teaching methods, thereby improving teaching effectiveness.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-03-09
- Publication Date
- 2026-06-09
AI Technical Summary
In current vocal music teaching, teachers rely on subjective evaluations, which makes it difficult to accurately locate subtle deviations in pitch, rhythm, and vocal stability. The lack of unified quantitative standards leads to inconsistent feedback and makes it difficult to achieve objective and repeatable teaching assessments. Error correction methods lack adaptability and systematization, and their performance improvement is limited after long-term use.
By employing speech signal processing, dynamic time warping algorithm, multi-scale temporal convolutional regression, CUSUM mutation detection, K-means clustering analysis and policy gradient learning techniques, acoustic feature alignment, error segment aggregation, state and action space are constructed to achieve refined error localization, adaptive feedback decision-making and closed-loop optimization.
It enables precise positioning of pitch, rhythm, and stability in vocal music teaching, adaptive error correction selection, and improves the ability to continuously optimize teaching effectiveness, thereby increasing the accuracy of error segment positioning and error correction efficiency.
Smart Images
Figure CN122175078A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of vocal music teaching optimization technology, and in particular to a vocal music teaching feedback optimization method based on intelligent voice. Background Technology
[0002] Current vocal instruction largely relies on teacher demonstrations and subjective listening feedback for correction. The teaching process primarily involves manual assessment of pitch deviations, rhythmic errors, and vocal stability, with verbal prompts or repetitive practice assigned to problematic segments. This method is highly dependent on teacher experience, and feedback is typically limited to the sentence or paragraph level, making it difficult to pinpoint subtle deviations at the frame or short-term segment level. Furthermore, manual assessment lacks standardized quantitative criteria, leading to inconsistent evaluation standards among teachers and resulting in significant fluctuations in training effectiveness, hindering objective and repeatable teaching evaluation.
[0003] With the development of speech signal processing and artificial intelligence technologies, some systems have begun to introduce acoustic feature extraction and template matching methods to compare the similarity between student performances and standard demonstrations, in order to achieve automatic scoring or error alerts. However, these methods mostly use overall similarity or simple distance metrics for evaluation, which can only provide a single score or coarse-grained difference results. They lack the ability to analyze continuous temporal errors in segments, cannot characterize the evolution trend of errors in the time dimension, and are difficult to form a structured state representation that can be used for subsequent decision-making.
[0004] In terms of generating further teaching feedback strategies, existing technologies typically employ error correction methods triggered by fixed rules or preset thresholds. For example, when the score is below the threshold, prompting the student to sing again or playing a demonstration audio. Such rule-based strategies lack the ability to adaptively distinguish between different error types and cannot dynamically adjust intervention methods based on historical training results. At the same time, there may be conflicts or redundancies among various error correction methods, and there is a lack of a systematic action selection and decision-making mechanism, making it difficult to achieve the optimal feedback combination in complex teaching scenarios.
[0005] Furthermore, existing systems generally lack a closed-loop update mechanism. A parameter iteration relationship based on reward signals is not established between teaching results and subsequent strategies. The strategy model cannot be continuously optimized based on actual teaching effectiveness, resulting in limited performance improvement after long-term use and making it difficult to form a continuously adaptive intelligent teaching feedback system.
[0006] Therefore, how to provide a method for optimizing vocal music teaching feedback based on intelligent voice is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention
[0007] One objective of this invention is to propose a vocal music teaching feedback optimization method based on intelligent speech. This invention fully utilizes speech signal processing, dynamic time warping algorithm, multi-scale temporal convolutional regression, CUSUM mutation detection, K-means clustering analysis, and policy gradient-based reinforcement learning decision-making technology. It details the complete algorithm flow from acoustic feature alignment modeling, error segment aggregation, state and action space construction, action value estimation sampling to closed-loop policy update, and has the advantages of refined error localization, adaptive feedback decision-making, constrained conflict control, and sustainable optimization of teaching effect.
[0008] A method for optimizing vocal music teaching feedback based on intelligent voice, according to an embodiment of the present invention, includes the following steps:
[0009] S1. Collect vocal speech data and preprocess it to form an acoustic feature sequence;
[0010] S2. Using the dynamic time warping algorithm, the acoustic feature sequence is matched with the preset standard singing template at the frame level, the frame-by-frame difference vector is calculated and the window is aggregated along the alignment path to construct the error feature matrix.
[0011] S3. Based on the error feature matrix, multi-scale convolution aggregation and cross-frame dependency propagation are performed along the time dimension. The aggregation result is continuously regressed to obtain the quality response value. The regression parameters are iteratively updated based on the response residual to form a quality evaluation function.
[0012] S4. Based on the quality assessment function, the error feature matrix is divided into continuous segments and aggregated by type. The error statistics and scoring trends of each segment are extracted and combined to form a training state representation. At the same time, the preset error correction method is discretized and encoded into a set of actions to construct a teaching decision space.
[0013] S5. Within the teaching feedback decision space, strategy value estimation is used to score the action value of each segment error feature and to sample the action. The sampling results are then filtered by combining preset conflict constraints and threshold constraints to generate optimized vocal teaching results.
[0014] S6. Collect feedback data corresponding to the optimization results of vocal music teaching, and update the policy value estimation parameters using the policy gradient method based on reward signals.
[0015] Optionally, the preprocessing specifically includes:
[0016] The acquired vocal speech signal is pre-emphasized and filtered. The filtered speech signal is then divided into frames and windowed to obtain a short-time speech frame sequence. A short-time Fourier transform is then performed on each short-time speech frame to obtain frame-level amplitude spectrum features.
[0017] Fundamental frequency trajectory features are obtained by performing fundamental frequency tracking on frame-level amplitude spectrum features, and formant features are obtained by performing linear prediction analysis on each short-time speech frame. At the same time, the short-time energy features and zero-crossing rate features of each short-time speech frame are calculated.
[0018] The frame-level amplitude spectrum features, fundamental frequency trajectory features, formant features, short-time energy features, and zero-crossing rate features are aligned by frame index and concatenated into a frame-level multidimensional feature vector. Normalization is performed on each dimension of the features and they are arranged in frame order to obtain the acoustic feature sequence.
[0019] Optionally, S2 specifically includes:
[0020] S21. Establish feature vector sets for the acoustic feature sequence and the preset standard singing template frame by frame, calculate the Euclidean distance between any two frame feature vectors and form an inter-frame distance matrix. The preset standard singing template is composed of frame-level acoustic feature sequences obtained by framing, spectral transformation and feature extraction of standard demonstration singing speech.
[0021] S22. On the inter-frame distance matrix, the cumulative distance matrix is calculated row by row and column by column using the dynamic time warping recursive formula, and the alignment path corresponding to the minimum cumulative distance is obtained by backtracking from the endpoint to the starting point.
[0022] S23. Extract the corresponding frame features according to the alignment path and perform a dimension-by-dimensional subtraction operation to obtain a frame-by-frame difference vector sequence, and perform resampling on repeated mapped frames to form an equally spaced difference sequence.
[0023] S24. On the equally spaced difference sequence, a fixed-length sliding window is used to perform weighted summation and mean calculation, and the error feature matrix is obtained by splicing the frames in order.
[0024] Optionally, S3 specifically includes:
[0025] S31. Set multiple convolution kernels of different lengths along the time axis of the error feature matrix, and slide each convolution kernel frame by frame along the time axis. Perform weighted filtering calculation on the adjacent error features covered by the convolution kernels to obtain the time-series feature response sequences of short-term, medium-term, and long-term scales respectively.
[0026] S32. Align and superimpose temporal feature responses of different scales at the same frame position, and combine them through weighted fusion to form an aggregated feature sequence containing multi-time scale information.
[0027] S33. Establish a cross-frame recursive update structure on the aggregated feature sequence, incorporating the aggregated features of the current frame and the output result of the previous frame into the calculation, and passing them frame by frame in chronological order to obtain continuously changing regression output values, forming a quality response value sequence, specifically including:
[0028] Perform linear mapping calculations on the aggregated features of the first frame to obtain the initial regression output value as the initial state;
[0029] Starting from the second frame, traverse each frame in chronological order, multiply the aggregated features of the current frame and the regression output value of the previous frame by the corresponding weight coefficients, and then perform weighted fusion to obtain the updated output value of the current frame.
[0030] The updated output value of the current frame is used as the historical input for the next frame to continue participating in the recursive calculation, and the continuous update of all frames is completed in sequence to obtain a sequence of quality response values arranged in chronological order.
[0031] S34. Compare the quality response value with the corresponding reference score frame by frame to obtain the response residual. Use the stochastic gradient descent method to update the regression parameters step by step based on the residual. Iterate and correct the regression parameters over time series to obtain the quality assessment function.
[0032] Optionally, S34 specifically includes:
[0033] S341. Initialize the regression parameters and read the feature vectors of each frame in the aggregated feature sequence in chronological order. At the same time, read the reference score corresponding to each frame as the supervision signal to construct a frame-by-frame training sample sequence.
[0034] S342. Perform linear regression calculation on the training samples of the current frame using regression parameters to obtain the predicted score value, and compare the predicted score value with the reference score to calculate the prediction residual of the current frame.
[0035] S343. Construct the parameter gradient direction based on the prediction residual of the current frame and the corresponding feature vector, and calculate the correction step size of the regression parameters in combination with the preset learning rate to obtain the parameter increment of the current frame.
[0036] S344. The parameter increment is added to the regression parameter to complete one parameter update. The prediction score calculation, residual calculation and parameter update operation are repeated on subsequent frames until the entire time series is traversed, completing one round of stochastic gradient descent iteration.
[0037] S345. Repeat the stochastic gradient descent iterations multiple times until the residual change between two adjacent iterations is less than a preset threshold, and obtain the final regression parameters as the parameter set of the quality evaluation function.
[0038] Optionally, S4 specifically includes:
[0039] S41. The quality response value sequence is obtained by calculating the error feature matrix frame by frame using the quality evaluation function, and the change point is determined by the CUSUM mutation detection algorithm based on the quality response value sequence. The error feature matrix is divided into multiple continuous error segments according to the location of the change point.
[0040] S42. Extract the statistical description vector of error features for each continuous error segment, and perform linear regression fitting on the quality response value within the segment to obtain the trend parameter. Combine the statistical description vector and the trend parameter to form the segment feature vector.
[0041] S43. Combine all the feature vectors of the segments into a feature set, use the K-means clustering algorithm to perform iterative clustering calculation, divide the feature vectors into several categories according to the minimum intra-class distance criterion and generate corresponding category labels to realize the type aggregation of error segments;
[0042] S44. Map the category labels and corresponding segment features of each segment to state index vectors in chronological order to form training state representations. Simultaneously, establish an action index mapping table for preset error correction methods and assign a unique integer code to each error correction method. Use all integer codes to construct a discrete action set, thus obtaining the teaching decision space, specifically including:
[0043] Based on the number of categories and the dimension of fragment features, a fixed-length state storage area is divided in the memory space, and the state storage area is further divided into two continuous sub-intervals: a category index segment and a feature segment, which are used to store category index and fragment feature data, respectively.
[0044] Calculate the offset address of the category index in the category index segment according to the category number to which the fragment belongs, and write the category number into the corresponding address position. At the same time, write the fragment feature vector into the preset continuous address range in the feature segment in dimensional order, so that the category index segment and the feature segment form a fixed-length state vector in the same storage area.
[0045] Establish an action lookup table structure for all preset error correction methods, assign a continuously increasing integer number to each error correction method, and record the bidirectional mapping relationship between the number and the error correction method in the lookup table;
[0046] All integer numbers are stored sequentially into a contiguous array to form an action space table, and the corresponding error correction method is directly located by array index in the subsequent decision-making process.
[0047] Optionally, S43 specifically includes:
[0048] S431. The feature vectors of each continuous error segment are combined into a feature matrix according to the segment index, and the dimensions of the feature matrix are standardized to unify the units.
[0049] S432. The K-means initialization method is used to determine the cluster centers. First, an eigenvector is selected from the feature matrix as the initial center. Then, the remaining centers are selected successively according to the probability distribution of the minimum distance to the selected centers, and a unique class number is assigned to each center.
[0050] S433. Perform clustering update according to Lloyd iteration rule. Calculate the distance from each feature vector in the feature matrix to each cluster center and assign it to the category with the smallest distance. Calculate the mean vector of the assigned feature vectors in each category and replace the corresponding cluster center. Repeat the category assignment and center update until the center displacement is less than the preset threshold or the number of iterations reaches the upper limit.
[0051] S434. During the iteration process, detect empty clusters. When there is a category that has not been assigned to any feature vector, select one of the feature vectors that is currently the furthest from its center as the new center of the empty cluster and re-execute the class assignment round.
[0052] S435. Output the category number corresponding to each continuous error segment as the segment type label, and use the final cluster center of each category as the type prototype vector to complete the type aggregation of the error segments.
[0053] Optionally, S5 specifically includes:
[0054] S51. Construct an action value function based on the training state representation and discrete action set, project the execution features of the training state representation to obtain the state feature vector, and jointly encode the state feature vector and action index. Use the action value function to batch calculate the encoding results to obtain the action value score vector corresponding to each action.
[0055] S52. The ε-greedy action sampling algorithm is used to sample the action value rating vector, specifically including:
[0056] A random sample number is generated within the action index interval of the preset exploration probability ε and the discrete action set, and the random sample number is compared with ε to determine the sampling branch;
[0057] When the number of random samples is less than ε, a uniformly distributed random action index is generated within the action index interval as an exploration action;
[0058] When the number of random samples is not less than ε, the maximum value search is performed on the action value score vector to obtain the optimal action index, and when there are ties for the maximum score, the action to be used is randomly selected from the set of ties;
[0059] The exploration and utilization actions are written into the candidate action index sequence and output for subsequent constraint filtering.
[0060] S53. Perform conflict constraint detection and threshold constraint verification on the candidate action index sequence in sequence. The conflict constraint detection includes locating the error correction method based on the action index mapping table and querying the conflict relationship with the selected error correction method in the preset conflict matrix. The threshold constraint verification includes comparing the score of the corresponding action index in the action value scoring vector with the preset scoring threshold and removing the candidate actions that fail the conflict constraint detection or threshold constraint verification.
[0061] S54. Determine the final action index from the remaining candidate actions according to the action value score from high to low, and locate the corresponding error correction method through the action index mapping table to generate the vocal teaching optimization result.
[0062] Optionally, S53 specifically includes:
[0063] S531. Assign a unique bit index to each action according to the number of actions in the discrete action set, and generate a conflict bit mask vector for each action based on the bit index. At the same time, establish an accumulated bit mask register variable to record the action bits that have passed the detection.
[0064] S532. Traverse the candidate action index sequence in order, perform a bitwise AND operation on the conflict bit mask and the cumulative bit mask of the current candidate action, record the candidate action index and merge its bits into the cumulative bit mask when the operation result is zero, and skip the candidate action when the operation result is non-zero.
[0065] S533. Read the action value score of each action index that passes the conflict detection, and compare it with the preset score threshold. Only retain the action indexes with scores not lower than the threshold to form a valid action sequence.
[0066] S534. Sort the effective action sequences in descending order according to their action value scores and determine the action index of the first sorted action.
[0067] Optionally, S6 specifically includes:
[0068] S61. After the vocal music teaching optimization results are executed, the feedback data of the corresponding segment is collected, and a reward signal sequence is constructed based on the feedback data. The reward signal sequence records the score change after each action is executed in chronological order.
[0069] S62. Traverse the reward signal sequence in reverse order of time, perform recursive calculation on the reward signal using the cumulative discount method to obtain the cumulative reward value sequence for each time step, and store the cumulative reward value in alignment with the corresponding training state representation and action index.
[0070] S63. Calculate the log probability gradient of the action corresponding to each time step based on the policy value estimation function, and multiply the log probability gradient with the corresponding cumulative reward value to construct the policy gradient vector;
[0071] S64. Perform incremental updates on the policy value estimation parameters along the policy gradient vector direction according to the preset learning rate to complete the iterative correction of the policy value estimation parameters.
[0072] The beneficial effects of this invention are:
[0073] First, this invention performs frame-level acoustic feature extraction and dynamic time warping alignment processing on vocal speech signals, and combines error feature matrix construction, multi-scale temporal regression modeling and continuous segment division mechanism to achieve fine positioning of pitch, rhythm and stability deviations during singing, transforming the traditional coarse-grained evaluation method that relies on human subjective judgment into a quantitative analysis method based on time series data.
[0074] Secondly, this invention introduces fragment type aggregation, state representation construction, and action value estimation and sampling decision-making mechanisms to evaluate the value of various error correction methods and screen for conflict constraints in the discrete action space, thereby realizing adaptive teaching intervention selection for different error types, compared with fixed rules or single threshold triggering methods.
[0075] Finally, a closed-loop optimization process is constructed using a policy gradient update method based on reward signals. The teaching results are then used to iteratively correct the policy value estimation parameters, enabling the system to continuously adjust its decision-making strategy during the training process and develop long-term adaptive evolution capabilities. Attached Figure Description
[0076] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:
[0077] Figure 1 This is a flowchart of a vocal music teaching feedback optimization method based on intelligent voice proposed in this invention;
[0078] Figure 2 This is a schematic diagram illustrating the multi-scale convolutional regression quality assessment and continuous quality response value calculation of a vocal music teaching feedback optimization method based on intelligent speech proposed in this invention.
[0079] Figure 3 This diagram illustrates the error segmentation, type aggregation, and teaching decision space construction of a vocal music teaching feedback optimization method based on intelligent speech proposed in this invention. Detailed Implementation
[0080] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.
[0081] refer to Figures 1-3 A method for optimizing vocal music teaching feedback based on intelligent voice includes the following steps:
[0082] S1. Collect vocal speech data and preprocess it to form an acoustic feature sequence;
[0083] S2. Using the dynamic time warping algorithm, the acoustic feature sequence is matched with the preset standard singing template at the frame level, the frame-by-frame difference vector is calculated and the window is aggregated along the alignment path to construct the error feature matrix.
[0084] S3. Based on the error feature matrix, multi-scale convolution aggregation and cross-frame dependency propagation are performed along the time dimension. The aggregation result is continuously regressed to obtain the quality response value. The regression parameters are iteratively updated based on the response residual to form a quality evaluation function.
[0085] S4. Based on the quality assessment function, the error feature matrix is divided into continuous segments and aggregated by type. The error statistics and scoring trends of each segment are extracted and combined to form a training state representation. At the same time, the preset error correction method is discretized and encoded into a set of actions to construct a teaching decision space.
[0086] S5. Within the teaching feedback decision space, strategy value estimation is used to score the action value of each segment error feature and to sample the action. The sampling results are then filtered by combining preset conflict constraints and threshold constraints to generate optimized vocal teaching results.
[0087] S6. Collect feedback data corresponding to the optimization results of vocal music teaching, and update the policy value estimation parameters using the policy gradient method based on reward signals.
[0088] In this embodiment, the preprocessing specifically includes:
[0089] The acquired vocal speech signal is pre-emphasized and filtered. The filtered speech signal is then divided into frames and windowed to obtain a short-time speech frame sequence. A short-time Fourier transform is then performed on each short-time speech frame to obtain frame-level amplitude spectrum features.
[0090] Fundamental frequency trajectory features are obtained by performing fundamental frequency tracking on frame-level amplitude spectrum features, and formant features are obtained by performing linear prediction analysis on each short-time speech frame. At the same time, the short-time energy features and zero-crossing rate features of each short-time speech frame are calculated.
[0091] The frame-level amplitude spectrum features, fundamental frequency trajectory features, formant features, short-time energy features, and zero-crossing rate features are aligned by frame index and concatenated into a frame-level multidimensional feature vector. Normalization is performed on each dimension of the features and they are arranged in frame order to obtain the acoustic feature sequence.
[0092] In this embodiment, S2 specifically includes:
[0093] S21. Establish feature vector sets for the acoustic feature sequence and the preset standard singing template frame by frame, calculate the Euclidean distance between any two frame feature vectors and form an inter-frame distance matrix, wherein the preset standard singing template is composed of frame-level acoustic feature sequences obtained by framing, spectral transformation and feature extraction of standard demonstration singing speech.
[0094] S22. On the inter-frame distance matrix, the cumulative distance matrix is calculated row by row and column by column using a dynamic time warping recursive formula, and the alignment path corresponding to the minimum cumulative distance is obtained by backtracking from the endpoint to the starting point. Specifically, this includes:
[0095] The starting element of the cumulative distance matrix is assigned the starting element of the inter-frame distance matrix, and the first row is incremented by column index, and the first column is incremented by row index to complete the boundary initialization.
[0096] Traverse the remaining elements of the cumulative distance matrix in row and column index order, and for each element, take the minimum cumulative distance among its left, top, and top-left adjacent elements and add it to the corresponding inter-frame distance to obtain the cumulative distance of that element.
[0097] Starting from the last element of the cumulative distance matrix, compare the cumulative distances of the adjacent elements on the left, top, and top left in sequence and select the smallest one as the backtracking direction. Backtrack to the starting element and record the index pair sequence to obtain the alignment path.
[0098] S23. Extract the corresponding frame features according to the alignment path and perform a dimension-by-dimensional subtraction operation to obtain a frame-by-frame difference vector sequence, and perform resampling on repeated mapped frames to form an equally spaced difference sequence.
[0099] S24. On the equally spaced difference sequence, a fixed-length sliding window is used to perform weighted summation and mean calculation, and the error feature matrix is obtained by splicing the frames in order.
[0100] In this embodiment, S3 specifically includes:
[0101] S31. Set multiple convolution kernels of different lengths along the time axis of the error feature matrix, and slide each convolution kernel frame by frame along the time axis. Perform weighted filtering calculation on the adjacent error features covered by the convolution kernels to obtain the time-series feature response sequences of short-term, medium-term, and long-term scales respectively.
[0102] S32. Align and superimpose temporal feature responses of different scales at the same frame position, and combine them through weighted fusion to form an aggregated feature sequence containing multi-time scale information.
[0103] S33. Establish a cross-frame recursive update structure on the aggregated feature sequence, incorporating the aggregated features of the current frame and the output result of the previous frame into the calculation, and passing them frame by frame in chronological order to obtain continuously changing regression output values, forming a quality response value sequence, specifically including:
[0104] Perform linear mapping calculations on the aggregated features of the first frame to obtain the initial regression output value as the initial state;
[0105] Starting from the second frame, traverse each frame in chronological order, multiply the aggregated features of the current frame and the regression output value of the previous frame by the corresponding weight coefficients, and then perform weighted fusion to obtain the updated output value of the current frame.
[0106] The updated output value of the current frame is used as the historical input for the next frame to continue participating in the recursive calculation, and the continuous update of all frames is completed in sequence to obtain a sequence of quality response values arranged in chronological order.
[0107] S34. Compare the quality response value with the corresponding reference score frame by frame to obtain the response residual. Use the stochastic gradient descent method to update the regression parameters step by step based on the residual. Iterate and correct the regression parameters over time series to obtain the quality assessment function.
[0108] In this embodiment, S34 specifically includes:
[0109] S341. Initialize the regression parameters and read the feature vectors of each frame in the aggregated feature sequence in chronological order. At the same time, read the reference score corresponding to each frame as the supervision signal to construct a frame-by-frame training sample sequence.
[0110] S342. Perform linear regression calculation on the training samples of the current frame using regression parameters to obtain the predicted score value, and compare the predicted score value with the reference score to calculate the prediction residual of the current frame.
[0111] S343. Construct the parameter gradient direction based on the prediction residual of the current frame and the corresponding feature vector, and calculate the correction step size of the regression parameters in combination with the preset learning rate to obtain the parameter increment of the current frame.
[0112] S344. The parameter increment is added to the regression parameter to complete one parameter update. The prediction score calculation, residual calculation and parameter update operation are repeated on subsequent frames until the entire time series is traversed, completing one round of stochastic gradient descent iteration.
[0113] S345. Repeat the stochastic gradient descent iterations multiple times until the residual change between two adjacent iterations is less than a preset threshold, and obtain the final regression parameters as the parameter set of the quality evaluation function.
[0114] In this embodiment, S4 specifically includes:
[0115] S41. The quality response value sequence is calculated frame-by-frame using the quality assessment function on the error feature matrix. Based on the quality response value sequence, the CUSUM mutation detection algorithm is used to determine the change points. According to the location of the change points, the error feature matrix is divided into multiple continuous error segments, specifically including:
[0116] The mean of the entire quality response value sequence is calculated as a reference value, and the reference value is subtracted from the response value of each frame to obtain the centered response sequence;
[0117] Establish positive cumulative sum variables and negative cumulative sum variables respectively, and perform recursive updates on the centralized response sequence in chronological order. The positive cumulative sum of the current frame is obtained by adding the positive cumulative sum of the previous frame to the centralized response value of the current frame and taking the non-negative value. The negative cumulative sum is obtained by subtracting the negative cumulative sum of the previous frame from the centralized response value of the current frame and taking the non-negative value.
[0118] The positive and negative cumulative sum variables are compared with preset detection thresholds respectively. When either cumulative sum exceeds the detection threshold, the current frame index is recorded as the change point and the corresponding cumulative sum variable is reset to zero.
[0119] The error feature matrix is divided along the time axis based on the index of all recorded change points to obtain multiple continuous error segments;
[0120] S42. Extract the statistical description vector of error features for each continuous error segment, and perform linear regression fitting on the quality response value within the segment to obtain the trend parameter. Combine the statistical description vector and the trend parameter to form the segment feature vector.
[0121] S43. Combine all the feature vectors of the segments into a feature set, use the K-means clustering algorithm to perform iterative clustering calculation, divide the feature vectors into several categories according to the minimum intra-class distance criterion and generate corresponding category labels to realize the type aggregation of error segments;
[0122] S44. Map the category labels and corresponding segment features of each segment to state index vectors in chronological order to form training state representations. Simultaneously, establish an action index mapping table for preset error correction methods and assign a unique integer code to each error correction method. Use all integer codes to construct a discrete action set, thus obtaining the teaching decision space, specifically including:
[0123] Based on the number of categories and the dimension of fragment features, a fixed-length state storage area is divided in the memory space, and the state storage area is further divided into two continuous sub-intervals: a category index segment and a feature segment, which are used to store category index and fragment feature data, respectively.
[0124] Calculate the offset address of the category index in the category index segment according to the category number to which the fragment belongs, and write the category number into the corresponding address position. At the same time, write the fragment feature vector into the preset continuous address range in the feature segment in dimensional order, so that the category index segment and the feature segment form a fixed-length state vector in the same storage area.
[0125] Establish an action lookup table structure for all preset error correction methods, assign a continuously increasing integer number to each error correction method, and record the bidirectional mapping relationship between the number and the error correction method in the lookup table;
[0126] All integer numbers are stored sequentially into a contiguous array to form an action space table, and the corresponding error correction method is directly located by array index in the subsequent decision-making process.
[0127] In this embodiment, S43 specifically includes:
[0128] S431. The feature vectors of each continuous error segment are combined into a feature matrix according to the segment index, and the dimensions of the feature matrix are standardized to unify the units.
[0129] S432. The K-means initialization method is used to determine the cluster centers. First, an eigenvector is selected from the feature matrix as the initial center. Then, the remaining centers are selected successively according to the probability distribution of the minimum distance to the selected centers, and a unique class number is assigned to each center.
[0130] S433. Perform clustering update according to Lloyd iteration rule. Calculate the distance from each feature vector in the feature matrix to each cluster center and assign it to the category with the smallest distance. Calculate the mean vector of the assigned feature vectors in each category and replace the corresponding cluster center. Repeat the category assignment and center update until the center displacement is less than the preset threshold or the number of iterations reaches the upper limit.
[0131] S434. During the iteration process, detect empty clusters. When there is a category that has not been assigned to any feature vector, select one of the feature vectors that is currently the furthest from its center as the new center of the empty cluster and re-execute the class assignment round.
[0132] S435. Output the category number corresponding to each continuous error segment as the segment type label, and use the final cluster center of each category as the type prototype vector to complete the type aggregation of the error segments.
[0133] In this embodiment, S5 specifically includes:
[0134] S51. Construct an action value function based on the training state representation and discrete action set, project the execution features of the training state representation to obtain the state feature vector, and jointly encode the state feature vector and action index. Use the action value function to batch calculate the encoding results to obtain the action value score vector corresponding to each action.
[0135] S52. The ε-greedy action sampling algorithm is used to sample the action value rating vector, specifically including:
[0136] A random sample number is generated within the action index interval of the preset exploration probability ε and the discrete action set, and the random sample number is compared with ε to determine the sampling branch;
[0137] When the number of random samples is less than ε, a uniformly distributed random action index is generated within the action index interval as an exploration action;
[0138] When the number of random samples is not less than ε, the maximum value search is performed on the action value score vector to obtain the optimal action index, and when there are ties for the maximum score, the action to be used is randomly selected from the set of ties;
[0139] The exploration and utilization actions are written into the candidate action index sequence and output for subsequent constraint filtering.
[0140] S53. Perform conflict constraint detection and threshold constraint verification on the candidate action index sequence in sequence. Conflict constraint detection includes locating the error correction method based on the action index mapping table and querying the conflict relationship with the selected error correction method in the preset conflict matrix. Threshold constraint verification includes comparing the score of the corresponding action index in the action value score vector with the preset score threshold and removing the candidate actions that fail the conflict constraint detection or threshold constraint verification.
[0141] S54. Determine the final action index from the remaining candidate actions according to the action value score from high to low, and locate the corresponding error correction method through the action index mapping table to generate the vocal teaching optimization result.
[0142] In this embodiment, S53 specifically includes:
[0143] S531. Assign a unique bit index to each action according to the number of actions in the discrete action set, and generate a collision bit mask vector for each action based on the bit index. At the same time, establish a cumulative bit mask register variable to record the action bits that have passed the detection.
[0144] S532. Traverse the candidate action index sequence in order, perform a bitwise AND operation on the conflict bit mask and the cumulative bit mask of the current candidate action, record the candidate action index and merge its bits into the cumulative bit mask when the operation result is zero, and skip the candidate action when the operation result is non-zero.
[0145] S533. Read the action value score of each action index that passes the conflict detection, and compare it with the preset score threshold. Only retain the action indexes with scores not lower than the threshold to form a valid action sequence.
[0146] S534. Sort the valid action sequences in descending order according to their action value scores and determine the action index of the first-ranked action.
[0147] In this embodiment, S6 specifically includes:
[0148] S61. After the vocal music teaching optimization results are implemented, the feedback data of the corresponding segment is collected, and a reward signal sequence is constructed based on the feedback data. The reward signal sequence records the score changes after each action is performed in chronological order.
[0149] S62. Traverse the reward signal sequence in reverse order of time, perform recursive calculation on the reward signal using the cumulative discount method to obtain the cumulative reward value sequence for each time step, and store the cumulative reward value in alignment with the corresponding training state representation and action index.
[0150] S63. Calculate the log probability gradient of the action corresponding to each time step based on the policy value estimation function, and multiply the log probability gradient with the corresponding cumulative reward value to construct the policy gradient vector.
[0151] S64. Perform incremental updates on the policy value estimation parameters along the policy gradient vector direction according to the preset learning rate to complete the iterative correction of the policy value estimation parameters.
[0152] Example 1: To verify the feasibility of this invention in practice, it was applied to a vocal music teaching and training environment. In this environment, students conduct daily vocal exercises and phrase-by-phrase singing training through terminal devices. The system collects students' singing voice data in real time and automatically generates teaching feedback. In traditional teaching methods, teachers mainly rely on auditory discrimination to make subjective evaluations of students' singing, often only pointing out general problems such as "unstable pitch" or "slow rhythm," making it difficult to pinpoint specific time segments. Students' repetitive practice is inefficient, and the error correction methods for different students are relatively fixed, lacking targeted adjustments, resulting in limited improvement after long-term training. This example introduces the vocal music teaching feedback optimization method based on intelligent voice proposed in this invention.
[0153] In practical applications, the system first segments the student's singing voice into frames and extracts acoustic feature sequences. Then, it uses a dynamic time warping algorithm to align the frame-by-frame path with a standard demonstration singing template, constructing an error feature matrix to accurately obtain the pitch deviation and energy fluctuation differences at each moment. Subsequently, a multi-scale convolutional regression structure is used to continuously assess the quality of the error features, obtaining quality response values that change over time. The CUSUM mutation detection algorithm is then used to automatically locate error change points, dividing the entire singing segment into multiple continuous error segments. After extracting statistical descriptive vectors and trend parameters for each segment, K-means clustering is used to form error types, enabling automatic classification of different problems such as "pitch deviation," "rhythmic deviation," and "insufficient breath stability."
[0154] Next, based on the constructed state space and action space, the system scores various error correction methods on the action value table and selects the optimal feedback combination through ε-greedy sampling and bitmask conflict detection, such as selecting one or more methods like "local slow singing along," "metronome assistance," and "vocal demonstration playback." After training, the system generates reward signals based on the trainee's performance in the next round and corrects the policy parameters through a policy gradient update mechanism.
[0155] Significant improvements were observed in multiple rounds of training comparison. Using traditional fixed-rule feedback, the average pitch deviation after a single practice session was approximately 27 cents, the average rhythm error was approximately 94 ms, the accuracy rate for locating error segments was less than 65%, and the average number of correction rounds for completing a piece of similar difficulty was approximately 8. After adopting the method of this invention, the pitch deviation decreased to approximately 11 cents, the rhythm error decreased to within 41 ms, the accuracy rate for locating error segments increased to over 92%, the average number of correction rounds decreased to approximately 3, and the feedback response time was shortened to approximately 45% of the original.
[0156] Table 1. Statistical Comparison of Vocal Music Teaching Feedback Optimization Effects
[0157]
[0158] As can be seen from the data in Table 1, the above implementation process verifies that the present invention can automatically identify and classify fine-grained errors in vocal performance, and achieve targeted feedback selection by combining strategy value estimation. At the same time, it continuously improves the decision-making effect by relying on the reward-driven parameter update mechanism, and shows high accuracy, stability and training efficiency in actual teaching scenarios.
[0159] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.
Claims
1. A method for optimizing vocal music teaching feedback based on intelligent voice, characterized in that, Includes the following steps: S1. Collect vocal speech data and preprocess it to form an acoustic feature sequence; S2. Using the dynamic time warping algorithm, the acoustic feature sequence is matched with the preset standard singing template at the frame level, the frame-by-frame difference vector is calculated and the window is aggregated along the alignment path to construct the error feature matrix. S3. Based on the error feature matrix, multi-scale convolution aggregation and cross-frame dependency propagation are performed along the time dimension. The aggregation result is continuously regressed to obtain the quality response value. The regression parameters are iteratively updated based on the response residual to form a quality evaluation function. S4. Based on the quality assessment function, the error feature matrix is divided into continuous segments and aggregated by type. The error statistics and scoring trends of each segment are extracted and combined to form a training state representation. At the same time, the preset error correction method is discretized and encoded into a set of actions to construct a teaching decision space. S5. Within the teaching feedback decision space, strategy value estimation is used to score the action value of each segment error feature and to sample the action. The sampling results are then filtered by combining preset conflict constraints and threshold constraints to generate optimized vocal teaching results. S6. Collect feedback data corresponding to the optimization results of vocal music teaching, and update the policy value estimation parameters using the policy gradient method based on reward signals.
2. The method for optimizing vocal music teaching feedback based on intelligent voice as described in claim 1, characterized in that, The preprocessing specifically includes: The acquired vocal speech signal is pre-emphasized and filtered. The filtered speech signal is then divided into frames and windowed to obtain a short-time speech frame sequence. A short-time Fourier transform is then performed on each short-time speech frame to obtain frame-level amplitude spectrum features. Fundamental frequency trajectory features are obtained by performing fundamental frequency tracking on frame-level amplitude spectrum features, and formant features are obtained by performing linear prediction analysis on each short-time speech frame. At the same time, the short-time energy features and zero-crossing rate features of each short-time speech frame are calculated. The frame-level amplitude spectrum features, fundamental frequency trajectory features, formant features, short-time energy features, and zero-crossing rate features are aligned by frame index and concatenated into a frame-level multidimensional feature vector. Normalization is performed on each dimension of the features and they are arranged in frame order to obtain the acoustic feature sequence.
3. The method for optimizing vocal music teaching feedback based on intelligent voice as described in claim 1, characterized in that, S2 specifically includes: S21. Establish feature vector sets for the acoustic feature sequence and the preset standard singing template frame by frame, calculate the Euclidean distance between any two frame feature vectors and form an inter-frame distance matrix. The preset standard singing template is composed of frame-level acoustic feature sequences obtained by framing, spectral transformation and feature extraction of standard demonstration singing speech. S22. On the inter-frame distance matrix, the cumulative distance matrix is calculated row by row and column by column using the dynamic time warping recursive formula, and the alignment path corresponding to the minimum cumulative distance is obtained by backtracking from the endpoint to the starting point. S23. Extract the corresponding frame features according to the alignment path and perform a dimension-by-dimensional subtraction operation to obtain a frame-by-frame difference vector sequence, and perform resampling on repeated mapped frames to form an equally spaced difference sequence; S24. On the equally spaced difference sequence, a fixed-length sliding window is used to perform weighted summation and mean calculation, and the error feature matrix is obtained by splicing the frames in order.
4. The method for optimizing vocal music teaching feedback based on intelligent voice as described in claim 1, characterized in that, S3 specifically includes: S31. Set multiple convolution kernels of different lengths along the time axis of the error feature matrix, and slide each convolution kernel frame by frame along the time axis. Perform weighted filtering calculation on the adjacent error features covered by the convolution kernels to obtain the time-series feature response sequences of short-term, medium-term, and long-term scales respectively. S32. Align and superimpose temporal feature responses of different scales at the same frame position, and combine them through weighted fusion to form an aggregated feature sequence containing multi-time scale information. S33. Establish a cross-frame recursive update structure on the aggregated feature sequence, incorporating the aggregated features of the current frame and the output result of the previous frame into the calculation, and passing them frame by frame in chronological order to obtain continuously changing regression output values, forming a quality response value sequence, specifically including: Perform linear mapping calculations on the aggregated features of the first frame to obtain the initial regression output value as the initial state; Starting from the second frame, traverse each frame in chronological order, multiply the aggregated features of the current frame and the regression output value of the previous frame by the corresponding weight coefficients, and then perform weighted fusion to obtain the updated output value of the current frame. The updated output value of the current frame is used as the historical input for the next frame to continue participating in the recursive calculation, and the continuous update of all frames is completed in sequence to obtain a sequence of quality response values arranged in chronological order. S34. Compare the quality response value with the corresponding reference score frame by frame to obtain the response residual. Use the stochastic gradient descent method to update the regression parameters step by step based on the residual. Iterate and correct the regression parameters over time series to obtain the quality assessment function.
5. The method for optimizing vocal music teaching feedback based on intelligent voice according to claim 4, characterized in that, S34 specifically includes: S341. Initialize the regression parameters and read the feature vectors of each frame in the aggregated feature sequence in chronological order. At the same time, read the reference score corresponding to each frame as the supervision signal to construct a frame-by-frame training sample sequence. S342. Perform linear regression calculation on the training samples of the current frame using regression parameters to obtain the predicted score value, and compare the predicted score value with the reference score to calculate the prediction residual of the current frame. S343. Construct the parameter gradient direction based on the prediction residual of the current frame and the corresponding feature vector, and calculate the correction step size of the regression parameters in combination with the preset learning rate to obtain the parameter increment of the current frame. S344. The parameter increment is added to the regression parameter to complete one parameter update. The prediction score calculation, residual calculation and parameter update operation are repeated on subsequent frames until the entire time series is traversed, completing one round of stochastic gradient descent iteration. S345. Repeat the stochastic gradient descent iterations multiple times until the residual change between two adjacent iterations is less than a preset threshold, and obtain the final regression parameters as the parameter set of the quality evaluation function.
6. The method for optimizing vocal music teaching feedback based on intelligent voice as described in claim 1, characterized in that, S4 specifically includes: S41. The quality response value sequence is obtained by calculating the error feature matrix frame by frame using the quality evaluation function, and the change point is determined by the CUSUM mutation detection algorithm based on the quality response value sequence. The error feature matrix is divided into multiple continuous error segments according to the location of the change point. S42. Extract the statistical description vector of error features for each continuous error segment, and perform linear regression fitting on the quality response value within the segment to obtain the trend parameter. Combine the statistical description vector and the trend parameter to form the segment feature vector. S43. Combine all the feature vectors of the segments into a feature set, use the K-means clustering algorithm to perform iterative clustering calculation, divide the feature vectors into several categories according to the minimum intra-class distance criterion and generate corresponding category labels to realize the type aggregation of error segments; S44. Map the category labels and corresponding segment features of each segment to state index vectors in chronological order to form training state representations. Simultaneously, establish an action index mapping table for preset error correction methods and assign a unique integer code to each error correction method. Use all integer codes to construct a discrete action set, thus obtaining the teaching decision space, specifically including: Based on the number of categories and the dimension of fragment features, a fixed-length state storage area is divided in the memory space, and the state storage area is further divided into two continuous sub-intervals: a category index segment and a feature segment, which are used to store category index and fragment feature data, respectively. Calculate the offset address of the category index in the category index segment according to the category number to which the fragment belongs, and write the category number into the corresponding address position. At the same time, write the fragment feature vector into the preset continuous address range in the feature segment in dimensional order, so that the category index segment and the feature segment form a fixed-length state vector in the same storage area. Establish an action lookup table structure for all preset error correction methods, assign a continuously increasing integer number to each error correction method, and record the bidirectional mapping relationship between the number and the error correction method in the lookup table; All integer numbers are stored sequentially into a contiguous array to form an action space table, and the corresponding error correction method is directly located by array index in the subsequent decision-making process.
7. The method for optimizing vocal music teaching feedback based on intelligent voice according to claim 6, characterized in that, Specifically, S43 includes: S431. The feature vectors of each continuous error segment are combined into a feature matrix according to the segment index, and the dimensions of the feature matrix are standardized to unify the units. S432. The K-means initialization method is used to determine the cluster centers. First, an eigenvector is selected from the feature matrix as the initial center. Then, the remaining centers are selected successively according to the probability distribution of the minimum distance to the selected centers, and a unique class number is assigned to each center. S433. Perform clustering update according to Lloyd iteration rule. Calculate the distance from each feature vector in the feature matrix to each cluster center and assign it to the category with the smallest distance. Calculate the mean vector of the assigned feature vectors in each category and replace the corresponding cluster center. Repeat the category assignment and center update until the center displacement is less than the preset threshold or the number of iterations reaches the upper limit. S434. During the iteration process, detect empty clusters. When there is a category that has not been assigned to any feature vector, select one of the feature vectors that is currently the furthest from its center as the new center of the empty cluster and re-execute the class assignment round. S435. Output the category number corresponding to each continuous error segment as the segment type label, and use the final cluster center of each category as the type prototype vector to complete the type aggregation of the error segments.
8. The method for optimizing vocal music teaching feedback based on intelligent voice according to claim 1, characterized in that, S5 specifically includes: S51. Construct an action value function based on the training state representation and discrete action set, project the execution features of the training state representation to obtain the state feature vector, and jointly encode the state feature vector and action index. Use the action value function to batch calculate the encoding results to obtain the action value score vector corresponding to each action. S52. The ε-greedy action sampling algorithm is used to sample the action value rating vector, specifically including: A random sample number is generated within the action index interval of the preset exploration probability ε and the discrete action set, and the random sample number is compared with ε to determine the sampling branch; When the number of random samples is less than ε, a uniformly distributed random action index is generated within the action index interval as an exploration action; When the number of random samples is not less than ε, the maximum value search is performed on the action value score vector to obtain the optimal action index, and when there are ties for the maximum score, the action to be used is randomly selected from the set of ties; The exploration and utilization actions are written into the candidate action index sequence and output for subsequent constraint filtering. S53. Perform conflict constraint detection and threshold constraint verification on the candidate action index sequence in sequence. The conflict constraint detection includes locating the error correction method based on the action index mapping table and querying the conflict relationship with the selected error correction method in the preset conflict matrix. The threshold constraint verification includes comparing the score of the corresponding action index in the action value scoring vector with the preset scoring threshold and removing the candidate actions that fail the conflict constraint detection or threshold constraint verification. S54. Determine the final action index from the remaining candidate actions according to the action value score from high to low, and locate the corresponding error correction method through the action index mapping table to generate the vocal teaching optimization result.
9. The method for optimizing vocal music teaching feedback based on intelligent voice according to claim 8, characterized in that, Specifically, S53 includes: S531. Assign a unique bit index to each action according to the number of actions in the discrete action set, and generate a conflict bit mask vector for each action based on the bit index. At the same time, establish an accumulated bit mask register variable to record the action bits that have passed the detection. S532. Traverse the candidate action index sequence in order, perform a bitwise AND operation on the conflict bit mask and the cumulative bit mask of the current candidate action, record the candidate action index and merge its bits into the cumulative bit mask when the operation result is zero, and skip the candidate action when the operation result is non-zero. S533. Read the action value score of each action index that passes the conflict detection, and compare it with the preset score threshold. Only retain the action indexes with scores not lower than the threshold to form a valid action sequence. S534. Sort the effective action sequences in descending order according to their action value scores and determine the action index of the first sorted action.
10. The method for optimizing vocal music teaching feedback based on intelligent voice according to claim 1, characterized in that, S6 specifically includes: S61. After the vocal music teaching optimization results are executed, the feedback data of the corresponding segment is collected, and a reward signal sequence is constructed based on the feedback data. The reward signal sequence records the score change after each action is executed in chronological order. S62. Traverse the reward signal sequence in reverse order of time, perform recursive calculation on the reward signal using the cumulative discount method to obtain the cumulative reward value sequence for each time step, and store the cumulative reward value in alignment with the corresponding training state representation and action index. S63. Calculate the log probability gradient of the action corresponding to each time step based on the policy value estimation function, and multiply the log probability gradient with the corresponding cumulative reward value to construct the policy gradient vector; S64. Perform incremental updates on the policy value estimation parameters along the policy gradient vector direction according to the preset learning rate to complete the iterative correction of the policy value estimation parameters.