Vocal singing practice quality grade evaluation method based on fractional order spectrogram deep learning

Abstract

The present application proposes a vocal singing practice quality grade evaluation method of fractional order spectrogram deep learning, including the following steps; Step S1, collect the audio signal of the singer's vocal vowel singing practice, and according to the singing index, label the corresponding quality grade, construct the sample data set, and use it for model training, testing and verification; Step S2, convert the audio signal into a series of fractional order spectrograms; Step S3, construct the fractional order spectrogram deep feature extraction network based on DenseNet and channel attention mechanism in the model, input the extracted fractional order spectrogram deep features into the BiLSTM network, and extract the time sequence features of the singing practice signal; Step S4, the quantum fireworks algorithm is used to optimize the hyperparameters of the kernel extreme learning machine of the model, and the extracted time sequence features are mapped to a high-dimensional space for quality grade decision, forming the evaluation result; Step S5, train the model; The present application can better adapt to the characteristics of non-stationary signals and provide more accurate spectrum analysis.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of audio processing, in particular to a vocal singing practice quality grade evaluation method based on fractional order spectrogram deep learning. BACKGROUND

[0002] Vocal singing practice is the basis of singers' singing skills and voice performance, and it has important background and significance in the music and entertainment industries. First, vocal singing quality evaluation is one of the important means of selecting and cultivating outstanding singers in the music industry. Through evaluation, singers with excellent musical talent and artistic performance can be selected to inject more high-quality music works and performances into the music market. Second, evaluating the quality of vocal singing practice helps to improve the professional level of singers. During the evaluation process, professional judges give comments and suggestions on singers' singing skills, pitch, tone, breath control, etc., helping singers find their shortcomings and provide improvement direction, which helps singers continuously improve their singing ability and shape a unique musical style.

[0003] Currently, there are many methods to evaluate the quality of vocal singing practice. The traditional method is subjective evaluation, usually conducted by professional musicians, vocal teachers or experienced listeners, who evaluate the singing quality by listening to the singers' singing skills, pitch, tone, expression and emotion. The second is the objective evaluation method, which is also the traditional speech evaluation method. Some scientific tools and technologies are used to quantify the quality of vocal singing practice. The features of the speech are extracted, such as short-time energy, short-time zero-crossing rate, spectrogram, MFCC, etc., to evaluate the singing quality. With the development of artificial intelligence, deep learning methods are introduced into the processing of speech signals. Common methods include convolutional neural networks (CNN), recurrent neural networks (RNN), long short-term memory networks (LSTM), and gated recurrent units (GRU). These methods have the advantages of automatic feature extraction, strong modeling ability, and adaptability to different time scales, which have brought significant performance improvement to speech recognition, speech synthesis, and speech emotion analysis tasks. However, these singing quality evaluation methods have certain limitations:

[0004] (1) Subjective evaluation method is a very subjective task in evaluating singing quality. Everyone's perception and preference of sound may be different;

[0005] (2) The feature extraction algorithm of traditional speech feature extraction recognition method is usually based on artificial design, which needs to rely on expert knowledge and experience, and may not be able to fully capture the complex features in the speech signal;

[0006] (3) In addition, the feature extraction and classification algorithm in the traditional method often cannot effectively model the long-term dependence in the speech signal, which limits the performance when performing the speech recognition task;

[0007] (4) Moreover, the traditional method is usually based on fixed time window feature extraction, which may not be well adapted to different time scale speech signals, resulting in the loss of some key information when processing long-time speech signals or short-time speech signals.

[0008] (5) The traditional method is susceptible to environmental noise and speaker variation, resulting in decreased recognition performance;

[0009] (6) Although the deep learning method solves the drawbacks of the traditional method, it usually needs a large amount of manually labeled data for model training and feature optimization. This limits the application of the traditional method in the case of data scarcity, and increases the time and cost of establishing a high-performance system. SUMMARY

[0010] The present application proposes a fractional order spectrogram deep learning vocal singing practice quality rating method, which can better adapt to the characteristics of non-stationary signals and provide more accurate spectral analysis.

[0011] The present application adopts the following technical solutions.

[0012] The fractional order spectrogram deep learning vocal singing practice quality rating method comprises the following steps:

[0013] Step S1, collect the audio signal of the singer's vocal singing practice, and according to the sound emission index, label the corresponding quality level, construct a sample data set, and use it for training, testing and verifying the vocal singing practice quality evaluation network model;

[0014] Step S2, the vocal singing practice quality evaluation network model converts the audio signal into a series of fractional order spectrograms based on short-time fractional order Fourier transform;

[0015] Step S3, construct a fractional order spectrogram deep feature extraction network based on DenseNet and channel attention mechanism in the vocal singing practice quality evaluation network model, input the extracted fractional order spectrogram deep features into the BiLSTM network, and extract the time sequence features of the sound emission practice signal;

[0016] Step S4, use the quantum fireworks algorithm to optimize the hyperparameters of the kernel extreme learning machine of the vocal singing practice quality evaluation network model, map the extracted time sequence features to a high-dimensional space for quality level decision, and form the evaluation result;

[0017] Step S5, the vocal singing voice practice quality evaluation network model is trained by using the training set and the validation set, the model is tested by the test set and the result is analyzed, and finally the voice quality evaluation result is obtained.

[0018] In step S1, singers of different ages and different genders are selected as collection objects, and the number of people is not less than 300; in order to ensure the robustness of the model, the same moving coil microphone is used to collect audio of vowel sound in early, middle and late three time periods for different collection objects, the collected audio is saved in wav format, the number of audio collected for each vowel is not less than 6000, and the audio collected for a, o, e, i and u five vowels is evaluated by authoritative evaluators according to the pitch, pitch, tone, rhythm index of the sound, and then the audio data is labeled according to the quality level. Finally, the data set is constructed, and the data is divided into training set, validation set and test set according to the ratio of 8:1:1.

[0019] The specific method of step S2 is: first, the vowel sound signal is preprocessed, the preprocessing method includes denoising processing and pre-emphasis processing to reduce noise and interference in the signal; second, the preprocessed vowel sound signal is analyzed in short time, the frame length and frame shift are set, the frame length is the window function length, the whole vowel sound signal is divided into frames according to the frame length and frame shift, each frame signal is multiplied by the window function, the signal is divided into a series of overlapping time windows, and the signal in each time window is subjected to fractional Fourier transform FrFT; finally, the energy spectrum generated by each time window is spliced in time sequence to obtain the fractional order spectrogram of the whole time sequence.

[0020] In step S3, the vocal singing voice practice quality evaluation network model extracts the features of the fractional order spectrogram from 0.1 to 1.0 by DenseNet network, and weights the extracted features by channel attention mechanism; then, the weighted features are input into the Bi-LSTM network for learning and training to extract the time sequence features of the vowel sound signal.

[0021] When extracting features, DenseNet is used to extract the initial features of the fractional order spectrogram, and then the channel attention mechanism is introduced based on DenseNet to further filter and enhance the extracted features, so as to effectively utilize the information in the input data and extract more representative and discriminative deep features.

[0022] The DenseNet network is a deep convolutional neural network with dense connections, in which the output of each layer is directly connected to the input of all subsequent layers, so that each layer can utilize the feature information of the fractional order spectrogram processed by all previous layers to solve the problems of gradient disappearance and model degradation, and improve the performance and generalization ability of the network.

[0023] The channel attention mechanism evaluates the importance of each channel of the fractional order spectrogram feature map and assigns different weights to each channel accordingly, so that the DenseNet network focuses on important channel features and ignores unimportant features, thereby improving the utilization rate of feature information and the recognition accuracy of the network.

[0024] The fractional order spectrogram features extracted by the DenseNet network combined with the channel attention mechanism are used to train the BiLSTM network, and the weights and bias terms of the vocal singing practice quality assessment network model are updated through the backpropagation algorithm and gradient descent optimizer to minimize the error between the predicted value and the actual value.

[0025] After the vocal singing practice quality assessment network model is trained, the model is used to extract time series features from new vowel singing signals, and these feature vectors are used for subsequent singing quality level evaluation.

[0026] In step S4, the kernel extreme learning machine uses an adaptive KELM, which maps the input data to a high-dimensional feature space by selecting an appropriate kernel function and determining the kernel parameters, thereby enhancing the separability of the data.

[0027] The quantum firework algorithm QFWA used in step S4 is a quantum version of the firework algorithm FWA, which represents fireworks and sparks in the solution space using quantum bits and updates the positions of fireworks and sparks using quantum rotation gates and other operations to find the optimal or approximate optimal solution to the problem. The specific method is as follows:

[0028] First, quantum encoding of the firework algorithm is performed, and the position of the firework is quantum encoded. The quantum encoding of the firework individual P j is as follows:

[0029]

[0030] In the formula, j is the firework individual number, n is the quantum encoding length, and θ jn is the quantum angle of each bit of quantum encoding. Second, the position of the firework is updated using a quantum rotation gate. The state |ψ> of each quantum bit is represented as a probability amplitude form The position of the firework individual is updated using a quantum rotation gate, and the update formula is as follows:

[0031]

[0032] In the formula, And is the probability amplitude of the i-th firework in the d-dimensional space in the j-th iteration, and Δθ is the update angle of the quantum rotation gate, And is the updated probability amplitude;

[0033] In this step, in order to prevent the influence of too small Δθ on the convergence efficiency of the algorithm or the problem of easy falling into local optimal solution caused by too large Δθ, a nonlinear adaptive rotation angle gradually changing with the number of iterations is used to improve the performance of the quantum firework algorithm, which is specifically shown in the following formula,

[0034]

[0035] In the formula, Wherein is the probability amplitude corresponding to a certain quantum bit of the current optimal firework, and is the probability amplitude corresponding to a certain quantum bit of the current firework, θ best And θ i are the angles of a certain quantum bit of the current optimal firework individual and the current firework individual on the unit circle respectively; θ0 is a dynamic rotation angle, and λ is a nonlinear modulation index for dynamic adjustment of θ0, which is in the range of 1.0-2.0 to ensure that |θ best -θ i | is larger, the adjustment amplitude is larger; K is the maximum number of iterations, and Δθ0 is a basic rotation angle which gradually decreases with the increase of the number of iterations

[0036] Finally, the firework individual mutation operation is performed, and after the position update of the quantum rotation gate, the quantum individual is subjected to quantum non-gate mutation operation with a certain probability, that is, the probability amplitude α i and β i of each dimension of the quantum individual are exchanged:

[0037]

[0038] In the formula, is the mutated quantum probability amplitude, is the probability amplitude before mutation.

[0039] Step S4 applies the quantum firework algorithm to optimize the hyperparameters of KELM to form a hybrid optimization and learning framework. Adaptive KELM adjusts the kernel parameters to adapt to different data distributions and characteristics, thereby improving the generalization ability and prediction accuracy of the model. The quantum firework algorithm is used to find the optimal regularization coefficient C and kernel function parameter S in the solution space, and adaptive KELM is used to construct and optimize the model based on these parameters. Specifically, the following steps are included:

[0040] Step 1, initialization: a set of initial fireworks is randomly generated in the solution space, which is the candidate kernel parameter S and the regularization coefficient C;

[0041] Step 2, fitness evaluation: use adaptive KELM to build a model based on the current fireworks parameters, and evaluate the performance of the model on the training data as the fitness of the fireworks; the evaluation includes the evaluation of accuracy and loss function value;

[0042] Step 3, quantum fireworks algorithm optimization: use quantum fireworks algorithm to evolve fireworks to produce new fireworks, that is, new candidate parameters; in this process, the number and explosion amplitude of fireworks can be adjusted according to the fitness value of fireworks to achieve more efficient search.

[0043] Step 4, update: select a part of excellent fireworks as the next generation of fireworks according to the fitness value of the fireworks, and update the solution space;

[0044] Step 5, iteration and termination: repeat steps 2-4 until the termination condition is met, that is, the maximum number of iterations is reached or the preset accuracy requirement is met;

[0045] Step 6, model construction: use the final fireworks parameters including kernel parameter S and regularization coefficient C to construct adaptive KELM model, and use it for subsequent prediction or classification tasks.

[0046] The evaluation method is used for real-time quantitative evaluation of the quality of vocal singing vowel sound production practice of singers, in step S5, the test set is used to test and analyze the vocal sound production practice quality evaluation network model, and finally the sound production quality evaluation result is obtained, and the result level of the sound production quality evaluation result is divided into: excellent, good, medium, pass and poor.

[0047] The present application proposes fractional order spectrogram deep learning instead of traditional speech feature extraction and recognition, the fractional order spectrogram can better adapt to the characteristics of non-stationary signals by introducing the concept of fractional order, and provide more accurate spectrum analysis; the channel attention mechanism can focus on important features or regions in the input data, so that the model can better understand and classify data, by dynamically adjusting the weights between different channels, the model can pay more attention to the information related to the current task, thereby improving the performance of the model; BI-LSTM has the advantages of bidirectional processing capability, long-term dependence capture capability, gradient vanishing / explosion problem alleviation, flexibility and high performance in time series feature deep learning, so that BI-LSTM performs well in time series speech recognition task; adaptive KELM can automatically adjust the parameters and structure of the model according to the distribution and characteristics of the data by introducing the adaptive mechanism, to adapt to the task of fractional order spectrogram deep learning.

[0048] The method can quantitatively evaluate the vocalization practice quality of a singer in real time, discards subjective feelings, more objectively evaluates the vocalization quality, and improves the efficiency and reliability of the vocalization quality evaluation, and can be applied to the auxiliary teaching of vocalization practice of vocalization, and improves the quality and efficiency of singing teaching.

[0049] In view of the problems that the traditional speech feature extraction cannot fully capture the complex features in the speech signal, cannot effectively model the long-term dependence in the speech signal, may not have good adaptability to the speech signal of different time scales, has poor noise robustness, and speech deep learning data is scarce, the present application proposes a fractional order spectrogram deep learning to replace the traditional speech feature extraction and recognition.

[0050] The channel attention mechanism of the present application can focus on important features or regions in the input data, enabling the model to better understand and classify data. By dynamically adjusting the weights between different channels, the model can pay more attention to information related to the current task, thereby improving the performance of the model.

[0051] The BI-LSTM used in the present application has the advantages of bidirectional processing capability, long-term dependence capture capability, gradient vanishing / explosion alleviation, flexibility and high performance in time series feature deep learning, which makes BI-LSTM perform well in various time series prediction, natural language processing and speech recognition tasks. KELM combines ELM (Extreme Learning Machine) and kernel trick, approximates the target function through linear combination of randomly generated hidden layer nodes, and obtains output weights by solving the least squares problem. This structure makes KELM have better generalization performance when processing nonlinear data, and can more accurately approximate complex function relationships.

[0052] Compared with traditional supervised learning methods, the KELM used in the present application adopts a random initialization method, which does not need to iteratively adjust the parameters of the hidden layer nodes, thereby greatly reducing the training time. The adaptive KELM can automatically adjust the parameters and structure of the model according to the distribution and characteristics of the data to adapt to different learning tasks through the introduction of an adaptive mechanism. This adaptive ability makes the model more flexible and effective when processing complex and variable data. BRIEF DESCRIPTION OF DRAWINGS

[0053] The present application will be further described in detail below in combination with the drawings and specific embodiments:

[0054] ATTACHMENT Figure 1is a flowchart of generating a fractional order spectrogram according to the present application;

[0055] attached Figure 2 is a schematic diagram of a vocal singing voice practice quality evaluation network model according to the present application;

[0056] attached Figure 3 is a schematic diagram of a channel attention mechanism structure according to the present application;

[0057] attached Figure 4 is a schematic diagram of an LSTM structure according to the present application;

[0058] attached Figure 5 is a schematic diagram of a KELM structure according to the present application;

[0059] attached Figure 6 is a schematic diagram of a step flowchart according to the present application;

[0060] attached Figure 7 is the waveform of the audio signal 1 of the "a" vowel sound and the corresponding fractional order spectrogram in the embodiment;

[0061] attached Figure 8 is the waveform of the audio signal 2 of the "a" vowel sound and the corresponding fractional order spectrogram in the embodiment. DETAILED DESCRIPTION

[0062] As shown in the figure, the vocal singing voice practice quality grade evaluation method of fractional order spectrogram deep learning includes the following steps:

[0063] Step S1, collect the audio signal of the vocal singing vowel sound practice of the singer, and according to the sound emission index, label the corresponding quality grade, construct a sample data set, and use it for training, testing and verifying the vocal singing voice practice quality evaluation network model;

[0064] Step S2, the vocal singing voice practice quality evaluation network model converts the audio signal into a series of fractional order spectrograms based on short-time fractional order Fourier transform;

[0065] Step S3, construct the fractional order spectrogram deep feature extraction network based on DenseNet and channel attention mechanism in the vocal singing voice practice quality evaluation network model, input the extracted fractional order spectrogram deep features into the BiLSTM network, and extract the time sequence features of the sound emission practice signal;

[0066] Step S4, use the quantum fireworks algorithm to optimize the hyperparameters of the kernel extreme learning machine of the vocal singing voice practice quality evaluation network model, map the extracted time sequence features to a high-dimensional space for quality grade decision, and form the evaluation result;

[0067] Step S5, the training set and the validation set are used to train the vocal singing voice practice quality evaluation network model, the model is tested through the test set and the result is analyzed, and finally the voice quality evaluation result is obtained.

[0068] In step S1, singers of different ages and different genders are selected as collection objects, and the number of people is not less than 300; in order to ensure the robustness of the model, the same moving coil microphone is used to collect audio of vowel sound in early, middle and late three time periods respectively, the collected audio is saved in wav format, the number of audio collected for each vowel is not less than 6000, and the audio collected for a, o, e, i and u five vowels is evaluated by authoritative evaluators according to the pitch, pitch, tone, rhythm index of the sound, and then the audio data is labeled according to the quality level. Finally, the data set is constructed, and the data is divided into training set, validation set and test set according to the ratio of 8:1:1.

[0069] As shown in Figure 1 , the specific method of step S2 is: first, the vowel sound signal is preprocessed, the preprocessing method includes denoising processing and pre-emphasis processing to reduce noise and interference in the signal; second, the preprocessed vowel sound signal is analyzed in short time, the frame length and frame shift are set, the frame length is the window function length, the whole vowel sound signal is divided into frames according to the frame length and frame shift, each frame signal is multiplied by the window function, the signal is divided into a series of overlapping time windows, and the signal in each time window is subjected to fractional Fourier transform FrFT; finally, the energy spectrum generated by each time window is spliced in time sequence to obtain the fractional order spectrogram of the whole time sequence.

[0070] The overall network structure of the vocal singing voice practice quality evaluation is as shown in Figure 2 : first, the input vowel sound signal is processed, in step S3, the vocal singing voice practice quality evaluation network model extracts the features of 0.1 order-1.0 order fractional order spectrogram through DenseNet network respectively, and the extracted features are weighted through channel attention mechanism; then, the weighted features are input into the Bi-LSTM network for learning and training to extract the time sequence features of the vowel sound signal.

[0071] When extracting features, DenseNet is used to extract the initial features of the fractional order spectrogram, and then the channel attention mechanism is introduced on the basis of DenseNet to further filter and enhance the extracted features, so as to effectively utilize the information in the input data and extract more representative and discriminative deep features;

[0072] The DenseNet network is a deep convolutional neural network with dense connections, in which the output of each layer is directly connected to the input of all subsequent layers, so that each layer can utilize the feature information of the fractional order spectrogram processed by all previous layers to solve the problems of gradient disappearance and model degradation, and improve the performance and generalization ability of the network.

[0073] The channel attention mechanism is a technique for enhancing the network's attention to channel features, and the specific structure is as shown in Figure 3

[0074] The channel attention mechanism evaluates the importance of each channel of the fractional order spectrogram feature map and assigns different weights to each channel accordingly, so that the DenseNet network focuses on important channel features and ignores unimportant features, thereby improving the utilization rate of feature information and the recognition accuracy of the network.

[0075] LSTM is a special RNN that is particularly suitable for processing sequential data, and is designed to avoid long-term dependency problems, successfully addressing the shortcomings of the original recurrent neural network, and becoming the most popular RNN, successfully applied in many fields such as speech recognition, image description, and natural language processing. The LSTM network can delete or add information to the cell state through a structure called a gate, as shown in Figure 4 The gate can selectively decide which information to pass through, and the structure of the gate is a combination of a sigmoid layer and a point multiplication operation. The sigmoid layer outputs a number between 0 and 1, describing how much of each part can pass through, with 0 representing no amount allowed to pass through and 1 representing any amount allowed to pass through. LSTM implements three gate calculations, namely the forget gate, the input gate, and the output gate, to protect and control the cell state.

[0076] BiLSTM is an extension of the traditional LSTM network, which combines forward LSTM and reverse LSTM to obtain more complete context information, thereby better capturing bidirectional semantic dependencies.

[0077] The fractional order spectrogram features extracted by the DenseNet network combined with the channel attention mechanism are used to train the BiLSTM network, and the weights and bias terms of the vocal singing practice quality assessment network model are updated through the backpropagation algorithm and the gradient descent optimizer to minimize the error between the predicted value and the actual value.

[0078] After the vocal singing practice quality assessment network model is trained, the model is used to extract time series features from new vowel singing signals, and these feature vectors are used for subsequent singing quality level evaluation.

[0079] ​Adaptive KELM is an improvement of extreme learning machine (ELM) based on kernel method, and the specific structure of KELM is shown in Figure 5 By selecting a suitable kernel function and determining the kernel parameter, the input data is mapped to a high-dimensional feature space, thereby enhancing the separability of the data. However, the random selection of the kernel parameter S and the regularization coefficient C of KELM will affect the overall generalization performance of the model, which needs to be further optimized.

[0080] In step S4, the kernel extreme learning machine adopts adaptive KELM, which maps the input data to a high-dimensional feature space by selecting a suitable kernel function and determining the kernel parameter, thereby enhancing the separability of the data.

[0081] The quantum firework algorithm QFWA used in step S4 is a quantum version of the firework algorithm FWA, which represents the firework and spark in the solution space by using quantum bits, and updates the positions of the firework and spark by using quantum rotation gates and other operations, so as to find the optimal solution or the approximate optimal solution of the problem. The specific method is as follows:

[0082] Firstly, quantum coding of the firework algorithm is performed, and the position of the firework is quantum coded, and the quantum coding of the firework individual P j is as follows:

[0083]

[0084] In the formula, j is the firework individual number, n is the quantum coding length, and θ jn is the quantum angle of each bit of quantum coding. Secondly, the position of the firework is updated by using the quantum rotation gate, and the amplitude of the state |ψ> of each quantum bit is expressed in the form of probability amplitude The position of the firework individual is updated by using the quantum rotation gate, and the update formula is as follows:

[0085]

[0086] In the formula, and are the probability amplitudes of the i-th firework in the d-dimensional space in the m-th iteration, Δθ is the update angle of the quantum rotation gate, and are the updated probability amplitudes;

[0087] In this step, in order to prevent the influence of too small Δθ on the convergence efficiency of the algorithm or the problem of easily falling into a local optimal solution due to too large Δθ, a nonlinear adaptive rotation angle that gradually changes with the iteration number is used to improve the performance of the quantum firework algorithm, which is specifically shown in the following formula,

[0088]

[0089] In the formula, wherein is the probability amplitude corresponding to a certain quantum bit of the current optimal firework, is the probability amplitude corresponding to a certain quantum bit of the current firework, and θ best and θ i are the angles of a certain quantum bit of the current optimal firework and the current firework on the unit circle, respectively; θ0 is the dynamic rotation angle, and λ is the nonlinear modulation index of the dynamic adjustment of θ0, with a value range of 1.0-2.0, to ensure that |θ best -θ i | is greater, the adjustment amplitude is greater; K is the maximum number of iterations, and Δθ0 is the basic rotation angle, which gradually decreases with the increase of the number of iterations

[0090] Finally, the firework individual mutation operation, the firework individual after the quantum rotation gate position update with a certain probability into the quantum non gate mutation operation, that is, the probability amplitude of each dimension in the quantum individual α i and β i are exchanged:

[0091]

[0092] In the formula, is the quantum probability amplitude after mutation, is the probability amplitude before mutation.

[0093] Step S4 applies the quantum firework algorithm to optimize the hyperparameters of KELM to form a hybrid optimization and learning framework. Adaptive KELM adjusts the kernel parameters to adapt to different data distributions and characteristics, thereby improving the generalization ability and prediction accuracy of the model. The quantum firework algorithm is used to find the optimal regularization coefficient C and kernel function parameter S in the solution space, and adaptive KELM is used to construct and optimize the model based on these parameters. Specifically, the following steps are included:

[0094] Step 1, initialization: randomly generate a set of initial fireworks in the solution space, i.e., candidate kernel parameters S and regularization coefficients C;

[0095] Step 2, fitness evaluation: use adaptive KELM to construct a model based on the current firework parameters, and evaluate the performance of the model on the training data as the fitness of the firework. The evaluation content includes the evaluation of accuracy and loss function value;

[0096] Step 3, quantum firework algorithm optimization: use the quantum firework algorithm to evolve the firework to produce new fireworks, i.e., new candidate parameters. In this process, the number of firework explosions and the explosion amplitude can be adjusted according to the fitness value of the firework to achieve more efficient search.

[0097] Step 4, update: select a part of excellent fireworks as the next generation of fireworks according to the fitness value of the fireworks, and update the solution space;

[0098] step5、iteration and termination: repeat steps 2-4 until the termination condition is met, i.e., the maximum number of iterations is reached or the preset accuracy requirement is met;

[0099] step6、model construction: using the final obtained fireworks parameter including kernel parameter S and regularization coefficient C to construct an adaptive KELM model, and used for subsequent prediction or classification tasks.

[0100] The evaluation method is used for real-time quantitative evaluation of the quality of vocal practice of the singer's vocal vowel sound, in step S5, the vocal sound practice quality evaluation network model is tested and the result is analyzed by the test set, and finally the sound quality evaluation result is obtained, and the result level of the sound quality evaluation result is divided into: excellent, good, medium, pass and poor.

[0101] In this example, the implementation process of the vocal sound practice quality evaluation is as shown in Figure 6 First, the audio signal of the singer's vocal vowel sound practice is collected, and the corresponding quality level is marked according to the pitch, pitch, tone, rhythm and other indicators of the sound; second, based on the short-time fractional Fourier transform, the vowel sound audio signal is converted into a fractional order spectrogram, a sample data set is constructed, and these data are divided into training set, validation set and test set according to the ratio of 8:1:1; second, a fractional order spectrogram deep feature extraction network based on DenseNet and channel attention mechanism is constructed, a series of fractional order spectrogram deep features extracted are input into the BiLSTM network, and the time sequence features of the vowel sound signal are extracted; then, the quantum fireworks algorithm is used to optimize the hyperparameters of the kernel extreme learning machine, and the extracted time sequence features are mapped to a high-dimensional space for quality level decision; finally, the training set and the validation set are used to train the overall model, the model is tested and the result is analyzed through the test set, and finally the sound quality evaluation result is obtained, and the result level is divided into: excellent, good, medium, pass and poor.

[0102] Embodiment:

[0103] The data processing device used in this example uses RTX 4090 (24GB) GPU and 12vCPU Intel(R) Xeon(R) Platinum 8255C CPU@2.50GHz processor, uses python programming language, uses Cuda version 11.0, and uses deep learning framework Pytorch1.7.0.

[0104] The example selects the vowel sound construction dataset sung by the teachers and students of the music department of Fujian Normal University. 300 teachers and students of different ages and genders are selected as the collection objects. In order to ensure the robustness of the model, the same moving coil microphone is used to collect the audio of vowel sound of different collection objects at early, middle and late three time periods. The collected audio is kept in wav format, 6000 pieces of audio are collected for each vowel, 30000 pieces of audio of a, o, e, i, u five vowels are collected, and industry experts are hired to evaluate the quality level according to the sound pitch, pitch, tone, rhythm and other indicators. Then the audio data is labeled according to the quality level. Finally, the dataset is constructed, and the data is divided into training set, validation set and test set according to the ratio of 8:1:1, that is, 24000:3000:3000.

[0105] In order to verify the generation effect of fractional order spectrogram, the audio signals of a, o, e, i, u five vowels are converted into fractional order spectrogram for feature extraction. Taking "a" vowel as an example, the fractional order spectrogram generation and comparison are carried out. The specific waveform is shown in Figure 7 and Figure 8 The quality level of audio 1 is medium, and the quality level of audio 2 is excellent. The 0.1 order-1.0 order fractional order spectrogram is generated, it can be obviously seen that the energy of audio 2 is more concentrated, and the continuity is stronger. Compared with the ordinary spectrogram, the fractional order spectrogram of audio signal of different quality levels has higher discrimination.

Claims

1. A method for evaluating the quality of vocal exercises based on deep learning of fractional-order spectrograms, characterized by: The following steps are included: Step S1: collecting audio signals of a singer's vocal vowel pronunciation practice, and marking corresponding quality levels according to vocalization indicators to construct a sample data set for training, testing and verification of a vocal pronunciation practice quality assessment network model; Step S2: The vocal music pronunciation practice quality assessment network model converts the audio signal into a series of fractional-order spectrograms based on short-time fractional-order Fourier transform; Step S3: constructing a fractional-order spectrogram deep feature extraction network based on DenseNet and channel attention mechanism in the vocal practice quality assessment network model, inputting the extracted fractional-order spectrogram deep features into the BiLSTM network, and extracting the time series features of the vocal practice signal; Step S4: using the quantum fireworks algorithm to optimize the hyperparameters of the kernel extreme learning machine of the vocal phonation practice quality assessment network model, mapping the extracted time series features into a high-dimensional space to make quality grade decisions and form an assessment result; Step S5: Use the training set and the validation set to train the vocal pronunciation practice quality assessment network model, and use the test set to test the model and analyze the results.

2. The method for evaluating the quality of vocal exercises based on deep learning of fractional-order spectrograms according to claim 1, characterized in that: In step S1, singers of different age groups and different genders are selected as collection objects, with a number of no less than 300 people; in order to ensure the robustness of the model, the same dynamic microphone is used to collect audio of vowel sounds of different collection objects in three time periods of morning, noon and evening. The collected audio is saved in wav format, and the number of audio collected for each vowel is no less than 6000 segments, and audio is collected for the five vowels a, o, e, i, and u. Authoritative evaluators assess the quality level of the audio signal based on the intonation, pitch, timbre, and rhythm indicators of the sound, and then label the quality level of these audio data. Finally, a data set is constructed, and these data are divided into training set, validation set, and test set in a ratio of 8:1:

1.

3. The method for evaluating the quality of vocal training based on deep learning of fractional-order spectrograms according to claim 1, characterized in that: The specific method of step S2 is: first, the vowel sound signal is preprocessed, and the preprocessing method includes denoising processing and pre-emphasis processing to reduce noise and interference in the signal; second, the preprocessed vowel sound signal is short-term analyzed, and the frame length and frame shift are set, where the frame length is the window function length, and the entire vowel sound signal is divided into frames according to the frame length and frame shift, and each frame signal is multiplied by the window function, and the signal is divided into a series of overlapping time windows, and the signal in each time window is subjected to fractional Fourier transform FrFT; finally, the energy spectrum generated by each time window is spliced ​​in chronological order to obtain a fractional-order spectrogram of the entire time series.

4. The method for evaluating the quality of vocal training based on deep learning of fractional-order spectrograms according to claim 3 is characterized in that: In step S3, the vocalization practice quality assessment network model extracts the features of the fractional-order spectrograms of order 0.1 to 1.0 respectively through the DenseNet network, and weights the extracted features through the channel attention mechanism; then, the weighted features are input into the Bi-LSTM network for learning and training to extract the time series features of the vowel vocalization signal; When performing feature extraction, DenseNet is first used to extract preliminary features of the fractional-order spectrogram. Then, a channel attention mechanism is introduced based on DenseNet to further filter and enhance the extracted features, effectively utilizing the information in the input data and extracting more representative and discriminative deep features. The DenseNet network is a densely connected deep convolutional neural network. The output of each layer of the network is directly connected to the input of all subsequent layers, so that each layer can use the feature information of the fractional-order spectrogram processed by all previous layers to solve the problems of gradient disappearance and model degradation, and improve the performance and generalization ability of the network. The channel attention mechanism evaluates the importance of each channel of the fractional-order spectrogram feature map and assigns different weights to each channel accordingly, so that the DenseNet network focuses on important channel features and ignores unimportant features, thereby improving the network's utilization of feature information and recognition accuracy. The fractional-order spectrogram features extracted by the DenseNet network and the channel-wise attention mechanism are used to train the BiLSTM network. The weights and biases of the vocal practice quality assessment network model are updated through the backpropagation algorithm and gradient descent optimizer to minimize the error between the predicted and actual values. After the vocal practice quality assessment network model is trained, the model is used to extract time series features from new vowel vocalization signals, and these feature vectors are used for subsequent vocalization quality level assessment.

5. The method for evaluating the quality of vocal training based on deep learning of fractional-order spectrograms according to claim 4 is characterized in that: In step S4, the kernel extreme learning machine adopts an adaptive KELM, which maps the input data to a high-dimensional feature space by selecting a suitable kernel function and determining kernel parameters, thereby enhancing the separability of the data.

6. The method for evaluating the quality of vocal training based on deep learning of fractional-order spectrograms according to claim 5, characterized in that: The quantum fireworks algorithm (QFWA) used in step S4 is a quantized version of the fireworks algorithm (FWA). It represents fireworks and sparks in the solution space using quantum bits and uses quantum rotating gate operations to update the positions of fireworks and sparks, thereby finding the optimal solution or a near-optimal solution to the problem. The specific method is as follows: First, the fireworks algorithm is quantum coded to encode the fireworks position. The fireworks individual P j The quantum code of is: Where j is the firework individual number, n is the quantum code length, θ jn The quantum angle of each quantum bit is encoded. Secondly, the position of the fireworks is updated using a quantum rotating gate. The amplitude of the state of each quantum bit |ψ> is expressed as a probability amplitude form The position of individual fireworks is updated using a quantum revolving door, and the update formula is: Where, and is the probability amplitude of the d-dimensional space of the i-th firework in the th iteration, Δθ is the update angle of the quantum revolving door, and is the updated probability amplitude; In this step, in order to prevent the problem of Δθ being too small and affecting the convergence efficiency of the algorithm during the iteration process, or Δθ being too large and easily falling into the local optimal solution, a nonlinear adaptive rotation angle that gradually changes with the number of iterations is used to improve the performance of the quantum fireworks algorithm, as shown in the following formula: Where, in is the probability amplitude corresponding to a certain quantum bit of the current optimal fireworks, is the probability amplitude corresponding to a certain quantum bit of the current fireworks, θ best and θ i are the angles of the current optimal firework individual and a quantum bit of the current firework individual on the unit circle; θ0 is the dynamic rotation angle, and λ is the nonlinear modulation index of the dynamic adjustment of θ0, with a value range of 1.0 to 2.0 to ensure |θ best -θ i The larger the value, the greater the adjustment range; K is the maximum number of iterations, Δθ0 is the basic rotation angle, which gradually decreases with the increase of the number of iterations. Finally, for the individual fireworks mutation operation, the fireworks individual enters the quantum NOT gate mutation operation with a certain probability after the quantum rotating door position is updated, that is, the probability amplitude α of each dimension in the quantum individual is increased. i and β i To make an exchange: Where, is the quantum probability amplitude after mutation, is the probability amplitude before mutation.

7. The method for evaluating the quality of vocal training based on deep learning of fractional-order spectrograms according to claim 5, characterized in that: Step S4 uses the quantum fireworks algorithm to optimize the hyperparameters of the KELM to form a hybrid optimization and learning framework. The adaptive KELM automatically adjusts the kernel parameters to adapt to different data distributions and characteristics, thereby improving the model's generalization ability and prediction accuracy. The quantum fireworks algorithm is used to find the optimal regularization coefficient C and kernel function parameter S in the solution space, while the adaptive KELM is used to build and optimize the model based on these parameters. The specific steps include the following: Step 1, initialization: randomly generate a set of initial fireworks in the solution space, namely the candidate kernel parameter S and regularization coefficient C; Step 2, fitness evaluation: Use adaptive KELM to build a model based on the current fireworks parameters and evaluate the performance of the model on the training data as the fitness of the fireworks; the evaluation content includes the evaluation of accuracy and loss function value; Step 3. Quantum Fireworks Algorithm Optimization: Use the quantum fireworks algorithm to evolve the fireworks and generate new sparks, i.e., new candidate parameters. In this process, the number of sparks and explosion amplitude generated by the fireworks can be adjusted according to the fitness value of the fireworks to achieve more efficient search. Step 4, Update: Select some excellent sparks as the next generation of fireworks according to the fitness value of the sparks, and update the solution space; Step 5, iteration and termination: Repeat steps 2-4 until the termination condition is met, that is, the maximum number of iterations is reached or the preset accuracy requirement is met; Step 6, model construction: Use the final fireworks parameters including kernel parameter S and regularization coefficient C to build an adaptive KELM model, and use it for subsequent prediction or classification tasks.

8. The method for evaluating the quality of vocal training based on deep learning of fractional-order spectrograms according to claim 1, characterized in that: The evaluation method is used to quantitatively evaluate the quality of a singer's vocal vowel pronunciation practice in real time. In step S5, the vocal pronunciation practice quality evaluation network model is tested and the results are analyzed using a test set, and finally a vocal quality evaluation result is obtained. The vocal quality evaluation result is graded into: excellent, good, medium, passing, and poor.

Images