Personalized vocal music intelligent teaching system and computer program product

Abstract

The invention provides a personalized vocal music intelligent teaching system and a computer program product. The system comprises a front-end data acquisition device and a background server. The front-end equipment is provided with a physiological, psychological and physique parameter determination module; the background server is provided with a vectorization processing device which comprises vectorization, fusion, matching and output units. The physiological module determines physiological parameters, the psychological module determines psychological parameters, and the physique module determines physique parameters. The vectorization module converts parameters into vectors, the fusion unit obtains user feature vectors, the matching unit matches tracks in a vocal music library, and the output unit displays the matched tracks. According to the method, the physiological parameters of the target user are combined with the psychological characteristics and the physical conditions to comprehensively understand the sound expression potential and limitation of the user, the accuracy and depth of sound characteristic positioning are ensured, the physiological, psychological and physical parameters are fused into the user characteristic description vector, and the comprehensive index provides a scientific basis for accurate music selection.

Description

Technical Field

[0001] The present invention relates to the technical field of vocal music teaching, and in particular to a personalized vocal music intelligent teaching system and a computer program product. Background Art

[0002] Traditional vocal music teaching has long been centered around the two core elements of timbre and rhythm. Although this method has cultivated countless outstanding vocal talents throughout history, its inherent limitations are becoming increasingly prominent in the context of modern education. First of all, the teaching method centered on timbre and rhythm often ignores individual differences in vocal music learning. Everyone's physiological structure, psychological characteristics, and physical conditions are unique, and these factors directly affect the production and control of sound. However, in traditional teaching, due to the lack of scientific evaluation and quantitative methods, it is difficult for teachers to accurately identify and provide personalized guidance based on these characteristics of students, resulting in uneven teaching results.

[0003] More importantly, traditional methods have significant deficiencies in achieving accurate individual song selection. Song selection is an important part of vocal learning, which is directly related to learners' interest stimulation, skill improvement and emotional expression. However, in traditional teaching, song selection is often based on teachers' experience judgment or students' subjective preferences, lacking scientific basis. This arbitrariness not only prevents students from fully realizing their potential, but also causes learning frustration due to the mismatch between the difficulty of the repertoire and their own abilities, affecting their learning motivation.

[0004] In addition, traditional vocal music teaching also faces the problem of inaccurate positioning of sound characteristics. Timbre, vocal position, and operability techniques are key elements in vocal music learning, but their evaluation and adjustment in traditional teaching often rely on teachers' auditory perception and experience accumulation, lacking objective and quantitative standards. This subjectivity not only limits the accuracy of teaching, but also causes students to encounter bottlenecks in sound control and skill improvement.

[0005] In summary, traditional vocal teaching methods have obvious technical problems in accurately screening and locating sound characteristics, achieving individual accurate song selection, and scientifically evaluating vocal skills. These problems not only limit the improvement of teaching effects, but also fail to meet the urgent needs of personalized and scientific teaching in modern education. Summary of the invention

[0006] In order to solve the above technical problems, the present application provides a personalized vocal intelligent teaching system and a computer program product to at least solve or alleviate the problems existing in the above-mentioned prior art.

[0007] The embodiment of the present application provides a personalized vocal intelligent teaching system, which includes: a front-end data acquisition device and a back-end server, wherein the front-end data acquisition device is configured with a physiological parameter determination module, a psychological parameter determination module, and a physical parameter determination module, and the back-end server is configured with a vectorization processing device, and the vectorization processing device includes: a vectorization module, a vector fusion unit, a vector matching unit, and an output unit;

[0008] The physiological parameter determination module is used to determine the physiological parameters of the target user;

[0009] The psychological parameter determination module is used to determine the psychological parameters of the target user;

[0010] The physical parameter determination module is used to determine the physical parameters of the target user;

[0011] The vectorization module is used to vectorize the physiological parameters, the psychological parameters, and the physical parameters respectively to obtain the physiological description vector, the psychological description vector, and the physical description vector of the target user;

[0012] The vector fusion unit is used to fuse the physiological description vector, the psychological description vector, and the physical description vector of the target user to obtain a user feature description vector;

[0013] The vector matching unit is used to match vocal functional repertoires in the constructed vocal knowledge base based on the user feature description vector;

[0014] The output unit is used to output the vocal functional music tracks matching the target user to the front-end data acquisition device for display.

[0015] A computer program product having a computer executable program stored thereon, wherein the computer executable program is run to perform the following steps:

[0016] Vectorizing the physiological parameters, psychological parameters, and physical parameters respectively to obtain a physiological description vector, a psychological description vector, and a physical description vector of the target user;

[0017] The physiological description vector, the psychological description vector, and the physical description vector of the target user are integrated to obtain a user feature description vector;

[0018] Matching vocal functional repertoires in a constructed vocal knowledge base based on the user feature description vector;

[0019] The vocal functional repertoire matching the target user is output.

[0020] In this application, the physiological parameters of the target user, such as vocal cord length, resonance cavity size, etc., are obtained through the front-end data acquisition device. These parameters directly affect the formation of timbre. Through vectorization processing, the system can accurately describe the physiological characteristics of the user and provide an objective basis for locating the sound characteristics. Combined with the user's psychological characteristics (such as emotional stability, creativity) and physical conditions (such as physical fitness, endurance, breathing frequency, and vital capacity), the system can more comprehensively understand the user's voice performance potential and limitations. This multi-dimensional analysis ensures the accuracy and depth of the sound feature positioning. The system integrates physiological, psychological, and physical parameters into a user feature description vector through a vector fusion unit. This comprehensive indicator provides a scientific basis for accurate song selection. Based on the user feature description vector, the system performs intelligent matching in the constructed vocal knowledge base to find the most suitable vocal functional repertoire for the user's current state and learning goals. This personalized song selection not only improves learning efficiency, but also enhances the fun and pertinence of learning. Moreover, based on the results of vectorization processing, the system can provide targeted skill improvement suggestions to help users gradually master standardized operability skills, so as to continuously improve in vocal learning. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The drawings constituting a part of this application are used to provide a further understanding of this application, so that other features, purposes and advantages of this application become more obvious. The schematic embodiment drawings and their descriptions of this application are used to explain this application and do not constitute an improper limitation on this application. In the drawings:

[0022] Figure 1 It is a structural schematic diagram of a personalized vocal intelligent teaching system according to an embodiment of the present application. DETAILED DESCRIPTION

[0023] In order to enable those skilled in the art to better understand the solution of the present application, the following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this application.

[0024] Figure 1 Schematic diagram of a personalized vocal intelligent teaching system according to an embodiment of the present application. Figure 1As shown, the personalized vocal intelligent teaching system includes: a front-end data acquisition device and a back-end server, the front-end data acquisition device is configured with a physiological parameter determination module, a psychological parameter determination module, and a physical parameter determination module, and the back-end server is configured with a vectorization processing device, and the vectorization processing device includes: a vectorization module, a vector fusion unit, a vector matching unit, and an output unit;

[0025] The physiological parameter determination module is used to determine the physiological parameters of the target user;

[0026] The psychological parameter determination module is used to determine the psychological parameters of the target user;

[0027] The physical parameter determination module is used to determine the physical parameters of the target user;

[0028] The vectorization module is used to vectorize the physiological parameters, the psychological parameters, and the physical parameters respectively to obtain the physiological description vector, the psychological description vector, and the physical description vector of the target user;

[0029] The vector fusion unit is used to fuse the physiological description vector, the psychological description vector, and the physical description vector of the target user to obtain a user feature description vector;

[0030] The vector matching unit is used to match vocal functional repertoires in the constructed vocal knowledge base based on the user feature description vector;

[0031] The output unit is used to output the vocal functional music tracks matching the target user to the front-end data acquisition device for display.

[0032] In this application, the physiological parameters of the target user, such as vocal cord length, resonance cavity size, etc., are obtained through the front-end data acquisition device. These parameters directly affect the formation of timbre. Through vectorization processing, the system can accurately describe the physiological characteristics of the user and provide an objective basis for locating the sound characteristics. Combined with the user's psychological characteristics (such as emotional stability, creativity) and physical conditions (such as physical fitness and endurance), the system can more comprehensively understand the user's voice performance potential and limitations. This multi-dimensional analysis ensures the accuracy and depth of the sound feature positioning. The system integrates physiological, psychological, and physical parameters into a user feature description vector through a vector fusion unit. This comprehensive indicator provides a scientific basis for accurate song selection. Based on the user feature description vector, the system performs intelligent matching in the constructed vocal knowledge base to find the most suitable vocal functional repertoire for the user's current state and learning goals. This personalized song selection not only improves learning efficiency, but also enhances the fun and pertinence of learning. Moreover, based on the results of vectorization processing, the system can provide targeted skill improvement suggestions to help users gradually master standardized operability skills, so as to continuously improve in vocal learning.

[0033] Optionally, the physiological parameters include at least one of breathing efficiency, vocal cord status, and resonance cavity.

[0034] Optionally, when determining the physiological parameters of the target user, the physiological parameter determination module obtains the respiratory monitoring data generated by the smart wearable device, and separates the respiratory monitoring data to obtain respiratory efficiency characterization data, vocal cord status characterization data, and resonance cavity characterization data, respectively, and performs feature extraction on the respiratory efficiency characterization data, the vocal cord status characterization data, and the resonance cavity characterization data, respectively, to obtain the respiratory efficiency, the vocal cord status, and the resonance cavity.

[0035] Optionally, the physiological parameter determination module includes: a data preprocessing module, a dual-channel cycle processing module, and a feature extraction module;

[0036] When determining the physiological parameters of the target user, the physiological parameter determination module includes the following steps:

[0037] The data preprocessing module communicates and interacts with the smart wearable device to obtain the generated respiratory monitoring data, and serializes the respiratory monitoring data to obtain serialized data, and blocks the serialized data to obtain data blocks that are continuous in time;

[0038] The dual-channel loop processing module performs dual-channel loop processing on the temporally continuous data blocks to separate the breathing efficiency representation data, the vocal cord state representation data, and the resonance cavity representation data from the temporally continuous data blocks;

[0039] The feature separation and extraction are used to respectively extract features from the breathing efficiency characterization data, the vocal cord state characterization data, and the resonance cavity characterization data to obtain the breathing efficiency, the vocal cord state, and the resonance cavity.

[0040] To this end, the above physiological parameter determination module obtains the respiratory monitoring data of the smart wearable device and performs a series of processing to determine the physiological parameters, which has the following technical benefits:

[0041] (1) Including breathing efficiency, vocal cord status, resonance cavity, etc. in the range of physiological parameters can more comprehensively reflect the physiological conditions related to the user's voice. Obtaining the breathing monitoring data generated by smart wearable devices provides an objective basis for evaluation and can accurately determine these physiological parameters. For example, accurate breathing efficiency data helps to understand the user's use of breath when speaking, thereby providing a basis for subsequent teaching guidance.

[0042] (2) The data preprocessing module serializes and blocks the respiratory monitoring data, converting it into a format that is convenient for subsequent analysis, thereby improving data processing efficiency. For example, serialization can unify the data format, and block division can make the data more orderly in the time dimension, which is convenient for the dual-channel loop processing module to operate and effectively extract valuable information.

[0043] (3) The design of dual-channel loop processing module and feature separation and extraction can deeply mine features such as breathing efficiency, vocal cord status, and resonance cavity from respiratory monitoring data. This helps to discover potential physiological feature relationships. For example, by analyzing the dependency between adjacent data blocks (dual-channel loop processing), parameters such as vocal cord status can be more accurately determined, thereby providing more targeted guidance for vocal teaching and helping users better understand and improve their own vocal physiological conditions.

[0044] When the physiological parameter determination module is specifically implemented to obtain the respiratory monitoring data of the smart wearable device and perform a series of processing to determine the physiological parameters, the following process is included:

[0045] For the data preprocessing module, a stable communication connection is established with the smart wearable device, and wireless communication protocols such as Bluetooth Low Energy (BLE) or Wi-Fi are used for data transmission to ensure the real-time and stability of data acquisition. After acquiring the respiratory monitoring data, it is converted into a format suitable for subsequent processing according to the pre-set data format specification, such as converting the original analog signal collected by the sensor into a digital signal, and organizing it according to a specific data structure to facilitate serialization operations. During the serialization process, an efficient encoding algorithm, such as binary encoding or a custom compact text encoding format, is used to convert the respiratory monitoring data into a continuous data stream to reduce the overhead of data storage and transmission. The block operation is divided based on the time window or the amount of data. For example, set every 5 seconds of data as a data block, or divide it into a new data block when the data volume reaches 1000 sampling points, to ensure that each data block has a certain continuity and integrity in time, which is convenient for the subsequent dual-channel loop processing module to extract local and global data features.

[0046] For the dual-channel recurrent processing module, the long short-term memory network (LSTM) or gated recurrent unit (GRU) is used as the basic unit of the first recurrent neural network, and its good processing ability for sequence data is used to analyze the respiratory monitoring data in a single data block. The network extracts local features related to respiratory efficiency, vocal cord state, and resonance cavity by learning the data pattern of each time step in the data block. For example, by analyzing the fluctuation law of respiratory monitoring data in a data block, the rhythm and depth changes of breathing are preliminarily judged, so as to obtain the relevant features of the respiratory efficiency characterization data. The second recurrent neural network is also constructed based on LSTM or GRU, which focuses on the dependency analysis between adjacent data blocks. By modeling the continuity characteristics of adjacent data blocks in time, the changing trends of breathing patterns, vocal cord vibration characteristics, and resonance cavity states on a longer time scale are learned. For example, the gradual change of respiratory frequency in multiple consecutive data blocks is observed to more accurately determine the overall respiratory efficiency characterization data, and the association and change law of vocal cord state and resonance cavity state in different time periods are identified, so as to generate a block-to-block dependency feature sequence containing rich time-dependent information.

[0047] For the multi-layer perception processing of the output neural network, the output neural network adopts a multi-layer perceptron (MLP) structure and receives the inter-block dependency feature sequence generated by the second recurrent neural network as input. The multi-layer perceptron performs nonlinear transformation and combination on the input features through multiple hidden layers, and gradually abstracts the high-level feature representations directly related to breathing efficiency, vocal cord status, and resonance cavity. For example, activation functions such as sigmoid or ReLU are used in the hidden layer to enhance the nonlinear expression ability of the network. Finally, these features are mapped into specific breathing efficiency representation data, vocal cord status representation data, and resonance cavity representation data through the output layer, realizing the effective separation and quantitative representation of physiological parameters.

[0048] For the feature separation and extraction module, for the respiratory efficiency characterization data, a method based on combining time domain and frequency domain analysis is used for feature extraction. In the time domain, features such as the respiratory cycle and the ratio of inhalation and exhalation time are calculated; in the frequency domain, the respiratory frequency component is analyzed by fast Fourier transform (FFT), and features such as the dominant frequency and frequency bandwidth are extracted. These features are combined to obtain accurate respiratory efficiency parameters. For the vocal cord state characterization data, audio signal processing technology is used, such as calculating the Mel-frequency cepstral coefficient (MFCC) to characterize the sound spectrum characteristics generated by the vibration of the vocal cords. At the same time, the fundamental frequency and harmonic structure analysis of the sound are combined, and the detailed features such as the closed and open states of the vocal cords are judged by detecting the mutation points and energy changes in the sound signal, so as to accurately extract the vocal cord state parameters. For the resonance cavity characterization data, the energy distribution of the sound in different frequency bands is analyzed, and the features related to the shape, size and resonance characteristics of the resonance cavity are determined by comparing with the standard resonance cavity model. For example, detecting enhanced energy in the low-frequency band indicates better chest resonance, and prominent energy in the high-frequency band is related to head cavity resonance. Based on this, the resonance cavity parameters are extracted to provide comprehensive and accurate physiological basic data for personalized guidance in subsequent vocal teaching.

[0049] Optionally, the dual-channel loop processing module performs dual-channel loop processing on the temporally continuous data blocks to separate the breathing efficiency characterization data, the vocal cord state characterization data, and the resonance cavity characterization data from the temporally continuous data blocks, including:

[0050] The first recurrent neural network in the dual-channel recurrent processing module performs an intra-block operation on a single data block to obtain data components of the single data block;

[0051] The second recurrent neural network in the dual-channel recurrent processing module performs inter-block operations on adjacent data blocks to obtain dependency relationships between data components of adjacent data blocks and generate an inter-block dependency feature sequence based on the dependency relationships;

[0052] The output neural network in the dual-channel loop processing module performs multi-layer perception processing on the dependent feature sequence to obtain breathing efficiency characterization data, vocal cord state characterization data, and resonance cavity characterization data.

[0053] To this end, the above solution has the following technical advantages:

[0054] (1) Through the intra-block operation of the first recurrent neural network and the inter-block operation of the second recurrent neural network, the local (within a single data block) and global (between adjacent data blocks) characteristics of the data can be captured simultaneously, realizing the fusion of multi-scale information. This multi-scale analysis helps to more comprehensively and accurately understand the information related to vocal physiology contained in the respiratory monitoring data, thus laying the foundation for the subsequent accurate separation of respiratory efficiency, vocal cord status, and resonance cavity characterization data.

[0055] (2) Recurrent Neural Networks (RNNs) and their variants (such as LSTM or GRU) have a natural advantage in processing time series data. In the analysis of vocal physiological parameters, breathing, vocal cord vibration, and resonance cavity state change dynamically over time. This architecture can adapt well to this dynamic nature, effectively mine the time series patterns in the data, and improve the understanding of the vocal physiological process and the accuracy of parameter extraction.

[0056] (3) Focusing on operations within a single data block, it can accurately identify and extract subtle features and patterns within each data block. For example, for respiratory monitoring data, in a shorter time segment, it can more clearly capture the rapid changes in breathing rhythm, short-term abnormal vocal cord vibration, or the immediate response characteristics of the resonance cavity. These local features are crucial for a detailed understanding of the physiological state at the moment of phonation, and provide rich detailed information for subsequent comprehensive analysis.

[0057] (4) During the intra-block operation, the noise in the data can be initially filtered by setting appropriate thresholds or using filtering algorithms. At the same time, the main data components closely related to breathing efficiency, vocal cord status, and resonance cavity status can be screened out to reduce the interference of irrelevant information and improve the efficiency and accuracy of subsequent processing.

[0058] (5) Analyzing the dependency of data components between adjacent data blocks can reveal the changing trends and mutual influence of vocal physiological parameters over a longer time frame. For example, observing the gradual changing trends of breathing depth and frequency in multiple consecutive data blocks can more accurately evaluate the overall breathing efficiency; analyzing the coherence and evolution of vocal cord vibration characteristics in a series of data blocks can help determine the stability and potential problems of the vocal cord state; for the resonance cavity, understanding the changes in its resonance characteristics in different time periods can better grasp its dynamic adjustment during the phonation process, thus providing a long-term perspective for a comprehensive evaluation of the vocal physiological mechanism.

[0059] (6) By considering the dependency between adjacent data blocks, the generated inter-block dependency feature sequence is more robust to local fluctuations and noise in the data. Even if there is a certain degree of measurement error or outlier in a data block, the overall feature sequence can still accurately reflect the real changes in the vocal physiological state due to the integration of information from adjacent data blocks, thus improving the reliability of subsequent data analysis and parameter extraction.

[0060] (7) The multi-layer structure and nonlinear activation functions (such as sigmoid or ReLU) of the multi-layer perceptron (MLP) enable it to perform complex nonlinear transformations and abstractions on the input inter-block dependent feature sequences. This capability helps map complex time series features into a feature space that is easier to understand and process, thereby more clearly separating the respiratory efficiency, vocal cord status, and resonance cavity representation data. For example, through the learning of the multi-layer perceptron, specific patterns related to respiratory efficiency can be accurately identified from the mixed physiological signal features, as well as the unique feature representations corresponding to the vocal cord status and resonance cavity status, thereby achieving accurate quantification and differentiation of different physiological parameters.

[0061] (8) The output neural network receives the global feature information (inter-block dependent feature sequence) from the second recurrent neural network and integrates and optimizes this information in the multi-layer perception processing process. It can comprehensively consider the relationship between each data block and the overall time series characteristics, thereby obtaining more representative and accurate data on breathing efficiency, vocal cord status, and resonance cavity characterization. This helps to overcome the limitations of only local features or simple linear analysis, and provide a more reliable physiological parameter basis for the vocal teaching system to achieve personalized teaching guidance and training optimization.

[0062] In the specific implementation, in the technical implementation details of the first recurrent neural network (RNN) block operation, the long short-term memory network (LSTM) is selected as the basic structure of the first recurrent neural network for network structure selection and initialization, because it can effectively avoid the gradient vanishing problem when processing long sequence data and better capture the long-term dependencies in the data. The weight matrix and bias vector of the LSTM network are initialized, and a random initialization method (such as Xavier initialization or He initialization) is used to ensure that the network has a suitable parameter distribution at the beginning of training, which is conducive to the subsequent learning process. In the data input and processing link, the respiratory monitoring data in a single data block is input into the LSTM network in time steps. For example, if the respiratory monitoring data is recorded at a frequency of 100 samples per second, and each data block contains 5 seconds of data, these 500 samples are input into the network as 500 time steps in sequence. Finally, at each time step, the LSTM unit calculates based on the input data, the hidden state of the previous moment, and the cell state. Through operations such as the forget gate, the input gate, the cell state update, and the output gate, the hidden state is gradually updated, thereby learning the temporal characteristics of the data in the data block.

[0063] For feature extraction and output, after processing the entire data block, the last hidden state of the LSTM network will be used as the data component representation of the data block. This hidden state contains the network's learning results of the breathing pattern, vocal cord vibration characteristics, and resonance cavity state within the data block within the local time range. In order to further enhance the feature expression capability, additional linear transformation or nonlinear activation operations can be performed on the last hidden state to obtain the final data component output of a single data block, providing basic data for subsequent inter-block operations.

[0064] For the inter-block operation of the second recurrent neural network, in the organization and input of adjacent data blocks, the adjacent data blocks are input into the second recurrent neural network in chronological order, and the LSTM structure is also used to process the relationship between blocks. For example, the output of the previous data block (i.e., its last hidden state) is used together with the input data of the current data block as the input of the current LSTM unit, so that the network can learn the dynamic relationship between the data components of adjacent data blocks. When inputting data, the output of the previous data block needs to be properly processed, such as dimensional adjustment or feature mapping, so that it matches the input data format of the current data block so that the LSTM unit can process it correctly.

[0065] For dependency learning and feature sequence generation, when the LSTM network processes adjacent data blocks, the network can learn the temporal dependencies of data components through the transmission and update of hidden states. For example, if the breathing rate in the previous data block gradually increases, and the amplitude of vocal cord vibration in the current data block also increases accordingly, the network can capture this association and encode it in the hidden state. At each time step, the current hidden state is calculated based on the current input data and the hidden state at the previous moment, and is stored as part of the inter-block dependency feature sequence. After processing all adjacent data blocks, a feature sequence reflecting the dependencies between data blocks is finally obtained, which contains the trend information of the change of vocal physiological parameters over a longer time range.

[0066] Technical implementation of multi-layer perception processing for output neural network, where for network architecture construction, a multi-layer perceptron (MLP) is constructed, including an input layer, multiple hidden layers and an output layer. The number of neurons in the input layer matches the dimension of the inter-block dependency feature sequence in order to receive the output from the second recurrent neural network. The number of hidden layers and the number of neurons are designed according to actual needs. In general, as the number of hidden layers increases, the expressive power of the network increases, but it also leads to an increased risk of overfitting. The appropriate hidden layer structure can be determined by experiments or by using regularization techniques (such as L2 regularization). For example, 2-3 hidden layers are set, each containing 64-128 neurons.

[0067] For data forward propagation and nonlinear transformation, the inter-block dependent feature sequence is input into the input layer of the MLP, and then the data is passed to the hidden layer and output layer in sequence through the forward propagation algorithm. In the hidden layer, each neuron performs weighted summation on the input data and activates it through a nonlinear activation function (such as the ReLU function), introducing nonlinear factors to enhance the network's ability to model complex feature relationships. As the data propagates in the multi-layer perceptron, the network gradually abstracts and transforms the input inter-block dependent feature sequence and maps it to a new feature space, so that the respiratory efficiency, vocal cord status, and resonance cavity representation data can be more clearly separated and represented in this space.

[0068] In the design and parameter prediction of the output layer, the number of neurons in the output layer is determined according to the number of physiological parameters to be predicted. In this case, there are 3 neurons (corresponding to the respiratory efficiency characterization data, the vocal cord state characterization data, and the resonance cavity characterization data). The activation function of the output layer is selected according to the specific prediction task. For example, for the respiratory efficiency characterization data, a linear activation function can be used because its value range is usually continuous; for the vocal cord state and resonance cavity characterization data, if classification or discretization is required (such as dividing the vocal cord state into normal, mildly abnormal, severely abnormal, etc.), a suitable classification activation function (such as the softmax function) can be selected. By training the MLP network, it can accurately predict the corresponding respiratory efficiency, vocal cord state, and resonance cavity characterization data based on the input inter-block dependency feature sequence, thereby realizing the accurate extraction of vocalization physiological parameters.

[0069] Optionally, after the output neural network in the dual-channel loop processing module performs multi-layer perception processing on the dependent feature sequence to obtain breathing efficiency characterization data, vocal cord state characterization data, and resonance cavity characterization data, it also includes: performing feature fusion on the breathing efficiency characterization data, vocal cord state characterization data, and resonance cavity characterization data to obtain sound fusion features; determining a breathing classification description based on the sound fusion features, so as to match vocal functional repertoires according to the breathing classification description and generate breathing adjustment instructions, wherein the breathing adjustment instructions are used to guide the user to adjust the breathing method.

[0070] To this end, based on the above-mentioned feature fusion, the physiological information of voice production can be integrated to achieve the fusion of breathing efficiency characterization data, vocal cord state characterization data, and resonance cavity characterization data, and the information of multiple physiological dimensions closely related to voice production can be integrated into a unified sound fusion feature. This makes the description of the voice production process more comprehensive and comprehensive, avoiding the limitations of considering a certain physiological parameter alone, thereby providing a rich information basis for a more accurate understanding of the user's voice mechanism and sound characteristics. To further enhance the representativeness of the features, through the fusion operation, the synergy and mutual relationship between different physiological parameters can be reflected, and the generated sound fusion features can better represent the overall physiological state of the user when speaking. For example, the relationship between breathing efficiency and the coordination of the resonance cavity, the degree of dependence of the vocal cord state on respiratory support, etc. can be reflected in the fusion features. Such features have stronger representativeness and distinguishing ability in subsequent analysis and application, which helps to improve the accuracy of the vocal teaching system in locating the user's sound characteristics.

[0071] In addition, when the breathing classification description is determined according to the sound fusion feature, personalized breathing evaluation can be achieved. Determining the breathing classification description based on the sound fusion feature can achieve a more detailed and personalized evaluation of the user's breathing method in the vocal sounding context. Different users have similar breathing efficiency values, but after combining the vocal cord state and the resonance cavity state, their actual breathing performance and applicability during vocalization are completely different. The breathing classification description obtained through this comprehensive evaluation can more accurately reflect the matching degree and characteristics of the user's breathing and vocalization, providing a key basis for personalized teaching. In addition, the selection of adaptive vocal functional repertoires is achieved, and different repertoires have different requirements for breathing control in vocal learning. By determining the breathing classification description, the user's breathing characteristics can be accurately matched with the breathing requirements of the repertoire in the vocal knowledge base. For example, for lyrical repertoires that require long-term stable breath support, the system can screen out users with good breathing control ability and suitable for such repertoires, and recommend corresponding repertoires for them, which improves the scientificity and pertinence of repertoire selection, helps users better play their own advantages in the singing process, and also helps to improve teaching effects.

[0072] Finally, generating breathing adjustment instructions can guide vocal training in real time. The generated breathing adjustment instructions can provide users with real-time, targeted feedback and guidance, helping users to adjust their breathing methods in time during vocal training. This helps users master correct breathing techniques more quickly, improve breathing efficiency, and optimize the coordination between breathing and vocalization, thereby improving sound quality and singing performance. For example, if the system detects that the user's breathing is too shallow when vocalizing and cannot provide sufficient breath support for the vocal cords, it can generate corresponding breathing adjustment instructions to guide the user to deepen the breathing depth and enhance breath control ability.

[0073] Moreover, it can also promote skill improvement and personalized training. It generates personalized breathing adjustment instructions based on the user's specific breathing classification description, making the training content more in line with the user's actual needs and avoiding the drawbacks of the unified guidance model in traditional teaching. Each user can conduct targeted breathing training according to their own characteristics, gradually overcome their own breathing problems, and continuously improve their vocal skills. At the same time, as the user's breathing skills improve, the system can re-evaluate and adjust the training plan according to the new breathing state, forming a dynamic and personalized teaching cycle, and promoting the user's continuous progress on the road of vocal learning.

[0074] In the specific implementation, for feature fusion, for example, feature selection and weighting strategies are adopted: First, analyze the breathing efficiency characterization data, vocal cord state characterization data, and resonance cavity characterization data to determine the importance of each feature in the vocalization process. For example, through experiments or based on the knowledge of vocal experts, it is found that the influence of vocal cord state on sound quality is relatively high, while the importance of breathing efficiency and resonance cavity characteristics varies in different singing situations. Assign corresponding weights to each feature according to importance. The weights can be set manually (based on prior knowledge) or automatically learned through machine learning algorithms (such as using the attention mechanism in a neural network or by training a small regression model to determine the weights). The weighted breathing efficiency characterization data, vocal cord state characterization data, and resonance cavity characterization data are linearly added or a more complex fusion function (such as a fusion method based on a kernel function) is used to obtain the sound fusion feature. For example, the sound fusion feature = w1*breathing efficiency characterization data + w2*vocal cord state characterization data + w3*resonance cavity characterization data, where w1, w2, and w3 are corresponding weights.

[0075] In addition, before feature fusion, the respiratory efficiency characterization data, vocal cord state characterization data, and resonance cavity characterization data are normalized to have the same dimension and numerical range. Common normalization methods include minimum-maximum normalization (mapping eigenvalues ​​to a specific interval, such as [0,1]) or standardization (converting eigenvalues ​​to a normal distribution with a mean of 0 and a standard deviation of 1). Normalization helps to avoid certain features dominating the fusion process due to excessively large or small values, ensuring that each feature can reasonably contribute information in the fusion and improving the accuracy and stability of the fused features. For example, in the case where the respiratory efficiency characterization data has a large range of values, while the vocal cord state characterization data has relatively small values, normalization can make them have equal influence during fusion.

[0076] To achieve the determination of the description of breathing classification, a classification model is established: a large amount of annotated breathing feature data (including breathing efficiency, vocal cord status, resonance cavity characteristics and corresponding breathing classification labels, such as shallow breathing, deep breathing, steady breathing, etc.) is collected to build a training data set. A breathing classification model is trained using machine learning algorithms (such as support vector machines, decision trees, neural networks, etc.). Taking neural networks as an example, a classification network is constructed that includes an input layer (receiving sound fusion features), a hidden layer (for learning the complex relationship between features and classifications), and an output layer (outputting breathing classification labels). During the training process, the model parameters are adjusted by optimizing the loss function (such as the cross entropy loss function) so that the model can accurately predict the breathing classification based on the input sound fusion features. Stochastic gradient descent (SGD) or its variants (such as Adagrad, Adadelta, etc.) algorithms can be used to update parameters to improve training efficiency and model convergence speed.

[0077] Regarding classification decision and description generation, in the specific implementation, the real-time acquired sound fusion features are input into the trained breathing classification model, and the model makes predictions based on the learned patterns and parameters to obtain the corresponding breathing classification labels. Generate corresponding breathing classification descriptions based on the breathing classification labels. For example, if the classification label is "shallow breathing", the breathing classification description can be "The user breathes shallowly when speaking, resulting in insufficient breath support and lack of strength and stability in the sound. It is recommended to try to deepen the breathing and increase breath reserves." The description not only contains the classification results, but also provides some simple explanations and improvement suggestions so that users can better understand their breathing status and improvement directions.

[0078] For the matching of vocal functional repertoires, in the specific implementation, a vocal knowledge base and feature extraction are constructed to collect various vocal functional repertoires (such as repertoires for training breath control, repertoires for demonstrating high-pitched skills, repertoires suitable for lyrical singing, etc.), and feature extraction is performed on each repertoire. The extracted features include but are not limited to the rhythm change characteristics of the repertoire (such as rhythm speed, rhythm stability), pitch characteristics (such as the highest note, the lowest note, and the pitch span), emotional expression characteristics (such as cheerful, sad, passionate and other emotional tendencies), and the degree of requirement for breathing control (based on music analysis and vocal expert experience annotation). These repertoire features are quantified, for example, using numerical vectors to describe the characteristics of each repertoire, and a vocal knowledge base is constructed. A variety of feature extraction methods can be used, such as audio signal processing technology (calculating Mel frequency cepstral coefficients, short-time energy, etc.) combined with music theory analysis (rhythm analysis, harmony analysis, etc.) to comprehensively and accurately describe the repertoire features.

[0079] For the matching algorithm, the tracks are matched in the vocal knowledge base according to the user's breathing classification description (reflecting the user's breathing ability and characteristics). The designed matching algorithm calculates the similarity or matching degree between the user's breathing characteristics and the breathing control requirement characteristics in the track. For example, the cosine similarity algorithm or a distance metric-based method (such as Euclidean distance) can be used to measure the difference between the user's breathing feature vector and the track's breathing feature vector. The higher the similarity, the more suitable the track is for the user's current breathing ability and needs. The tracks in the vocal knowledge base are sorted according to the matching degree, and the top N tracks with the highest matching degree are selected as recommended tracks and output to the user, so as to achieve personalized matching of vocal functional tracks.

[0080] Regarding the technical implementation of generating breathing adjustment instructions, for example, instruction templates and rules can be used: according to different breathing classification descriptions, corresponding breathing adjustment instruction templates and rules are formulated. For example, for the "shallow breathing" classification, the instruction template can be "Please try to breathe through the abdomen, relax the abdominal muscles, expand the abdomen outward when inhaling, slowly contract the abdomen when exhaling, extend the exhalation time, and each exercise lasts for [X] minutes." The instruction rules should be based on the correct breathing methods and training principles in vocal teaching, and personalized adjustments should be made based on the actual situation of the user. Detailed instruction rules can be formulated by referring to the teaching experience and related research results of vocal experts to ensure that the adjustment instructions are scientific and effective.

[0081] For dynamic adjustment of instruction parameters, the parameters in the breathing adjustment instructions are dynamically adjusted according to the user's real-time breathing monitoring data and changes in sound fusion characteristics. For example, if the user's breathing depth improves but still does not reach the ideal state after following the instructions for breathing training, the system can appropriately increase the practice time or adjust the breathing rhythm requirements to further optimize breathing control. By continuously monitoring the user's training progress and feedback information, the breathing adjustment instructions are optimized and customized in real time, so that the instructions always adapt to the user's needs, helping the user to gradually improve their breathing skills and achieve better vocalization effects.

[0082] Optionally, the psychological parameter includes at least one of a personality classification, an emotional expression description, and a psychological development stage identification.

[0083] Optionally, when determining the psychological parameters of the target user, the psychological parameter determination module obtains psychological assessment data, and separates the psychological assessment data to obtain personality classification representation data, emotional expression description representation data, and psychological development stage representation data, respectively, and performs feature extraction on the personality classification representation data, emotional expression description representation data, and psychological development stage representation data, respectively, to obtain the personality classification, the emotional expression description, and the psychological development stage identifier.

[0084] Optionally, the psychological parameter determination module includes: an encoding module, a feature head module, and a context module;

[0085] The psychological parameter determination module performs the following steps when determining the psychological parameters of the target user:

[0086] Encoding the psychological assessment data based on the encoding module to obtain a hidden feature vector;

[0087] Based on the feature head module, the hidden feature vector is nonlinearly mapped to obtain feature representations related to personality classification, emotional expression, and psychological development stage;

[0088] Based on the context module, context-related decision prediction processing is performed on the feature representations related to personality classification, emotional expression, and psychological development stage to extract features of the personality classification representation data, emotional expression description representation data, and psychological development stage representation data, respectively, to obtain the personality classification, the emotional expression description, and the psychological development stage identifier.

[0089] To this end, we use multi-dimensional psychological parameters to comprehensively understand the psychological characteristics of users, and include personality classification, emotional expression description, and psychological development stage identification in the scope of psychological parameters, which can deeply analyze the psychological conditions of target users from multiple dimensions. Different personality types affect the attitude and method of users towards vocal learning. For example, extroverts are more inclined to express themselves in performances, while introverts have unique advantages in delicate emotional expression; emotional expression descriptions help understand the depth and diversity of emotions that users can convey in vocal singing; psychological development stage identification can reflect the maturity of users in cognition, emotion, etc., thereby providing a richer basis for personalized teaching and making teaching strategies more in line with the psychological characteristics of users.

[0090] In addition, based on the processing of the encoding module, data compression and feature abstraction are realized. The encoding module encodes the psychological assessment data to obtain hidden feature vectors, realizing data compression and abstract representation of features. Psychological assessment data often has high dimensions and complexity, and direct processing faces problems such as large amount of calculation and information redundancy. Through encoding, the original data can be converted into low-dimensional, compact hidden feature vectors, which not only retains key information, but also reduces the amount of data for subsequent processing, improves processing efficiency, and helps to highlight the main feature patterns in the data, laying the foundation for subsequent analysis and prediction.

[0091] In addition, the feature head module uses nonlinear mapping to separate the features of different psychological dimensions. The feature head module maps the hidden feature vectors to the probability distribution space of personality type, the vector space of emotional expression tendency, and the label space of psychological development stage through nonlinear mapping, and can effectively separate the feature representations related to personality classification, emotional expression, and psychological development stage. This separation makes the features of each psychological dimension clearer and more independent, avoids confusion and interference between different psychological features, and helps to more accurately identify and understand the characteristics of users in various psychological dimensions, thereby providing more targeted guidance for personalized vocal teaching, such as selecting appropriate teaching methods based on the personality characteristics of users, and designing corresponding repertoire training based on emotional expression characteristics.

[0092] Furthermore, the decision prediction processing of the context module takes into account the dynamic nature of psychological characteristics, so that the context module can fully consider the dynamic changes of the user's psychological characteristics in different situations and time based on the context-related decision prediction processing. In the process of vocal learning, the user's psychological state will change due to various factors, such as learning progress, performance pressure, etc. By introducing context information, the system can capture these changes more accurately, and dynamically update and extract features of personality classification representation data, emotional expression description representation data, and psychological development stage representation data, so that the obtained psychological parameters can better reflect the user's current true psychological state, so as to adjust the teaching strategy in time, provide personalized teaching services that are more in line with the actual needs of users, and promote the continuous development of users in vocal learning.

[0093] In the specific implementation, for the acquisition and preprocessing of psychological assessment data, a variety of psychological assessment methods are used to collect data, such as questionnaires, psychological testing software, and behavioral observation and analysis under audio / video recording. Questionnaires can design a series of questions related to personality, emotional expression and psychological development, covering the user's interests, interpersonal communication methods, emotional response patterns, growth experiences, etc.; psychological testing software can use professional psychological assessment tools, such as MBTI (Myers-Briggs Type Indicator) personality test, Big Five personality test, etc., to obtain more standardized personality classification data; behavioral observation and analysis records the user's audio and video in specific situations (such as singing, communication, performance, etc.), analyzes their voice intonation, body language, expression changes and other non-verbal information, and extracts features related to emotional expression and psychological state. The collected various psychological assessment data are sorted and cleaned to remove invalid data, duplicate data and erroneous data to ensure the accuracy and completeness of the data. At the same time, the data from different sources are formatted and standardized for subsequent analysis and processing. For example, converting text responses in a questionnaire into numerical codes and converting audio / video data into analyzable feature vectors.

[0094] For the technical implementation of the encoding module, a variety of encoding algorithms are used to encode the psychological assessment data, such as principal component analysis (PCA) and autoencoder. PCA projects the original high-dimensional data into a low-dimensional space through linear transformation, reduces the data dimension while retaining the main variance of the data, and obtains a hidden feature vector; Autoencoder is a neural network architecture that encodes the input data into a low-dimensional representation (hidden layer output) through learning, and then decodes it back to the original data. During the training process, the encoding and decoding process is continuously optimized to minimize the reconstruction error, thereby obtaining a representative hidden feature vector. When using PCA, the covariance matrix of the psychological assessment data is calculated, and its eigenvalues ​​and eigenvectors are calculated. The main eigenvectors are selected according to the size of the eigenvalues ​​to form a projection matrix, and the original data is projected into a low-dimensional space to obtain a hidden feature vector. For Autoencoder, a neural network structure including an input layer, a hidden layer, and an output layer is designed. The number of neurons in the input layer is the same as the dimension of the psychological assessment data, the number of neurons in the hidden layer is set according to the desired encoding dimension, and the number of neurons in the output layer is the same as that in the input layer. The network is trained through the back-propagation algorithm to make the output as close to the input as possible, and finally the hidden feature vector is obtained.

[0095] For the feature head module, for the personality classification feature head, a softmax function or a multi-layer perceptron (MLP) is used for nonlinear mapping. If the softmax function is used, the hidden feature vector is used as input, and the probability distribution of the user belonging to different personality types is calculated through the softmax function. For example, the personality types are divided into extrovert, introvert, intermediate, etc., and a probability vector is output to indicate the probability that the user belongs to each category. If MLP is used, a neural network structure is constructed that includes an input layer (receiving hidden feature vectors), one or more hidden layers (using nonlinear activation functions such as ReLU), and an output layer (outputting personality classification results). The network parameters are adjusted through training so that the output accurately reflects the user's personality classification. The emotional expression description feature head can use a recurrent neural network (RNN) or a long short-term memory network (LSTM) to encode the hidden feature vector for cyclic dependency. The hidden feature vector is input into the RNN / LSTM network in time steps. The network updates the hidden state at each time step, learns the temporal dependency in the feature vector, and finally outputs a vector describing the user's emotional expression tendency. This vector can represent the characteristics of the user in terms of the intensity, stability, and diversity of emotional expression, such as the richness of emotional expression and emotional fluctuations. The psychological development stage feature head uses a convolutional neural network (CNN) or a deep belief network (DBN) to encode the hidden feature vector layer by layer. For CNN, a suitable convolution kernel size and step size are designed to perform convolution operations on the hidden feature vector, extract features at different levels, downsample through the pooling layer to reduce the data dimension, and finally output the user's psychological development stage label through the fully connected layer, such as childhood, adolescence, and adulthood; DBN builds a multi-layer restricted Boltzmann machine (RBM) and performs unsupervised pre-training layer by layer to learn the hierarchical feature representation of the data. Finally, a classification layer is added on the top to classify and predict the user's psychological development stage.

[0096] For the context module, the context module collects context information related to the user's vocal learning, such as learning history, practice environment, recent performance experience, etc. The learning history includes the repertoire that the user has learned in the past, the skills mastered, the progress made, etc.; the practice environment covers the quietness of the practice venue, the equipment conditions, etc.; the recent performance experience records the user's performance, audience feedback, self-evaluation, etc. These context information are quantified, for example, the learning history is converted into sequence data of the difficulty of the repertoire and the learning time, the practice environment is converted into the numerical characteristics of environmental factors (such as noise level, space size, etc.), and the recent performance experience is converted into quantitative indicators of performance evaluation (such as score, satisfaction, etc.). The context information is fused with the feature representation related to personality classification, emotional expression, and psychological development stage output by the feature head module. The context feature vector and the psychological feature vector can be combined by splicing, weighted summation, etc. to form a fused feature vector. For example, the context feature vector and the personality classification feature vector are spliced ​​in dimension to obtain a new vector as input, which is sent to the decision prediction model for processing.

[0097] For decision prediction model construction and training, a decision prediction model based on machine learning algorithms is constructed, such as support vector machine (SVM), random forest (RF), neural network, etc. Taking neural network as an example, a neural network structure is designed, which includes an input layer (receiving fused feature vectors), a hidden layer (using appropriate activation functions, such as ReLU, tanh, etc.) and an output layer (outputting prediction results of personality classification, emotional expression description, and psychological development stage identification). The decision prediction model is trained using labeled training data. The training data should contain psychological assessment data, corresponding contextual information, and accurate personality classification, emotional expression description, and psychological development stage identification labels. By optimizing the loss function (such as cross entropy loss function, mean square error loss function, etc.), the model parameters are adjusted using gradient descent or its variant algorithms (such as Adagrad, Adadelta, Adam, etc.), so that the model can accurately predict the user's psychological parameters based on the input fused feature vector. During the training process, cross-validation and other techniques can be used to evaluate the model performance, prevent overfitting, improve the generalization ability of the model, and ensure that the model can accurately determine the target user's psychological parameters in practical applications.

[0098] Optionally, when the feature head module performs nonlinear mapping on the hidden feature vector to obtain feature representations related to personality classification, emotional expression, and psychological development stage, the following steps are performed:

[0099] The personality classification feature head in the feature head module performs a first nonlinear mapping on the hidden feature vector to map the hidden feature vector to a personality type probability distribution space to obtain probabilities indicating that the user belongs to different personality types;

[0100] The emotion expression description feature head in the feature head module performs a second nonlinear mapping on the hidden feature vector to perform cyclic dependency encoding on the hidden feature vector to obtain a vector describing the user's emotion expression tendency;

[0101] The psychological development stage feature head in the feature head module performs a third nonlinear mapping on the hidden feature vector to perform layer-by-layer feature encoding on the hidden feature vector to obtain a label of the user's psychological development stage.

[0102] To this end, the existence of the personality classification feature head maps the hidden feature vector to the probability distribution space of personality type, which can provide a quantitative and detailed personality assessment method. Traditional personality classification is often more general, while this method can obtain the probability of users belonging to different personality types, for example, there is a 30% probability that the user belongs to an extroverted personality, and a 70% probability that the user belongs to an intermediate personality. This makes the understanding of the user's personality more comprehensive and accurate, avoids the limitations brought by simple binary or limited classification, and provides a more accurate personality basis for personalized vocal teaching. In teaching, it is more appropriate to encourage extroverted users to show themselves on the stage, while for introverted users, we can first focus on cultivating the delicate expression of their inner emotions. According to the probability distribution, more targeted and flexible teaching strategies can be formulated. People's personalities are usually complex and diverse, and cannot be completely summarized by a single type. Through the probability distribution method, we can better capture the various tendencies and mixed characteristics in the user's personality, so as to better adapt to this complexity. For example, some users exhibit extroverted characteristics in certain situations, but introverted traits in other situations. This versatility can be clearly presented through probability distribution, helping teachers or teaching systems to more deeply understand users' behavioral patterns and psychological needs in the process of vocal learning and performance, and thus provide guidance and training programs that are more in line with the users' actual conditions.

[0103] The existence of the emotional expression description feature head, and the cyclic dependency encoding of the hidden feature vector to obtain the vector describing the user's emotional expression tendency, can effectively capture the dynamic changes and timing information in the user's emotional expression. In vocal singing, the expression of emotion is often not static, but changes with the development of the melody, rhythm and lyrics of the song. Circular dependency encoding can learn the pattern and law of such changes, such as the intensity and method of the user's emotional expression in the climax of the song, and the ups and downs of emotions in the transition paragraphs. This enables the system to better understand the user's ability and characteristics in emotional expression, thereby providing strong support for selecting tracks suitable for the user's emotional expression style and providing targeted emotional expression training, helping users to convey the emotional connotation of the song more vividly and accurately. Through cyclic dependency encoding, not only can the surface changes of emotional expression be captured, but also the depth and details of emotional expression can be deeply explored. For example, when a user expresses sad emotions, does he only stay on the surface of sad imitation, or can he go deep into the sad emotions in his heart and express them delicately? These deep emotional features can be reflected through the encoded vector. This is crucial for vocal teaching, because vocal art is not only a display of sound skills, but also a deep transmission of emotions. Understanding the depth and details of users' emotional expression can help teachers guide users to further improve the quality of emotional expression and make singing more contagious.

[0104] Finally, determining the user's psychological development stage label can achieve accurate matching of vocal teaching with the user's psychological maturity. Users at different psychological development stages have differences in learning ability, cognitive level, self-awareness, etc., and their acceptance and needs for vocal learning are also different. For example, for users in childhood, the teaching method focuses more on fun and intuitiveness to stimulate their interest in vocal music; for adolescent users, the depth requirements of skill training and emotional expression can be gradually increased; adult users pay more attention to the connotation understanding of vocal art and the shaping of personal style. By accurately identifying the psychological development stage, the teaching system can provide users with the most suitable teaching content and methods for their current stage, improve teaching effects and user satisfaction. Human psychological development is a dynamic process that changes with the accumulation of time and experience. The feature head can perform layer-by-layer feature encoding based on the user's current hidden feature vector to obtain the psychological development stage label, which means that the system can track the changes in the user's psychological development in real time and adjust the teaching strategy in time. If the user's psychological state changes during the learning process, such as the change in psychological characteristics from adolescence to adulthood, the system can keenly capture this change and provide the user with corresponding upgraded or adjusted teaching plans to ensure that the teaching is always in line with the user's psychological development and promote the user's continued healthy growth on the path of vocal learning.

[0105] In the specific implementation, for the personality classification feature head, in order to map the hidden feature vector to the personality type probability distribution space, the softmax function is usually used as the first nonlinear mapping function. The formula of the softmax function is: Where \(z_i\) is the \(i\)th element of the hidden feature vector after linear transformation (such as multiplication with the weight matrix and adding the bias term), \(n\) is the number of personality types, and \(P(y_i)\) represents the probability that the user belongs to the \(i\)th personality type. In the implementation process, it is necessary to first determine the classification system of personality types. For example, the common MBTI personality classification system includes 16 personality types, or it can be simplified to several main types such as extrovert, introvert, and intermediate. According to the selected classification system, the corresponding weight matrix \(W\) and bias vector \(b\) are designed. By calculating \(z=Wx+b\) (where \(x\) is the hidden feature vector), the \(z\) value of the input softmax function is obtained, thereby obtaining the probability distribution of users belonging to different personality types.

[0106] For model training and optimization, a large amount of sample data with personality type annotations (including psychological assessment data and its corresponding accurate personality type) is collected to train the personality classification feature head model. The cross entropy loss function is used to measure the difference between the predicted probability distribution and the actual personality type. The cross entropy loss function formula is: Where \(y_i\) is the one-hot encoding of the true personality type (if the user belongs to the \(i\)th personality type, then \(y_i=1\), otherwise \(y_i=0\)).

[0107] The gradient descent algorithm or its variants (such as Adagrad, Adadelta, Adam, etc.) is used to optimize the model parameters (weight matrix \(W\) and bias vector \(b\)). During the training process, according to the gradient of the loss function to the parameters, the parameters are updated according to the selected optimization algorithm rules, and the loss value is continuously reduced, so that the model can more accurately predict the probability distribution of the user's personality type. At the same time, regularization techniques (such as L1 regularization or L2 regularization) can be used to prevent the model from overfitting and improve the generalization ability of the model.

[0108] For the emotion expression description feature head, a recurrent neural network architecture is selected to implement the cyclic dependency encoding of the hidden feature vector. Common choices include long short-term memory network (LSTM) or gated recurrent unit (GRU). Taking LSTM as an example, it better handles long-term dependencies in long sequence data by introducing memory units and gating mechanisms to avoid gradient vanishing or explosion problems. Construct an LSTM network structure, including input layer, hidden layer and output layer. The input layer receives the hidden feature vector, the hidden layer contains multiple LSTM units, and the output layer is designed according to the requirements. For example, the output layer can be designed as a fully connected layer, the number of neurons of which matches the dimension of the emotion expression tendency vector, and is used to output a vector describing the user's emotion expression tendency.

[0109] For the cyclic dependency encoding process, the hidden feature vector is input into the LSTM network sequentially in time steps. At each time step, the LSTM unit performs calculations based on the current input, the hidden state and the cell state at the previous moment. Through operations such as the forget gate, input gate, cell state update, and output gate, the hidden state and cell state are updated to learn the temporal dependencies in the hidden feature vector. After processing the entire hidden feature vector sequence, the hidden state of the last time step of the LSTM network will be output as a vector describing the user's tendency to express emotion. This vector can contain characteristics of the user in terms of the intensity, stability, and diversity of emotional expression. For example, some elements in the vector can represent the intensity of the user's expression of positive emotions, while other elements can reflect the fluctuation of emotional expression over time, that is, emotional stability.

[0110] For model training and parameter adjustment, it is also necessary to collect sample data with emotional expression description annotations to train the emotional expression description feature head model. Annotations can be obtained through user self-assessment, expert evaluation, or performance evaluation based on specific emotional expression tasks. A loss function is used to measure the difference between the model output and the true emotional expression description, such as the mean square error (MSE) loss function. The gradient of the loss function to the model parameters (weights and biases of LSTM units, etc.) is calculated through the back-propagation algorithm, and the parameters are updated using an optimization algorithm (such as the Adam optimizer), and the model is continuously adjusted to improve the prediction accuracy of the emotional expression tendency vector. During the training process, the hyperparameters of the model (such as the number of hidden layers, the number of LSTM units, the learning rate, etc.) need to be adjusted and optimized, and the best hyperparameter combination is selected through cross-validation and other techniques to ensure that the model has good performance.

[0111] For the psychological development stage feature head, methods such as convolutional neural network (CNN) or deep belief network (DBN) are used to realize layer-by-layer feature encoding of hidden feature vectors to obtain psychological development stage labels. Taking CNN as an example, it automatically extracts the feature hierarchy in the data through a combination of convolutional layers, pooling layers and fully connected layers. Design the CNN architecture and determine parameters such as convolution kernel size, step size, and padding method. For example, select a convolution kernel size of 3x3 or 5x5, a step size of 1 or 2, and determine the appropriate padding method according to the dimension of the hidden feature vector to maintain the consistency of the feature map size. The convolution layer is used to extract local features, and the pooling layer (such as maximum pooling or average pooling) is used to reduce the data dimension, reduce the amount of calculation and retain the main features. The hidden feature vector is input into the CNN, and after being processed by multiple convolutional layers and pooling layers, more abstract and advanced feature representations are gradually extracted. Finally, the extracted features are mapped to the psychological development stage label space through the fully connected layer. The number of neurons in the fully connected layer is set according to the number of categories of the psychological development stage. For example, if the psychological development stage is divided into childhood, adolescence, and adulthood, the fully connected layer can have 3 neurons. The softmax activation function is used to obtain the probability distribution of the user belonging to each stage, and the label with the highest probability is taken as the final predicted psychological development stage label.

[0112] For model training and evaluation, prepare sample data with psychological development stage annotations, build a model according to the above CNN architecture and train it. Use the cross entropy loss function to evaluate the difference between the predicted label and the true label, calculate the gradient through the back-propagation algorithm, and use the optimization algorithm (such as stochastic gradient descent or its variants) to update the model parameters (convolution kernel weights, biases, etc.). During the training process, use evaluation indicators (such as accuracy, recall, F1 value, etc.) to evaluate the model performance. You can use the validation set data to regularly verify the model during the training process, and adjust the model hyperparameters (such as the number of convolutional layers, learning rate, regularization parameters, etc.) according to the evaluation indicators to prevent overfitting, improve the generalization ability of the model, and ensure that the model can accurately classify and predict the user's psychological development stage.

[0113] Optionally, when determining the physical parameters of the target user, the physical parameter determination module obtains physical monitoring data generated by a smart wearable device, and performs feature extraction on the physical monitoring data to obtain the physical parameters.

[0114] Optionally, the physical parameter determination module performs the following steps when determining the physical parameter of the target user:

[0115] Identifying data dimensions of the physical fitness monitoring data, and calculating covariance matrices of different identified dimensions;

[0116] Performing eigendecomposition on the covariance matrix to obtain eigenvalues ​​and corresponding eigenvectors;

[0117] According to the size of the eigenvalue, the eigenvectors corresponding to the first k largest eigenvalues ​​are selected as principal components;

[0118] Projecting the physical fitness monitoring data onto the principal component to obtain a principal component score matrix;

[0119] An exploratory analysis is performed on the principal component score matrix to perform a linear regression on the principal component loading matrix to obtain predicted physical parameters.

[0120] To this end, by calculating the covariance matrix and performing eigendecomposition in data dimension reduction and feature extraction, the eigenvectors corresponding to the first k largest eigenvalues ​​are selected as principal components, which can effectively extract the key information from the original physical fitness monitoring data. The data collected by smart wearable devices are often of high dimension and contain a lot of redundant information or noise. In this way, the data dimension can be reduced, focusing on the main components that best reflect the user's physical characteristics, avoiding the interference of information overload on subsequent analysis, and making the assessment of the user's physical fitness more accurate and efficient. For example, among the many sports-related data dimensions, the principal components that are highly correlated with key physical factors such as physical fitness and endurance are screened out, highlighting the role of these important features in the assessment. Moreover, high-dimensional data consumes a lot of resources and time during calculation and processing. Principal component analysis (PCA) is used to reduce data dimension, and the physical fitness monitoring data is projected onto the principal components to obtain the principal component score matrix, which greatly reduces the amount of data and significantly reduces the complexity of subsequent calculations. Whether storing data or performing analytical calculations, efficiency can be improved, allowing the system to process the user's physical data more quickly, provide relevant basis for vocal teaching in a timely manner, and improve the response speed and practicality of the entire teaching system.

[0121] In addition, exploratory analysis and linear regression, exploratory analysis of the principal component score matrix can deeply explore the structure and potential relationships within the data. By visualizing the principal component score matrix (such as drawing scatter plots, heat maps, etc.), the distribution of different samples (users) in the principal component space can be intuitively observed, and clustering patterns, outliers and other information in the data can be found. This helps to better understand the overall characteristics of the user's physical data, such as whether there are groups with different physical types, or whether the physical data of some users deviate from the normal range, etc., providing more valuable insights for further analysis and personalized teaching. Furthermore, in the accurate prediction of physical parameters, the results of exploratory analysis are used to perform linear regression on the principal component loading matrix, which can establish a quantitative relationship model between the principal components and the actual physical parameters. Through the known training data (including physical monitoring data and the corresponding accurate physical parameter measurements), the coefficients of the linear regression model are determined, so that the user's physical parameters can be accurately predicted according to the principal component score matrix. This data-driven prediction method is more scientific and accurate than traditional empirical judgment or simple statistical methods. It can provide a reliable quantitative basis for formulating personalized teaching plans according to the user's physical conditions in vocal teaching. For example, it can determine the appropriate practice intensity and duration based on the predicted physical fitness and endurance levels to avoid learning difficulties or excessive fatigue due to physical intolerance.

[0122] When implementing data dimension identification and covariance matrix calculation, the physical monitoring data collected by smart wearable devices contains multiple types of information, such as acceleration (including X, Y, and Z axis directions), heart rate, body temperature, exercise steps, sleep data (sleep duration, sleep stage, etc.). First, determine the dimensions by analyzing the format and structure of the data. For example, acceleration data has three-dimensional values ​​at each time point, and heart rate data is a one-dimensional time series. These different types of data are considered as different dimensions. For some complex data structures, specific algorithms or tools are used to parse and identify dimensions. For example, for situations involving the fusion of multiple sensor data, different dimensions can be distinguished based on the sensor identification and data recording method. At the same time, check the integrity of the data to ensure that the data of each dimension exists and is valid. If missing or outliers are found, appropriate data cleaning and preprocessing are required, such as using mean filling, interpolation, and other methods to handle missing values, and identifying and correcting outliers by setting reasonable thresholds.

[0123] In the process of calculating the covariance matrix, the physical fitness monitoring data is assumed to be a matrix \(X\), where each row represents a time point or a data sample, and each column represents a dimension of data. The calculation method of the covariance matrix \(C\) is Where \(n\) is the number of samples. In the calculation process, the data is first centered, that is, the mean of each dimension is subtracted from the data of that dimension. This step can eliminate the translation effect of the data, so that the covariance matrix can more accurately reflect the relationship between the data dimensions. Then, according to the rules of matrix multiplication, X is calculated. T X, and finally divided by \(n-1\) to get the covariance matrix \(C\). The element C in the covariance matrix \(C\) ij It represents the covariance between the \(i\)th dimension and the \(j\)th dimension, which reflects the correlation between the changes in the data of these two dimensions. For example, if the covariance between the heart rate dimension and the exercise intensity dimension (calculated by acceleration and other data) is large and positive, it means that the heart rate and exercise intensity have a strong positive correlation in the change trend, that is, when the exercise intensity increases, the heart rate usually increases.

[0124] During eigendecomposition and principal component selection, the calculated covariance matrix \(C\) is subjected to eigendecomposition, usually using numerical calculation methods such as the Jacobi method or the QR decomposition algorithm. Taking the Jacobi method as an example, it gradually transforms the covariance matrix into a diagonal matrix through a series of rotation transformations. The elements on the diagonal are the eigenvalues, and the column vectors of the corresponding rotation transformation matrix are the eigenvectors. In actual calculations, since the covariance matrix is ​​usually a symmetric positive definite matrix, these algorithms can effectively calculate its eigenvalues ​​and eigenvectors. The eigenvalue represents the variance corresponding to each eigenvector, and the eigenvector determines the direction of the data in the new feature space. For example, the eigenvector corresponding to a larger eigenvalue represents the direction in which the data changes more, that is, the direction in which the main information in the data is located.

[0125] In the principal component selection strategy, the eigenvectors corresponding to the first \(k\) largest eigenvalues ​​are selected as principal components according to the size of the eigenvalues. There are many ways to select \(k\). A common method is to determine it by cumulative variance contribution rate. Calculate the proportion of each eigenvalue to the sum of all eigenvalues, and then add these proportions from large to small. When the cumulative proportion reaches a certain threshold (such as 85% or 90%), the number of corresponding eigenvalues ​​\(k\) is the number of principal components selected. Another method is to manually select \(k\) based on the needs of the actual problem and the understanding of the data. For example, if it is known that in vocal learning, physical fitness and endurance are two main aspects closely related to physical fitness, and through analysis, it is found that the principal components corresponding to the first two eigenvalues ​​have a strong correlation with physical fitness and endurance indicators, then \(k=2\) can be selected. Selecting the appropriate principal component can minimize the data dimension while retaining key information, and improve the efficiency of subsequent analysis and processing.

[0126] In the data projection and exploratory analysis phase, the original physical fitness monitoring data \(X\) is projected onto the selected principal components to obtain the principal component score matrix \(P\). The projection calculation method is P=XW, where \(W\) is a matrix composed of the selected principal components (eigenvectors). Specifically, for each sample (row vector) in \(X\), the coordinates (scores) of the sample in the principal component space are obtained by multiplying it with \(W\), thereby converting the high-dimensional original data into a low-dimensional principal component score matrix. For example, if the original data has \(m\) dimensions and \(k\) principal components are selected, then each sample in \(P\) will change from \(m\) dimensions to \(k\) dimensions. This process achieves data dimensionality reduction while retaining the main information in the data, making subsequent analysis easier and more effective.

[0127] In the exploratory analysis phase, visualizing the principal component score matrix \(P\) is an important exploratory analysis method. A scatter plot of the principal component scores can be drawn to visualize the position of each sample (user) in the principal component space. For example, with the first principal component score as the horizontal coordinate and the second principal component score as the vertical coordinate, a scatter plot is drawn to observe the distribution of the sample points. If it is found that the sample points show a certain clustering trend, it means that there are user groups with different physical types; if there are outliers, it means that the user's physical data is quite different from that of other users, and further attention and analysis are needed. In addition to the scatter plot, the correlation coefficient between the principal component scores can also be calculated to analyze the linear relationship between different principal components. In addition, the principal component scores are statistically described, such as calculating statistical quantities such as the mean, standard deviation, skewness, and kurtosis, to understand the distribution characteristics of the principal component scores, to determine whether the data conforms to assumptions such as normal distribution, and to provide basic information for subsequent linear regression analysis.

[0128] In the linear regression and physical parameter prediction link, in the construction of the linear regression model, the predicted physical parameter is set as \(y\) (such as physical fitness index, endurance level, etc.), the principal component score matrix \(P\) is the independent variable, and the linear regression model y=β0+β1p1+β2p2+…+β k p k +∈, where \(\beta_0,\beta_1,\cdots,\beta_k\) are regression coefficients, \(p_1,p_2,\cdots,p_k\) are the column vectors of principal component scores in \(P\), and \(\epsilon\) is the error term.

[0129] The regression coefficient \(\beta_i\) is estimated through known training data (including physical fitness monitoring data and its corresponding accurate physical fitness parameter measurement values). The least squares method can be used to solve it, that is, to minimize the sum of squares of the error between the predicted value \(y\) and the actual measurement value Where \(y_i\) is the actual measured value, \(\hat{y}_i\) is the predicted value. The solution formula of the least squares method is β=(P T P) -1 P T y, where \(\beta\) is the regression coefficient vector.

[0130] In the process of physical fitness parameter prediction, after obtaining the regression coefficient \(\beta\), for the new user's physical fitness monitoring data, first calculate the principal component score matrix \(P\) according to the previous steps, and then substitute \(P\) into the linear regression model to calculate the predicted physical fitness parameter \(y\). For example, if you want to predict a user's physical fitness index, the user's physical fitness monitoring data is dimensionally identified, the covariance matrix is ​​calculated, the feature decomposition, the principal component selection and data projection are performed to obtain the principal component score, and then multiplied by the regression coefficient and the intercept term \(\beta_0\) is added to obtain the predicted physical fitness index value. At the same time, in order to evaluate the accuracy of the prediction, some evaluation indicators, such as mean square error (MSE), mean absolute error (MAE), determination coefficient (\(R^2\)), etc., can be used to evaluate and optimize the performance of the model on the validation set or test set, and continuously improve the accuracy of physical fitness parameter prediction.

[0131] Optionally, when the vector fusion unit fuses the physiological description vector, the psychological description vector, and the physical description vector of the target user to obtain a user feature description vector, the following steps are performed:

[0132] Performing multi-scale alignment on the physiological description vector, the psychological description vector, and the physical description vector of the target user;

[0133] Construct a fusion matrix based on the aligned physiological description vector, psychological description vector, and physical description vector of the target user;

[0134] Perform singular value decomposition on the constructed fusion matrix to obtain the transpose of the left singular vector matrix and the right singular vector matrix;

[0135] The singular value and its corresponding left and right singular vectors are concatenated to obtain a user feature description vector.

[0136] To this end, the above multi-scale alignment can integrate multi-source heterogeneous data. The physiological description vector, psychological description vector and physical description vector come from different data collection and processing methods, and have different scales and feature representations. Multi-scale alignment can process these heterogeneous data under a unified framework, so that they can achieve a certain degree of consistency in the feature space. For example, physiological data may be presented in the form of numerical values ​​of physiological indicators, psychological data may be represented by probabilities or features after encoding and mapping, and physical data also has its specific vector form after processing such as principal component analysis. Through multi-scale alignment, the corresponding relationship and common features between them can be found, and the scattered information can be integrated to lay the foundation for a comprehensive and accurate description of user characteristics, avoiding information conflicts or mismatches caused by differences in data sources and forms. Aligning vectors at different scales helps to explore potential correlation information between physiological, psychological and physical characteristics. For example, in vocal learning, it may be found that there is a subtle connection between a specific physiological structure (such as the state of the vocal cords) and a certain personality type (such as extroverts tend to make full use of the tension of the vocal cords when speaking) and physical conditions (such as users with better physical fitness have more stable breath support when speaking, which is associated with physiological characteristics related to breathing efficiency). Multi-scale alignment can highlight these association patterns hidden in the data, providing a new perspective for in-depth understanding of the comprehensive performance of users in vocal learning, thereby providing a richer and more targeted basis for personalized teaching.

[0137] In addition, the fusion matrix and singular value decomposition are constructed. The fusion matrix integrates the aligned physiological, psychological and physical description vectors into one matrix, which can consider the relationship between multiple feature dimensions as a whole. Singular value decomposition (SVD) is a powerful matrix decomposition technology, which can decompose the fusion matrix into the transpose of the left singular vector matrix, the singular value matrix and the right singular vector matrix. The singular value reflects the importance of each eigenvector in the matrix. The user feature description vector is obtained by splicing the singular value and its corresponding left and right singular vectors, which can comprehensively consider the information of all feature dimensions and automatically assign different weights to different features according to the size of the singular value. The user feature description vector obtained in this way can more comprehensively and accurately characterize the comprehensive characteristics of the user in vocal learning, which not only contains the original information of each dimension, but also reflects the relationship and importance difference between them.

[0138] In dimensionality reduction and feature extraction, SVD has a dimensionality reduction effect to a certain extent. By selecting singular vectors corresponding to larger singular values ​​for splicing, the dimension of the vector can be reduced while retaining the main information, reducing data redundancy. This helps to improve the efficiency of subsequent operations such as matching vocal functional repertoires based on user feature description vectors, while avoiding the computational complexity and overfitting problems caused by high-dimensional data. In addition, the singular value decomposition process can also be regarded as a feature extraction method, which can extract the most representative and influential feature components from the original complex feature combination, making the user feature description more concise and effective, and highlighting the impact of key features on the user's vocal learning ability and needs.

[0139] Furthermore, in the user feature description vector obtained by splicing, the singular values ​​and their corresponding left and right singular vectors are spliced ​​to form a user feature description vector, realizing the organic integration of multi-dimensional information of physiology, psychology and physique. This vector is no longer a simple list of features of each dimension, but combines features of different dimensions in a meaningful way through singular value decomposition and splicing operations. For example, it can integrate physiological characteristics closely related to the physiological mechanism of vocalization, psychological characteristics that affect emotional expression and learning attitude, and physical characteristics that determine physical fitness and endurance levels to support vocalization into a unified description, providing a comprehensive indicator for comprehensively evaluating the user's potential, advantages and limitations in vocal learning. Based on this user feature description vector, the system can more accurately match users with suitable vocal functional repertoires, formulate personalized teaching strategies, and provide targeted skill improvement suggestions, thereby improving the quality and effectiveness of vocal teaching.

[0140] The spliced ​​user feature description vector has a unified format and structure, which is convenient for various processing and analysis operations in the subsequent vocal teaching system. Whether it is matched with the repertoire features in the vocal knowledge base, or used as input for other machine learning models (such as models for evaluating user learning progress and predicting learning effects), data interaction and calculation can be conveniently performed. This unified vector representation improves the compatibility and scalability of the entire vocal intelligent teaching system, allowing the system to more flexibly respond to different teaching needs and functional expansions, and provide users with a more comprehensive and personalized vocal learning experience.

[0141] Optionally, a vocal music knowledge base creation module is configured on the background server, and the vocal music knowledge base creation module is used to execute the following steps to construct the vocal music knowledge base:

[0142] Get vocal functional repertoire;

[0143] Extracting features of the functional vocal music repertoire to obtain a repertoire feature description vector;

[0144] A mapping relationship between the vocal functional repertoire and the repertoire feature description vector is established to construct the vocal knowledge base.

[0145] Optionally, when the vocal knowledge base creation module extracts the vocal functional repertoire features to obtain the repertoire feature description vector, the vocal knowledge base creation module performs the following steps:

[0146] Constructing a vocal functional repertoire set, wherein the vocal functional repertoire set includes a plurality of vocal functional repertoires;

[0147] For each vocal functional track, audio feature extraction, emotional expression feature classification, and adaptive physical classification are performed;

[0148] Based on the audio feature extraction, emotion expression feature classification, and adaptive constitution classification, vocal feature metadata is generated;

[0149] The vocal feature metadata is vectorized to obtain the track feature description vector.

[0150] To this end, by constructing a vocal knowledge base system, the system can accurately match the characteristics of the tracks in the knowledge base according to the unique characteristics of the user (such as physiology, psychology, and physique). This helps to provide each user with the most suitable vocal functional tracks for their current status and learning goals, and realize personalized teaching. For example, for users with weak physical fitness but delicate emotional expression, the system can filter out tracks with a relatively slow rhythm, rich emotional expression, and low physical fitness requirements from the knowledge base, thereby improving learning efficiency, enhancing the interest and pertinence of learning, and meeting the diverse needs of different users in vocal learning. By integrating many vocal functional tracks into the knowledge base and systematically extracting and classifying their features, teaching resources are effectively organized and managed. Teachers and students can more easily find suitable track resources quickly according to different teaching purposes and learning needs, avoiding the waste of time in blindly selecting from a large number of tracks, improving the utilization efficiency of teaching resources, and promoting the orderly development of vocal teaching activities.

[0151] In addition, in the step of extracting track features, audio feature extraction, emotional expression feature classification and adaptive physical fitness classification are performed on each vocal functional track, which can comprehensively characterize the characteristics of the track from multiple dimensions. Audio features (such as rhythm, pitch, timbre, etc.) reflect information about the musical structure and singing difficulty of the track, which is of great significance for evaluating the user's singing skills and ability matching; emotional expression feature classification helps to understand the emotional connotation conveyed by the track, so as to match it with the user's emotional expression ability and needs, so that students can better interpret the emotions of the song in singing; adaptive physical fitness classification takes into account the requirements of physical fitness, endurance and other physical conditions for the singing of the track, ensuring that the recommended track is within the user's physical tolerance range, avoiding excessive fatigue or injury to the user due to the high difficulty of the track, and at the same time giving full play to the user's physical advantages.

[0152] In providing precise matching, vocal feature metadata is generated based on these multi-dimensional feature extractions, and further vectorized to obtain the track feature description vector, providing a reliable basis for the precise matching between the track and the user features. The track feature description in vector form makes the matching calculation more convenient and efficient. Various distance measurement algorithms (such as cosine similarity, Euclidean distance, etc.) can be used to measure the similarity between the user feature description vector and the track feature description vector, so as to quickly and accurately find the track that best suits the user. This precise matching not only improves the teaching effect, but also stimulates the user's interest and enthusiasm in learning, because they can sing tracks that match their own abilities and characteristics, and are more likely to gain a sense of accomplishment.

[0153] In addition, in the establishment of a mapping relationship, it is convenient for rapid retrieval and recommendation, and a mapping relationship is established between vocal functional tracks and track feature description vectors, so that suitable tracks can be quickly retrieved according to user characteristics during the teaching process. When the system obtains the user's feature description vector, it can quickly locate tracks with a high degree of match by searching the mapping relationship, and realize real-time track recommendations. This efficient retrieval mechanism improves the response speed of the teaching system, provides users with instant learning resource support, helps maintain the continuity and fluency of learning, avoids the sense of interruption of learning caused by users while waiting for track recommendations, and thus improves the user's learning experience.

[0154] When specifically acquiring functional vocal repertoires, in addition to conventional music databases and education platforms, functional vocal repertoires can also be obtained from vocal competition repertoire libraries, vocal teaching demonstration albums, folk music collections with regional characteristics and other channels to further enrich the diversity of repertoires. When screening repertoires, not only their functionality in vocal skill training is considered, but also their cultural connotations and artistic value. For example, for the teaching goal of cultivating students' cultural literacy, classic repertoires with a profound cultural heritage are given priority. In addition, you can also cooperate with industry experts such as vocal teachers and professional singers to obtain high-quality functional repertoires recommended by them. With their rich teaching and performance experience, these experts can recommend some repertoires that have been proven to have good training effects in actual teaching and singing. At the same time, they can also provide valuable suggestions on the difficulty grading of the repertoire, applicable teaching scenarios, etc., which will help to build a more accurate repertoire collection.

[0155] Furthermore, a complete copyright management system should be established to record in detail the copyright source, authorization scope, and usage period of each acquired track. For some tracks that require payment, reasonable authorization fees and usage methods should be negotiated with the copyright holder to ensure that the track resources are used legally and in compliance with regulations. In terms of data integration, tracks acquired from different sources will be uniformly numbered and classified. Indexes will be established based on multiple dimensions such as track style, singing difficulty, and emotional type to facilitate subsequent retrieval and calls. At the same time, metadata related to the tracks (such as lyrics, creative background, singing version information, etc.) will be sorted and standardized to ensure the integrity and accuracy of the metadata, providing comprehensive support for track feature extraction and teaching applications.

[0156] For audio feature extraction, an adaptive audio frame segmentation method is used to dynamically adjust the frame length according to the rhythm changes of the track. For tracks with faster rhythms, the frame length is appropriately shortened to more accurately capture the rapidly changing audio signals; for tracks with slower rhythms, the frame length can be appropriately extended to reduce the amount of data while ensuring the accuracy of feature extraction. For example, by analyzing the rhythm estimation value of the initial part of the track, the appropriate frame length range is determined, and then fine-tuned according to the actual changes in the rhythm in subsequent processing. When segmenting audio frames, the energy distribution of the audio signal is considered to avoid unreasonable truncation at energy mutation points (such as the start and end of notes). An energy threshold-based segmentation method can be used. When the energy of the audio signal is lower than a certain threshold and lasts for a period of time, the frame is divided. This ensures that the audio signal in each audio frame has relative stability and integrity, which is conducive to the accuracy of subsequent feature extraction.

[0157] For time-frequency analysis optimization, the appropriate time-frequency resolution parameters are selected when performing short-time Fourier transform (STFT). According to the characteristics of the audio signal of the track, the optimal values ​​of window length and frame shift are determined by experiments or analysis of the spectral characteristics of the audio signal. For example, for tracks containing rich high-frequency components, a shorter window length is selected to improve the high-frequency resolution; for tracks with large rhythm changes, the frame shift is adjusted appropriately to ensure that the rhythm changes can be accurately tracked in time. In addition to the Mel spectrum, other time-frequency domain features can also be calculated, such as the spectral centroid, spectral bandwidth, zero-crossing rate, etc., to describe the characteristics of the audio signal from different angles. The spectral centroid reflects the distribution center of the audio signal energy on the frequency axis, which can be used to determine the approximate range of the pitch; the spectral bandwidth reflects the distribution width of the frequency components of the audio signal, which is related to the richness of the timbre; the zero-crossing rate can reflect the speed of the frequency change of the audio signal, which is helpful for rhythm analysis. Combining these features with the Mel spectrum features can more comprehensively characterize the audio characteristics of the track.

[0158] For feature compression and selection, after logarithmically compressing the Mel spectrum, methods such as principal component analysis (PCA) are used to further compress and select features. PCA can project high-dimensional spectral features into low-dimensional space, reducing data dimensions while retaining the main information. By calculating the eigenvalue and contribution rate, the principal component with a larger contribution rate is selected as the final audio feature, which can not only reduce data storage and calculation, but also highlight the key feature information in the audio signal, improving the efficiency and effectiveness of feature extraction.

[0159] In the classification of emotional expression features, when extracting multimodal emotional features, in addition to the classification of emotional expression features based on audio features, emotional features can also be extracted by combining lyrics text analysis. Use natural language processing technology to perform lexical, syntactic and semantic analysis on lyrics. For example, by identifying emotional keywords (such as "love", "sadness", "happiness", etc.), emotional phrase structures (such as "tears streaming down the face" and "full of joy", etc.) and emotional semantic relationships (such as causal relationships, contrast relationships, etc.) in the lyrics, the emotional information in the lyrics is extracted and fused with the emotional information in the audio features to form a multimodal emotional feature representation.

[0160] Consider the influence of the emoticons in music performance (such as dynamics markings, tempo markings, expression terms, etc.) on emotional expression. These emoticons are quantified, for example, the dynamics markings are converted into the corresponding volume value range, the tempo markings are converted into rhythm speed values, and the expression terms are converted into emotional tendency values. Then, these quantized emoticon information is integrated into the emotional feature vector to further enrich the description of emotional expression features and improve the accuracy of emotional classification.

[0161] In addition, deep learning models are used to classify emotional expression features, such as convolutional neural networks (CNN) or recurrent neural networks (RNN) and their variants (such as long short-term memory networks LSTM, gated recurrent units GRU). Taking CNN as an example, a network structure containing multiple convolutional layers, pooling layers, and fully connected layers is constructed. The convolutional layer is used to automatically extract local feature patterns in audio features and lyrics text features (if combined with lyrics analysis), the pooling layer performs feature dimensionality reduction, and the fully connected layer is used for the final classification decision.

[0162] A large amount of vocal music data with emotion categories labeled is used to train the deep learning model. During the training process, appropriate loss functions (such as the cross entropy loss function) and optimization algorithms (such as the Adam optimizer) are used to adjust the model parameters through the back propagation algorithm so that the model can accurately identify the emotional expression characteristics of different songs. At the same time, data enhancement techniques (such as random cropping of audio signals, adding noise, etc., random replacement and insertion of lyrics text, etc.) are used to expand the training data, improve the generalization ability of the model, and avoid overfitting.

[0163] Finally, in the classification of physical fitness, personalized physical fitness indicators are considered to deeply study the relationship between vocal singing and different physical fitness indicators, not only limited to physical fitness and endurance, but also other factors such as breathing control ability (related to lung capacity, respiratory muscle strength, etc.), voice endurance (related to laryngeal muscle endurance, vocal cord health, etc.), body coordination (affecting the overall performance of stage performance and vocalization), etc. For each physical fitness indicator, a corresponding quantitative evaluation method or a way to obtain relevant data is formulated. - For example, more detailed physical fitness data of users can be obtained by cooperating with professional physical fitness testing institutions or using wearable devices (such as respiratory monitors, laryngeal muscle activity monitoring devices, etc.). Then, the vocal functional repertoire is adapted and classified according to these personalized physical fitness indicators, and the repertoire is matched with different physical fitness characteristics, so that the classification results are more in line with the actual physical fitness needs of users, providing a more accurate basis for repertoire recommendation for personalized teaching.

[0164] When constructing a dynamic adaptation model, a dynamic adaptation physique classification model is constructed to take into account the changes in the user's physique during the vocal learning process. As users improve their physical fitness and enhance their breathing control ability through training, the types of repertoires that are suitable for them may also change. A method based on time series analysis or incremental learning is used to update the adaptation physique classification results based on the user's physique data at different learning stages. - For example, the user's physique is regularly assessed, and new physique data is input into the model. The model adjusts the adaptation relationship between the repertoire and the physique based on the changing trends of the new data and historical data. This ensures that the repertoire recommended to users is always adapted to their current physical condition, promotes the user's continuous progress in the vocal learning process, and avoids learning difficulties or physical injuries caused by the mismatch between the difficulty of the repertoire and the physique.

[0165] When generating the track feature description vector and constructing the mapping relationship, in the vector fusion strategy, after the vocal feature metadata such as audio features, emotional expression features and adaptive physique classification results are vectorized, the track feature description vector is generated by weighted fusion or feature fusion method based on deep learning. For weighted fusion, weights are manually or automatically assigned according to the importance of each feature in vocal teaching. For example, for the stage focusing on skill training, the audio feature weight is higher; for the emotional expression training stage, the emotional expression feature weight increases. Then the weighted feature vectors are spliced ​​or summed to obtain the track feature description vector. Feature fusion based on deep learning can use neural networks (such as multi-layer perceptron MLP) to learn the complex relationship between different features and automatically determine the fusion weight. The audio feature vector, emotional expression feature vector and adaptive physique classification vector are used as the input layer, and after nonlinear transformation of the hidden layer, the fused track feature description vector is obtained at the output layer. This method can better explore the intrinsic connection between features and generate more representative track feature description vectors.

[0166] When storing and managing the mapping relationship, a mapping relationship storage structure is established, such as using a database management system (DBMS) to store the mapping relationship between vocal functional repertoires and repertoire feature description vectors. In the database, a reasonable table structure is designed to associate and store repertoire information (such as repertoire name, ID, artist, etc.), repertoire feature description vectors, and related index information (such as an index tree constructed based on feature vectors for rapid retrieval of similar repertoires).

[0167] Optionally, the vocal music knowledge base creation module performs the following steps when extracting audio features for each vocal music functional track:

[0168] Each vocal functional track is divided into a number of audio frames, and each audio frame is subjected to windowing processing;

[0169] Performing time-frequency conversion on the windowed audio frame to obtain a corresponding spectrum, and converting the spectrum into a Mel scale;

[0170] The Mel scales corresponding to all video frames are summed to obtain the Mel spectrum;

[0171] Logarithmically compress the Mel spectrum, and perform discrete cosine transform on the compressed Mel spectrum to obtain the extracted audio features.

[0172] To this end, audio frame segmentation and windowing processing have realized the basis for local feature analysis. By segmenting vocal functional tracks into audio frames, local feature analysis of audio signals can be performed. Music signals change continuously in time, and the audio features in different time periods may be different. For example, in the prelude, verse, chorus and other parts of a song, features such as rhythm, pitch and timbre will change. Through audio frame segmentation, we can focus on the audio characteristics in each short time segment, so as to capture these local changes more carefully. Windowing processing further optimizes this local analysis and reduces the spectrum leakage generated when the audio frame is truncated. Spectral leakage will lead to inaccurate spectrum analysis and cause interference between different frequency components. Windowing can make the audio signal transition smoothly within the window, reduce the impact of spectrum leakage, improve the accuracy of subsequent spectrum analysis, and lay the foundation for accurate extraction of audio features.

[0173] Time-frequency conversion and Mel scale conversion realize the acquisition of time-frequency features. Time-frequency conversion (such as short-time Fourier transform) converts audio signals from time domain to frequency domain, providing information about audio signals in two dimensions: time and frequency. In vocal music, pitch is related to frequency, rhythm is related to time, and timbre is related to the distribution of frequency components. The spectrum obtained by time-frequency conversion can clearly observe the changes of different frequency components in the audio signal over time, so as to accurately obtain information about important audio features such as pitch, rhythm and timbre. For example, the pitch can be determined by the peak frequency in the spectrum, the speed of the rhythm can be analyzed by the speed of change of the frequency components, and the timbre characteristics can be evaluated by the shape and energy distribution of the spectrum. Converting the spectrum to Mel scale is of great significance because the Mel scale is designed based on the auditory perception characteristics of the human ear. The human ear's perception of sound frequency is nonlinear and is more sensitive to low-frequency sounds. The Mel scale can better simulate this nonlinear perception. In vocal music teaching, audio features that are consistent with human auditory perception are more helpful in understanding and evaluating students' singing performance, and are also more conducive to selecting appropriate repertoires for training according to students' auditory characteristics. For example, when comparing two notes of similar pitch but different frequencies, the Mel scale can more accurately reflect the difference in how the human ear perceives them, allowing for more reasonable decisions in repertoire selection and teaching guidance.

[0174] Mel spectrum summation and logarithmic compression, the Mel scale corresponding to all audio frames is summed to obtain the Mel spectrum, which can describe the spectral characteristics of the track as a whole. It integrates the frequency information of each audio frame at the Mel scale and provides an audio feature representation from a global perspective. This overall spectrum feature is very useful for distinguishing tracks of different styles and emotional expressions. For example, a cheerful track may have relatively high Mel spectrum energy in the high frequency band, while a sad track may have more prominent Mel spectrum energy in the low frequency band. Through the Mel spectrum, the overall audio characteristics of the track can be quickly identified, providing an important basis for track classification and matching. Logarithmic compression of the Mel spectrum helps to highlight the low-frequency components in the spectrum, which is consistent with the characteristic that the human ear is more sensitive to low-frequency sounds. In vocal music, the low-frequency part often contains more information related to rhythm, harmony foundation, and emotional atmosphere creation. For example, the bass line of a song plays an important role in determining the stability of the rhythm and the heaviness of the emotion. Logarithmic compression can make these low-frequency features more obvious, and at the same time compress the dynamic range, reducing the problem of large differences in eigenvalues ​​caused by large differences in audio signal strength, so that the subsequent discrete cosine transform (DCT) can more effectively extract key audio features and improve the accuracy and effectiveness of feature extraction.

[0175] Discrete cosine transform (DCT) can convert Mel spectrum data into a set of coefficients with the characteristic of concentrated energy. DCT concentrates the energy of the signal on a few low-frequency coefficients, which means that by selecting these low-frequency coefficients as audio features, the main information of the audio signal can be represented with less data. In vocal teaching, this main information is usually closely related to the basic characteristics of pitch and rhythm. For example, the first coefficient of DCT (DC coefficient) can reflect the average energy level of the audio signal, which is related to the overall loudness of the sound; the subsequent low-frequency coefficients can represent the energy distribution of the audio signal in different frequency ranges, which is used to analyze the characteristics of pitch and timbre. Selecting appropriate low-frequency DCT coefficients as audio features can not only reduce data storage space and calculation amount, but also accurately capture the most valuable information in the audio signal for vocal teaching and repertoire matching, and improve the operating efficiency and accuracy of the system.

[0176] Optionally, the vocal knowledge base creation module performs the following steps when classifying the emotional expression features of each vocal functional track:

[0177] Performing a nonlinear transformation on the audio feature to map the audio feature into an emotion expression feature space to obtain a mapping feature vector;

[0178] Based on the set kernel function, the mapped feature vector is mapped onto a linear classification plane to obtain an emotion expression feature classification.

[0179] To this end, by mapping the audio features to the emotion expression feature space through nonlinear transformation, and then mapping them to the linear classification plane for classification based on the kernel function, the complex nonlinear relationship between audio features and emotion expression can be better captured. The emotional expression of vocal works is often not simply linearly corresponding to a few features of the audio, but the result of multiple audio features interacting in a complex way. This method can adapt to this complex relationship, thereby more accurately classifying the emotional expression features of the repertoire, which helps to match more suitable repertoires for vocal learners with different emotional needs.

[0180] The use of kernel functions makes it possible to perform classification operations in high-dimensional feature space, even if the original audio feature dimension is relatively low. It can transform problems that may be difficult to linearly classify in low-dimensional space into linear classification problems that are easier to handle in high-dimensional space. For example, some tracks with similar emotions but slightly different audio features may be difficult to clearly distinguish in the original audio feature space, but after being mapped to high-dimensional space through kernel functions, more suitable classification boundaries can be found, improving the accuracy and reliability of classification, thereby providing strong support for accurately matching tracks with consistent emotional expressions in vocal teaching.

[0181] The classification method that adapts to multiple emotion types can effectively handle a variety of different emotion types, such as cheerfulness, sadness, excitement, lyricism, etc. The differences in audio features of vocal works of different emotion types may be subtle. Through the above-mentioned nonlinear transformation and kernel function mapping classification process, these differences can be distinguished more finely and the tracks can be accurately classified into different emotion categories. This is very important for recommending tracks based on learners' emotional expression abilities and needs. For example, for learners with delicate emotional expression, tracks with accurate emotional classification and in line with their emotional expression style can be recommended to improve learning effects.

[0182] There are many nonlinear mapping functions for audio feature nonlinear transformation that map audio features to the emotion expression feature space, such as radial basis function (RBF) kernel function, polynomial kernel function, etc. Taking RBF kernel function as an example, its expression is K(x i , x j ) = exp(-γ||x i -x j || 2), where \(x_i\) and \(x_j\) are audio feature vectors, and \(\gamma\) is the kernel parameter. Through this function, the input audio feature vector can be nonlinearly transformed and mapped to a higher-dimensional emotion expression feature space to obtain a mapping feature vector. In practical applications, it is necessary to select a suitable nonlinear mapping function according to the specific situation. Generally, experiments are conducted to compare the performance of different functions on audio data sets with known emotion classification annotations, and the function that can achieve the best classification effect is selected. At the same time, the value of the kernel parameter \(\gamma\) will also affect the mapping effect. It is usually necessary to determine its optimal value through cross-validation and other methods to ensure that the mapped feature vector can better reflect the emotion expression characteristics.

[0183] In addition to the kernel function method, neural networks can also be used to achieve nonlinear transformation of audio features. For example, a multi-layer perceptron (MLP) is constructed, whose input layer receives the audio feature vector, the middle hidden layer uses a nonlinear activation function (such as the ReLU function) to perform feature transformation, and the output layer maps the audio features to the emotional expression feature space to obtain a mapping feature vector. When training a neural network, it is necessary to collect a large amount of audio data with emotional classification annotations as a training set. By defining a suitable loss function (such as the cross entropy loss function) and using an optimization algorithm (such as the Adam optimizer), the network parameters are adjusted using the back propagation algorithm so that the network can accurately map the audio features to the emotional expression feature space and obtain a mapping feature vector that meets the requirements.

[0184] Linear classification based on kernel functions, kernel functions include RBF kernel function, polynomial kernel function, etc. When mapping feature vectors to linear classification planes for emotional expression feature classification, it is also necessary to select a suitable kernel function according to the specific situation. Generally, the selection is based on the test results of different kernel functions on data with known emotional classification annotations. For the selected kernel function, determine its related parameters. Taking the RBF kernel function as an example, in addition to the kernel parameter \(\gamma\) mentioned above, penalty parameters \(C\) and the like may also be involved. These parameters are usually determined by methods such as cross-validation, and multiple experiments are conducted on the training data set to find the parameter combination that enables the classifier to achieve the best classification effect.

[0185] The tool for linear classification based on kernel function is support vector machine (SVM). SVM separates the mapping feature vectors of different emotion categories as much as possible by finding an optimal classification hyperplane. When using SVM, the mapping feature vector obtained by nonlinear transformation is used as input, configured according to the kernel function and parameters determined previously, and then the SVM classifier is trained using the training data with known emotion classification annotations.

[0186] During the training process, by adjusting the parameters of the classifier (such as kernel parameters, penalty parameters, etc.), the classifier can accurately classify the new mapping feature vector (i.e., the vector obtained after the audio features of the new vocal functional repertoire are transformed nonlinearly as described above) for emotional expression features. At the same time, in order to evaluate the performance of the classifier, evaluation methods such as cross-validation and leave-one-out can be used to judge the classification effect of the classifier based on evaluation indicators such as accuracy, recall rate, and F1 value, and further optimize the parameters and structure of the classifier based on the evaluation results to improve the accuracy and generalization ability of classification.

[0187] Optionally, the vocal knowledge base creation module performs the following steps when classifying each vocal functional repertoire according to its fitness:

[0188] Map each physical classification to a target variable;

[0189] Separating a pitch feature and a rhythm feature from the audio feature, and calculating the audio short-time energy based on the pitch feature and the rhythm feature;

[0190] The correlation coefficient between the short-time energy and the target variable is calculated, so as to select the physical classification corresponding to the maximum correlation coefficient as the suitable physical classification.

[0191] To this end, by associating physical classification with audio features for adaptive physical classification, more suitable vocal functional repertoires can be recommended to users based on their different physical conditions. Users with different physical conditions may have different performances and tolerances when singing vocal works. For example, users with better physical fitness may be more suitable for singing repertoires with strong rhythms and large pitch spans, while users with slightly weaker physical fitness may be more suitable for relatively soothing repertoires. This method of adapting physical classification helps to achieve personalized repertoire recommendations and improve the comfort and effect of users in the process of vocal learning and singing.

[0192] In vocal singing, singing a piece of music requires a certain amount of physical support, especially for some pieces with frequent pitch changes and complex rhythms, which require higher physical fitness such as breathing control and vocal endurance of the singer. By calculating the short-term energy of the audio and analyzing its correlation with different physical classifications, it is possible to indirectly evaluate the degree of physical fitness required by the piece of music from the perspective of audio characteristics, and then match the piece of music with the appropriate physical type. This can avoid recommending overly difficult pieces of music to users with mismatched physical conditions, and prevent users from feeling frustrated due to excessive fatigue or inability to meet singing requirements. At the same time, it can also give full play to the user's physical advantages and improve the quality of singing.

[0193] The method of mining the potential connection between audio features and physique helps to mine the potential connection between audio features (such as pitch features and rhythm features) and physique classification. Although audio features mainly reflect the musical characteristics of the track itself, through the association analysis with physique classification, it can be found that tracks with different musical characteristics may be more suitable for users with certain specific physique characteristics. This provides a new perspective for further understanding the relationship between vocal singing and human physique, and also provides a valuable basis for the subsequent optimization of the track recommendation system and the formulation of more accurate vocal teaching plans.

[0194] In the specific implementation, the physical classification is mapped to the target variable. First, a clear physical classification system needs to be determined. For example, physical fitness can be divided into different categories based on common physical fitness, endurance, flexibility, etc., such as high physical fitness, medium physical fitness, low physical fitness, high endurance, medium endurance, low endurance, etc. Or according to more comprehensive indicators, such as dividing physical fitness into energetic type, ordinary type, weak type, etc. Each physical fitness classification is mapped to the target variable according to certain rules. For example, different physical fitness classifications can be simply represented by numbers, such as high physical fitness type is assigned 1, medium physical fitness type is assigned 2, low physical fitness type is assigned 3, etc. The purpose of this is to facilitate subsequent quantitative processing when calculating correlation coefficients and other operations, so that different physical fitness classifications can participate in the association analysis with audio features in a unified form.

[0195] When separating pitch features and rhythm features and calculating the short-time energy of audio, a variety of methods can be used to separate pitch features from audio features. A common method is to obtain the spectrum of the audio through short-time Fourier transform (STFT), and then find the peak frequency in the spectrum, which usually corresponds to the pitch. In addition, some special pitch detection algorithms can also be used, such as pitch detection algorithms based on autocorrelation functions, pitch detection algorithms based on harmonic structures, etc. These algorithms accurately extract pitch features by analyzing the periodicity and other characteristics of audio signals. For the extraction of rhythm features, STFT and some subsequent processing methods can also be used. For example, by analyzing the energy changes of the spectrum at different time points, combined with indicators such as the zero-crossing rate of the audio signal, the rhythm features such as the speed of the rhythm and the pattern of the rhythm can be determined. Some special rhythm analysis algorithms can also be used, such as rhythm analysis algorithms based on time domain analysis, to analyze rhythm features by calculating the time interval between adjacent peaks in the audio signal.

[0196] When calculating the short-time energy of audio, the short-time energy of audio is calculated based on the separated pitch features and rhythm features. A common approach is to first determine a shorter time window (such as a few milliseconds to tens of milliseconds), and then calculate the energy of the audio signal within this window. The short-time energy can be obtained by summing the square values ​​of the audio signal within the window. For example, if the audio signal is \(x(t)\) and the time window is \([t_1,t_2]\), the calculation formula for the short-time energy \(E\) is: The short-time energy calculated in this way can reflect the intensity of the audio signal within the short time window, and is associated with the pitch characteristics and rhythm characteristics, thereby providing basic data for subsequent analysis of its correlation with the target variable (physique classification).

[0197] When calculating the correlation coefficient and determining the classification of the suitable constitution, the method for calculating the correlation coefficient is:

[0198] Where \(x_i\) and \(y_i\) represent two sets of data (here, short-time energy data and target variable data), \(\bar{x}$ and \(\bar{y}$ represent the means of the two sets of data, and \(n$ is the number of data points. By calculating the Pearson correlation coefficient, the degree of linear relationship between short-time energy and target variable can be quantitatively expressed.

[0199] When determining the suitable physical classification, after calculating the correlation coefficient between the short-time energy and each physical classification (target variable), the physical classification corresponding to the largest correlation coefficient is selected as the suitable physical classification. This means that the track has the strongest linear relationship with the selected physical classification in terms of audio characteristics (reflected by short-time energy), which implies to some extent that the track may be more suitable for users with the characteristics of this physical classification to sing. For example, if the calculation results show that a certain track has the largest correlation coefficient with the high physical fitness type physical classification, then the high physical fitness type is determined as the suitable physical classification for the track, so that in the subsequent vocal knowledge base construction and track recommendation links, the track can be accurately recommended to suitable users based on the user's physical condition.

[0200] Optionally, when the vector matching unit matches the vocal functional repertoire in the constructed vocal knowledge base based on the user feature description vector, the vector matching unit performs the following steps:

[0201] Calculating the adaptation distance between the user feature description vector and the song feature description vector;

[0202] Based on the mapping relationship, the vocal functional track corresponding to the track feature description vector when the adaptation distance is the largest is determined as the vocal functional track matching the target user.

[0203] Specifically, for example, a matching algorithm may be used to calculate the adaptation distance between the user feature description vector and the track feature description vector in the vocal knowledge base.

[0204] To this end, by calculating the adaptation distance between the user feature description vector and the track feature description vector, and selecting the vocal functional track corresponding to the maximum adaptation distance, accurate personalized track matching can be achieved. Everyone has different physiological, psychological and physical characteristics, and their needs and abilities for vocal learning and singing are also different. This matching method fully considers the comprehensive characteristics of the user, and finds the track that best suits their own characteristics for them, which helps to improve the user's participation, learning effect and performance in the vocal learning process, and meet the personalized needs of different users. Matching the right track for the user can optimize his vocal learning experience. If the track is too difficult or the style does not match the user's characteristics, it may cause the user to feel frustrated or unable to give full play to his own advantages during the learning process. Through precise matching, users can be exposed to tracks that are more suitable for them, and it is easier to master skills and express emotions during singing, thereby enhancing their self-confidence, maintaining their enthusiasm for vocal learning, and promoting their continuous progress in the field of vocal music.

[0205] The matching mechanism that fully utilizes knowledge base resources enables the repertoire resources in the vocal knowledge base to be more fully utilized. By accurately screening out suitable repertoires based on user characteristics, repertoires of different styles, difficulties, and emotional expressions have the opportunity to be recommended to matching users, avoiding the situation where high-quality repertoires are buried due to lack of accurate recommendations, and improving the utilization rate and value of the entire vocal knowledge base.

[0206] To this end, when calculating the adaptation distance, the cosine value of the angle between two vectors is calculated to measure their similarity. For the user feature description vector \(\vec{u}\) and the track feature description vector \(\vec{v}\), the cosine similarity formula is: Where \(\vec{u}\cdot\vec{v}\) is the dot product of the two vectors, \(\vert\vec{u}\vert\) and \(\vert\vec{v}\vert\) are the moduli of the two vectors. The value of cosine similarity ranges from -1 to 1. The closer the value is to 1, the more similar the two vectors are and the higher the degree of adaptation.

[0207] Alternatively, for the user feature description vector \(\vec{u}\) and the track feature description vector \(\vec{v}\), the formula for calculating the similarity using distance is: Where \(u_i\) and \(v_i\) are the \(i$th elements of vectors \(\vec{u}\) and \(\vec{v}\) respectively, and \(n$ is the dimension of the vector. The smaller the distance, the closer the two vectors are and the higher the degree of adaptation.

[0208] Alternatively, the formula for calculating the Manhattan distance between the user feature description vector \(\vec{u}\) and the track feature description vector \(\vec{v}\) is: where \(u_i$ and $vi_i$ are the $i$th element of vectors $\vec{u}$ and $\vec{v}$, respectively, and $n$ is the dimension of the vector. The smaller the Manhattan distance, the more compatible the two vectors are.

[0209] When implementing vector preprocessing, it may be necessary to preprocess the user feature description vector and the track feature description vector before calculating the adaptation distance. For example, normalize the vectors so that they have the same value range to avoid distance calculation deviation caused by different value ranges of vector elements. Common normalization methods include minimum-maximum normalization, which normalizes each element \(x\) in the vector according to the formula Processing is performed, where \(min$ and $max$ are the minimum and maximum values ​​of the elements in the vector respectively, and \(x'$ is the normalized element. Through normalization, the accuracy and fairness of the adaptation distance calculation can be improved.

[0210] When determining matching tracks based on the adaptation distance, a mapping relationship between vocal functional tracks and track feature description vectors has been established in the vocal knowledge base. After calculating the adaptation distance between the user feature description vector and each track feature description vector, according to this mapping relationship, the vocal functional track corresponding to the track feature description vector corresponding to the maximum adaptation distance (or meeting specific conditions, such as being greater than a certain threshold and maximum) can be quickly determined. For example, if cosine similarity is used as a matching algorithm, then after calculating the cosine similarity of all track feature description vectors and the user feature description vector, find the track feature description vector with the largest cosine similarity value, and then find the corresponding vocal functional track through the mapping relationship, which is the vocal functional track matched to the target user.

[0211] With regard to multi-condition matching, in some cases, matching tracks may not only be determined based on the adaptation distance. For example, in addition to considering the maximum adaptation distance, it may also be combined with the user's specific needs or preferences, such as the user's desire to sing a track in a certain style, or having specific requirements for the difficulty of the track. In this case, when determining the matching tracks, it is necessary to further screen out tracks that meet the user's other requirements on the basis of meeting the adaptation distance conditions. This may involve analyzing and comparing other elements of the track feature description vector in the vocal knowledge base (such as style labels, difficulty levels, etc.) to ensure that the final matching track meets both the adaptability to the user's characteristics and other expectations of the user.

[0212] The present application also provides a computer program product on which a computer executable program is stored, and the computer executable program is run to perform the following steps:

[0213] Vectorizing the physiological parameters, psychological parameters, and physical parameters respectively to obtain a physiological description vector, a psychological description vector, and a physical description vector of the target user;

[0214] The physiological description vector, the psychological description vector, and the physical description vector of the target user are integrated to obtain a user feature description vector;

[0215] Matching vocal functional repertoires in a constructed vocal knowledge base based on the user feature description vector;

[0216] The vocal functional repertoire matching the target user is output.

[0217] The embodiment of the present application also provides a personalized vocal functional repertoire recommendation method, which includes:

[0218] Obtain physiological parameters, psychological parameters, and physical parameters of target users;

[0219] Vectorizing the physiological parameters, the psychological parameters, and the physical parameters respectively to obtain a physiological description vector, a psychological description vector, and a physical description vector of the target user;

[0220] The physiological description vector, the psychological description vector, and the physical description vector of the target user are integrated to obtain a user feature description vector;

[0221] Matching vocal functional repertoires in a constructed vocal knowledge base based on the user feature description vector;

[0222] The vocal functional repertoire matching the target user is output.

[0223] The embodiment of the present application also provides a personalized vocal functional music repertoire recommendation device, which includes:

[0224] A parameter acquisition unit, used to acquire physiological parameters, psychological parameters, and physical parameters of the target user;

[0225] A vectorization unit, used to vectorize the physiological parameters, the psychological parameters, and the physical parameters, respectively, to obtain a physiological description vector, a psychological description vector, and a physical description vector of the target user;

[0226] A fusion unit, used for fusing the physiological description vector, the psychological description vector, and the physical description vector of the target user to obtain a user feature description vector;

[0227] A matching unit, used for matching vocal functional repertoires in a constructed vocal knowledge base based on the user feature description vector;

[0228] An output unit is used to output the vocal functional repertoire matching the target user.

[0229] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A personalized vocal intelligent teaching system, characterized in that: include: A front-end data acquisition device and a back-end server, wherein the front-end data acquisition device is equipped with a physiological parameter determination module, a psychological parameter determination module, and a physical parameter determination module, and the back-end server is equipped with a vectorization processing device, which includes: a vectorization module, a vector fusion unit, a vector matching unit, and an output unit; The physiological parameter determination module is used to determine the physiological parameters of the target user; The psychological parameter determination module is used to determine the psychological parameters of the target user; The physical parameter determination module is used to determine the physical parameters of the target user; The vectorization module is used to vectorize the physiological parameters, the psychological parameters, and the physical parameters respectively to obtain the physiological description vector, the psychological description vector, and the physical description vector of the target user; The vector fusion unit is used to fuse the physiological description vector, the psychological description vector, and the physical description vector of the target user to obtain a user feature description vector; The vector matching unit is used to match vocal functional repertoires in the constructed vocal knowledge base based on the user feature description vector; The output unit is used to output the vocal functional music tracks matching the target user to the front-end data acquisition device for display.

2. A personalized vocal intelligent teaching system according to claim 1, characterized in that: The physiological parameters include at least one of breathing efficiency, vocal cord status, and resonance cavity.

3. A personalized vocal intelligent teaching system according to claim 2, characterized in that: When determining the physiological parameters of the target user, the physiological parameter determination module obtains the respiratory monitoring data generated by the smart wearable device, and separates the respiratory monitoring data to obtain respiratory efficiency characterization data, vocal cord state characterization data, and resonance cavity characterization data, respectively, and performs feature extraction on the respiratory efficiency characterization data, the vocal cord state characterization data, and the resonance cavity characterization data to obtain the respiratory efficiency, the vocal cord state, and the resonance cavity.

4. A personalized vocal intelligent teaching system according to claim 3, characterized in that: The physiological parameter determination module includes: a data preprocessing module, a dual-channel cycle processing module, and a feature extraction module; When determining the physiological parameters of the target user, the physiological parameter determination module includes the following steps: The data preprocessing module communicates and interacts with the smart wearable device to obtain the generated respiratory monitoring data, and serializes the respiratory monitoring data to obtain serialized data, and blocks the serialized data to obtain data blocks that are continuous in time; The dual-channel loop processing module performs dual-channel loop processing on the temporally continuous data blocks to separate the breathing efficiency representation data, the vocal cord state representation data, and the resonance cavity representation data from the temporally continuous data blocks; The feature separation and extraction are used to respectively extract features from the breathing efficiency characterization data, the vocal cord state characterization data, and the resonance cavity characterization data to obtain the breathing efficiency, the vocal cord state, and the resonance cavity.

5. A personalized vocal intelligent teaching system according to claim 4, characterized in that: The dual-channel loop processing module performs dual-channel loop processing on the temporally continuous data blocks to separate the breathing efficiency characterization data, the vocal cord state characterization data, and the resonance cavity characterization data from the temporally continuous data blocks, including: The first recurrent neural network in the dual-channel recurrent processing module performs an intra-block operation on a single data block to obtain data components of the single data block; The second recurrent neural network in the dual-channel recurrent processing module performs inter-block operations on adjacent data blocks to obtain dependency relationships between data components of adjacent data blocks and generate an inter-block dependency feature sequence based on the dependency relationships; The output neural network in the dual-channel loop processing module performs multi-layer perception processing on the dependent feature sequence to obtain breathing efficiency characterization data, vocal cord state characterization data, and resonance cavity characterization data.

6. A personalized vocal intelligent teaching system according to claim 5, characterized in that: After the output neural network in the dual-channel loop processing module performs multi-layer perception processing on the dependent feature sequence to obtain breathing efficiency characterization data, vocal cord state characterization data, and resonance cavity characterization data, it also includes: performing feature fusion on the breathing efficiency characterization data, vocal cord state characterization data, and resonance cavity characterization data to obtain sound fusion features; determining a breathing classification description based on the sound fusion features, matching vocal functional repertoires based on the breathing classification description, and generating breathing adjustment instructions, wherein the breathing adjustment instructions are used to guide the user to adjust the breathing method.

7. The personalized vocal intelligent teaching system according to claim 1, characterized in that: The psychological parameter includes at least one of a personality classification, an emotional expression description, and a psychological development stage identifier.

8. The personalized vocal intelligent teaching system according to claim 4, characterized in that: When determining the psychological parameters of the target user, the psychological parameter determination module obtains psychological assessment data and separates the psychological assessment data to obtain personality classification representation data, emotional expression description representation data, and psychological development stage representation data, respectively, and performs feature extraction on the personality classification representation data, emotional expression description representation data, and psychological development stage representation data to obtain the personality classification, the emotional expression description, and the psychological development stage identifier.

9. A personalized vocal intelligent teaching system according to claim 8, characterized in that: The psychological parameter determination module includes: an encoding module, a feature head module, and a context module; The psychological parameter determination module performs the following steps when determining the psychological parameters of the target user: Encoding the psychological assessment data based on the encoding module to obtain a hidden feature vector; Based on the feature head module, the hidden feature vector is nonlinearly mapped to obtain feature representations related to personality classification, emotional expression, and psychological development stage; Based on the context module, context-related decision prediction processing is performed on the feature representations related to personality classification, emotional expression, and psychological development stage to extract features of the personality classification representation data, emotional expression description representation data, and psychological development stage representation data, respectively, to obtain the personality classification, the emotional expression description, and the psychological development stage identifier.

10. A computer program product, characterized in that A computer executable program is stored thereon, and the computer executable program is run to perform the following steps: Vectorizing the physiological parameters, psychological parameters, and physical parameters respectively to obtain a physiological description vector, a psychological description vector, and a physical description vector of the target user; The physiological description vector, the psychological description vector, and the physical description vector of the target user are integrated to obtain a user feature description vector; Matching vocal functional repertoires in a constructed vocal knowledge base based on the user feature description vector; The vocal functional repertoire matching the target user is output.

Images