Sound standardization evaluation system based on multi-dimensional acoustic parameters and dynamic analysis of emotional scenes

Abstract

The present invention provides a sound standardization evaluation system for dynamic analysis of multi-dimensional acoustic parameters and emotional scenes, aiming to achieve structured and objective evaluation of speech performance; the system comprises an audio acquisition module, an audio processing module, a standard evaluation module and an integrated processing module; the audio processing module performs noise reduction, pre-emphasis, sampling rate adjustment, framing and slicing operations on original speech data; the standard evaluation module performs pronunciation accuracy analysis, acoustic basic skill feature evaluation or emotion and scene adaptability analysis based on multimodal fusion according to the evaluation type set by the user; parameters including fundamental frequency, resonance peak, sound intensity, fundamental frequency perturbation, amplitude perturbation, etc. are extracted during the evaluation, and classification prediction is completed in combination with a machine learning model; finally, the integrated processing module generates an evaluation report including scoring results, question prompts and personalized training suggestions, providing closed-loop feedback to support voice training and expression optimization.

Description

Technical Field

[0001] The present invention relates to the technical field of sound evaluation, and in particular to a sound standardization evaluation system for dynamic analysis of multi-dimensional acoustic parameters and emotional scenes. Background Art

[0002] With the widespread application of speech training and pronunciation assessment technologies in education, entertainment, healthcare, and other fields, related technical systems are gradually moving from manual scoring to automated evaluation. Existing speech assessment systems mostly focus on basic speech recognition and pronunciation accuracy assessment, such as Mandarin tonal recognition and English phonetic pronunciation comparison. Some systems have initially introduced emotion recognition models to identify basic emotions such as joy, anger, sadness, and fear, and use speech scoring and feedback mechanisms to assist language learning. However, most systems still rely on static scoring mechanisms and are primarily based on a single dimension.

[0003] The above-mentioned systems generally have the following problems: First, in terms of multi-dimensional sound control, existing technologies have failed to achieve systematic quantitative evaluation of parameters such as breath strength, resonance cavity (such as oral resonance, chest resonance), and virtual and real sound conversion; second, in terms of adaptability to emotions and performance scenes, they can only identify basic emotion types, lack the ability to adapt to fine-grained emotions such as gloating, begging, and questioning, and lack the dynamic prompts and integration capabilities for scenes such as role-playing and narration; third, in terms of pronunciation parameter system, existing solutions mostly stay at the level of word pronunciation correctness, and do not cover professional intonation features such as speech flow sound change, rhythm control, and pauses; fourth, the system structure has problems such as module separation, single evaluation algorithm, and fragmented feedback mechanism, and has not yet formed a closed-loop system integrating evaluation, analysis and training.

[0004] Therefore, there is an urgent need to provide a voice standardization evaluation system that integrates acoustic analysis, linguistic modeling and multimodal emotion recognition to meet the needs of comprehensive evaluation of multi-dimensional vocal performance and interpretation ability. Summary of the Invention

[0005] The present application provides a sound standardization evaluation system for dynamic analysis of multi-dimensional acoustic parameters and emotional scenes, so as to realize the fusion evaluation of multi-dimensional acoustic features and complex emotional scenes in speech expression, thereby improving the accuracy of sound evaluation.

[0006] This application provides a sound standardization evaluation system for dynamic analysis of multi-dimensional acoustic parameters and emotional scenes, including:

[0007] Audio acquisition module, used to collect user voice audio and generate raw audio data;

[0008] An audio processing module, configured to pre-process the raw audio data to obtain pre-processed audio data; wherein the pre-processing includes noise reduction, sampling rate adjustment, pre-emphasis, framing, and slicing operations;

[0009] The standard evaluation module is configured to determine an evaluation type according to a user setting; receive the pre-processed audio data and perform an evaluation operation corresponding to the evaluation type, wherein the evaluation operation includes at least one of the following operations:

[0010] Recognizing the speech units in the preprocessed audio data, calculating the accuracy of the linguistic features of initials, finals, and speech flow sound changes, and generating a word pronunciation evaluation result;

[0011] Extracting acoustic features from the preprocessed audio data, including fundamental frequency, formant, sound intensity, fundamental frequency perturbation, and amplitude perturbation, and inputting the features into machine learning models trained for different evaluation items for classification and prediction to generate basic skills evaluation results;

[0012] Based on the user's emotions or situational information, a prompt word template is constructed, and a multimodal fusion analysis is performed on the pre-processed audio data and the situational text, outputting emotion matching, scene restoration, and emotion coherence indicators to generate an emotion evaluation result;

[0013] The integration processing module is used to summarize the evaluation results generated by the standard evaluation module, generate a structured evaluation report, which includes dimension scores, corresponding question prompts and personalized training suggestions, and transmit it back to the user terminal.

[0014] The beneficial effects of this application mainly include: (1) By constructing a complete audio preprocessing chain, including noise reduction, sampling rate adjustment, pre-emphasis, framing and slicing operations, the quality of audio signals and the stability of subsequent analysis are significantly improved, providing technical support for high-precision acoustic feature extraction. (2) By integrating linguistic features, acoustic parameters and emotional context, it is possible to conduct multi-dimensional evaluation of user voice, achieving a comprehensive evaluation from word pronunciation accuracy to vocal skills to emotional expression, overcoming the problem of dimensional fragmentation in existing technologies. (3) Based on the evaluation task call and training model matching mechanism, it is possible to dynamically select the optimal algorithm model for classification prediction, ensuring that various evaluation items operate under the dual optimization of accuracy and adaptability, and improving the scientific nature of the evaluation and personalized matching capabilities. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 This is a schematic diagram of a sound standardization evaluation system for dynamic analysis of multi-dimensional acoustic parameters and emotional scenes provided in the first embodiment of the present application.

[0016] Figure 2 This is a timing diagram of the user voice evaluation process involved in the first embodiment of the present application.

[0017] Figure 3 It is a heat map of the correlation matrix between acoustic features involved in the first embodiment of the present application. DETAILED DESCRIPTION

[0018] The following description sets forth many specific details to facilitate a thorough understanding of the present application. However, the present application can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the scope of the present application. Therefore, the present application is not limited to the specific implementations disclosed below.

[0019] The first embodiment of the present application provides a sound standardization evaluation system for dynamic analysis of multi-dimensional acoustic parameters and emotional scenes. Figure 1 , which is a schematic diagram of the first embodiment of this application. Figure 1 The first embodiment of the present application provides a sound standardization evaluation system for dynamic analysis of multi-dimensional acoustic parameters and emotional scenes.

[0020] The sound standardization evaluation system for dynamic analysis of multi-dimensional acoustic parameters and emotional scenes includes an audio acquisition module 101 , an audio processing module 102 , a standard evaluation module 103 and an integration processing module 104 .

[0021] The audio collection module 101 is used to collect user voice audio and generate original audio data.

[0022] In the present invention, the function of the audio acquisition module 101 is to collect the user's voice audio and generate raw audio data, specifically including but not limited to obtaining voice signals from the user's terminal device (such as a microphone, mobile phone, tablet computer or computer with voice input function), and converting the obtained analog audio signals into digital signal form in real time or non-real time for subsequent data processing and analysis.

[0023] In a preferred embodiment, the audio acquisition module 101 includes an audio input interface, an analog-to-digital conversion unit, and a cache unit. The audio input interface is used to receive the user's original voice, such as the user reading a demonstration text, emotionally reciting a text, or performing free expression voice input. The analog-to-digital conversion unit converts the analog voice signal into digital audio data. The sampling rate is preferably set to 16 kHz or 44.1 kHz, and the quantization bit depth is not less than 16 bits to ensure that the sound quality is clear enough to facilitate the subsequent accurate extraction of acoustic features. The sampling duration is dynamically controlled according to the specific application task. For example, the sampling duration for basic pronunciation evaluation is generally 3 to 10 seconds, and the audio duration for emotional interpretation or role-playing evaluation can be 10 to 60 seconds.

[0024] During the audio acquisition process, the audio acquisition module 101 preferably integrates a silence detection mechanism to filter out invalid input. For example, if the user is silent or the background noise intensity exceeds a threshold, the acquisition process will not be initiated or the user will be prompted to re-record. Furthermore, to ensure recording quality, the system can monitor the signal-to-noise ratio (SNR) of the recording signal in real time. If the SNR is detected to be below a set standard (e.g., below 30 dB), a user interface prompt can be issued or the acquisition process can be automatically paused.

[0025] In its implementation, the audio acquisition module 101 supports multiple re-recording and version control mechanisms for voice acquisition. Users can enter multiple recording versions for the same task. The system uses a metadata management mechanism to identify and number the audio data, ensuring that each piece of raw voice data has a unique identifier and is accompanied by information such as the acquisition time, acquisition device number, and acquisition user ID, facilitating subsequent task invocation, training optimization, and evaluation comparison.

[0026] The audio acquisition module 101 may further include echo suppression and mute noise reduction functions to reduce the echo interference caused by the hardware device itself and eliminate the ambient background noise in non-voice segments, such as air conditioning sounds, electric fan sounds, etc., to improve the cleanliness and effectiveness of the original audio data from the source.

[0027] Therefore, audio acquisition module 101 not only assumes the basic responsibility of digitizing user voice signals, but also provides raw voice input data with reliable technical quality, clear structure, and strong traceability for subsequent evaluation processes through mechanisms such as analog-to-digital conversion, signal-to-noise monitoring, silence filtering, multi-version recording, and raw data identification. This module constitutes the data entry point for the entire sound standardization evaluation system and provides fundamental support for system performance.

[0028] The audio processing module 102 is used to pre-process the original audio data to obtain pre-processed audio data; wherein the pre-processing includes noise reduction, sampling rate adjustment, pre-emphasis, framing and slicing operations.

[0029] Audio processing module 102 performs comprehensive preprocessing on the raw audio data generated by audio acquisition module 101. This aims to improve audio quality, unify data formats, and provide a stable, standardized input foundation for feature extraction and model analysis in subsequent evaluation phases. This module specifically includes a series of sequential processing steps, including noise reduction, sampling rate adjustment, pre-emphasis, framing, and slicing. Each step has a clear technical goal and implementation method.

[0030] In actual system operation, the audio processing module first performs noise reduction on the raw audio input data. This process employs classic signal processing methods such as spectral subtraction or wavelet denoising to estimate and filter out background noise. The system then establishes a noise model by analyzing silent segments and removes background interference from the entire audio signal, reducing the risk of bias caused by ambient noise in subsequent acoustic feature extraction.

[0031] After noise reduction, the system adjusts the sampling rate of the audio data according to a unified standard. Preferably, if the original audio sampling rate is higher or lower than the standard value (for example, 16kHz), it will be upsampled or downsampled through a bandpass filter and a resampling algorithm to ensure that all input data has consistent frequency domain characteristics. This process not only helps improve system robustness but also ensures the alignment of the time window processing required for subsequent feature extraction.

[0032] After the sampling rate is adjusted, the audio data enters the pre-emphasis stage. This step mainly applies a linear filter function (such as ,in The value ranges from 0.95 to 0.97) to enhance the proportion of high-frequency components and suppress low-frequency noise interference, thereby improving the expression of high-frequency speech features such as voiceless consonants and plosives.

[0033] The system then performs frame processing on the audio signal, dividing the continuous audio signal into short time frames of equal length. Preferably, each frame is 20 milliseconds long, with a 50% overlap between adjacent frames, resulting in a 10 millisecond frame shift. This setting ensures local stability of the speech signal within the time window, enabling the extracted features, such as fundamental frequency and formants, to have higher temporal accuracy and robustness against jitter.

[0034] Finally, to accommodate the input requirements of various evaluation models, the audio processing module also slices the framed data. Slicing combines several consecutive frames into a set of temporal context segments, for example, every five frames as an analysis unit, thereby supporting feature modeling and model input based on short-term semantic context. Slicing also dynamically adjusts segment length based on the speech duration of different tasks, such as 0.5-second segments for short sentence pronunciation assessments and longer than 1.5 seconds for long text interpretation assessments.

[0035] To achieve this, the audio processing module can encapsulate the underlying processing logic based on existing open-source toolkits (such as PyDub) and gradually transmit the processing results through a data pipeline to ensure data consistency between each step. All pre-processed audio data output uses a unified floating-point PCM format and is accompanied by timestamps and frame number information, which the standard evaluation module 103 uses for feature alignment and model scheduling during the evaluation process.

[0036] The standard evaluation module 103 is configured to determine an evaluation type according to a user setting; receive the pre-processed audio data and perform an evaluation operation corresponding to the evaluation type, wherein the evaluation operation includes at least one of the following operations:

[0037] Recognizing the speech units in the preprocessed audio data, calculating the accuracy of the linguistic features of initials, finals, and speech flow sound changes, and generating a word pronunciation evaluation result;

[0038] Extracting acoustic features from the preprocessed audio data, including fundamental frequency, formant, sound intensity, fundamental frequency perturbation, and amplitude perturbation, and inputting the features into machine learning models trained for different evaluation items for classification and prediction to generate basic skills evaluation results;

[0039] According to the user's emotions or situational information, a prompt word template is constructed, and a multimodal fusion analysis is performed on the preprocessed audio data and the situational text, and the emotion matching degree, scene restoration degree and emotion coherence indicators are output to generate the emotion evaluation results.

[0040] The Standard Assessment Module 103 is the core analysis unit within the system responsible for speech analysis, feature classification, and assessment output. Its functionality directly corresponds to the assessment objectives set by the user. It receives pre-processed audio data from the Audio Processing Module 102 and executes different analysis paths based on the assessment type, ultimately generating structured and actionable assessment results. This module supports three types of assessment operations: pronunciation assessment, voice basics assessment, and emotion and contextual adaptability assessment, enabling multi-dimensional speech ability assessment and personalized feedback.

[0041] For word pronunciation assessment, the module segments and recognizes pre-processed audio data. By incorporating a pre-trained speech recognition model, it automatically identifies the initials, finals, and whole syllables in the user's speech, further refining it to include intra-syllable variations such as light tones, erhua (erhua) sounds, and changes in modal particles. These models typically utilize deep neural networks combined with time series modeling, adapting to varying pronunciation styles and speaking rates. The system compares each recognized syllable with the standard pronunciation, aligning them at the phoneme level and determining whether the pronunciation is accurate. If a syllable deviates from the standard pronunciation, the system identifies the mispronunciation and generates corresponding error messages, such as "tongue position too high," "insufficient aspiration," or "indistinct nasal consonant at the end of the syllable." The assessment results include not only the overall accuracy score but also the location of the error, phoneme analysis, and spoken language suggestions for each segment, enabling users to make individual improvements.

[0042] In terms of basic voice assessment, the module extracts core parameters that reflect vocal physiology and technical capabilities based on continuous frame-level audio feature extraction and combines it with acoustic signal processing algorithms. These parameters include but are not limited to fundamental frequency, resonance peaks, sound intensity, fundamental frequency perturbations, and amplitude perturbations. The system dynamically tracks and analyzes the frequency changes of each frame, and combines the information of the previous and next frames to determine whether the voice is stable, the pitch is accurate, the speaking speed is uniform, and the breath is coherent. For example, if the amplitude fluctuations in a certain section of speech are large and the period is irregular, the system can be identified as unstable breath, prompting the user to strengthen breathing control exercises.

[0043] For each specific vocalization dimension, the system has pre-defined evaluation items and their corresponding highly correlated acoustic feature combinations. For example, voice brightness is evaluated by analyzing sound intensity and high-frequency energy ratio; oral resonance relies primarily on spectral center frequency and formant shift; and breath control is analyzed by combining sound intensity fluctuation and cyclic stability. During the data preparation phase, the system constructed a sample set based on approximately 28,000 expert-annotated data points. Through feature selection, cross-validation, and performance metric testing, the system trained the best-performing machine learning model for each evaluation dimension. These models include support vector machines, random forests, gradient boosting trees, multi-layer perceptrons, decision trees, and logistic regression, all of which can be dynamically scheduled through model configuration files. Based on each model's ability to distinguish positive and negative samples, actual accuracy, and operational efficiency, the system intelligently selects the most appropriate model type for real-time classification and prediction, and outputs a score, scoring basis, and training recommendations for that dimension.

[0044] The execution mechanism for emotion and situational assessment is more complex. The system first constructs a corresponding prompt word template from a library of emotion templates based on the user's selected target emotion or the specified role and scenario information. The prompt word template not only includes emotion types (such as sadness, anger, surprise, and pleading), but also incorporates role identity settings (such as age, gender, and social status) and scene context (such as formal meetings, family interactions, and public speeches). For example, if the user selects "Simulate an elderly person expressing apology," the system will generate a prompt word template that incorporates features such as low tone, slow speech speed, and long interjections.

[0045] Next, the module feeds the cue template, preprocessed audio data, and sample audio of reference emotions into a multimodal fusion analysis engine. This engine simultaneously processes acoustic features (such as pitch range, speech rate, intensity, and tonal coherence) and semantic content (emotional keywords, tone structure, and word choice in the text transcribed by automatic speech recognition). Through joint modeling, it determines whether the actual expression matches the target emotion, character setting, and contextual requirements. Evaluation outputs include emotional match (i.e., whether the specified emotion is present), scene fidelity (i.e., whether the expression fits the setting), and emotional coherence (i.e., whether the emotional expression in the speech is consistent and flows naturally).

[0046] The multimodal fusion analysis engine is used to jointly model and quantitatively evaluate the emotional matching, scene restoration, and emotional coherence of user voice samples.

[0047] The input of the engine consists of three parts: the frame-level acoustic feature sequence of the user's pre-processed audio sample, the time-aligned semantic coding stream converted from the structured prompt word template, and the frame-level acoustic comparison features of the reference demonstration audio. The first part of the input comes from the audio processing module, whose feature dimensions include MFCC (Mel-frequency cepstral coefficient), pitch, energy, formant position, pitch curvature (such as delta-pitch), speech rate boundary and rhythm change, etc. All frame features are organized in the form of a two-dimensional tensor with a tensor dimension of , where T is the number of audio frames and F is the number of feature dimensions per frame. The second input embeds the cue word template using a syntactic-semantic embedding model (such as BERT), resulting in a structured semantic encoding stream. Temporal position encoding is appended to each semantic segment vector to maintain frame-level alignment with the audio frame, generating a semantic tensor of dimension T × E, where E is the embedding dimension. The third input is a reference audio tensor with the same structure as the preprocessed audio, providing a baseline for the target emotional expression.

[0048] The engine itself uses a fusion structure based on a dual-channel attention mechanism, in which one channel input is an acoustic feature sequence and the other channel is a semantic encoding stream. In the core cross-attention layer, the system constructs multiple attention heads (the recommended number is 4) to perform feature mapping and weight calculations on the acoustic frame features and semantic segments respectively. Each attention head uses the standard dot-product attention method to calculate the similarity between the audio frame and the corresponding semantic vector to derive the mutual attention weight matrix. To avoid dependence on large models, the attention calculation process can be linearly transformed and normalized using preset projection matrices (such as W_Q, W_K, W_V), allowing the system to run even on resource-constrained platforms.

[0049] The preset projection matrix is a linear transformation weight matrix used to map the input acoustic feature vector or semantic encoding vector to the unified attention space, which represents the transformation matrices of query, key and value respectively. For example, W_Q is used to map the acoustic input vector to the query vector, and W_K and W_V are used to map the semantic input vector to the key vector and value vector respectively. Each projection matrix is a two-dimensional floating-point array, whose dimensions are set according to the input and output dimensions (for example, the input dimension is F or E, and the output is a unified dimension D), and is optimized and learned during the training phase after random initialization or importing the pre-trained model. In order to ensure feasibility under minimum implementation conditions, these matrices can also be fixed to manually set small-scale parameter matrices for the initial realization of attention mapping capabilities.

[0050] After completing mutual attention, the system concatenates the weight matrix with the original features to form a fused feature representation, which is then fed into a two-layer fully connected neural network. This network has 128 and 64 nodes per layer, respectively, and uses the ReLU activation function, providing basic nonlinear expression capabilities. The final output is three independent scalars, corresponding to emotional matching, scene reproduction, and emotional coherence. Each scalar, ranging from 0 to 1, represents the degree of alignment with the target and is used to drive subsequent structured evaluation feedback.

[0051] Through the collaborative work of the above structures, the multimodal fusion analysis engine can simultaneously capture the acoustic change characteristics in speech and the emotional expression direction in the semantics of the prompt words, extract the dynamic coupling relationship between them, and thus achieve deep modeling of expression consistency, semantic matching and emotional flow.

[0052] When the assessment task is "Role Play," the system also performs a voice feature matching analysis on the character parameters entered by the user. For example, older characters typically have lower pitch, slower speech, and less undulating voice; whereas lively, child-like characters tend to have noticeable jumps in voice and large fluctuations in intensity. The system evaluates the degree of match between the relevant indicators in the user's voice and the target features, generating a character fit score to determine the success of the voice interpretation.

[0053] The standard evaluation module 103 also has an intelligent adaptive mechanism. During the execution process, it can combine the user's historical evaluation records to dynamically fine-tune the threshold parameters, prompt word generation logic and feature weight strategy of each model to adapt to individual voice differences and optimize the stability and personalization of the evaluation results.

[0054] The evaluation results are output in a unified format, including dimensional scores, score explanations, error locations, improvement suggestions and priority training directions for each evaluation type, for integration, visualization and personalized training plan push by the integration processing module 104.

[0055] In order to further improve the accuracy and pertinence of the evaluation model, the standard evaluation module 103 introduces a systematic feature correlation analysis and algorithm matching mechanism in the process of feature processing and model training. Specifically, for the basic voice evaluation part, the system first performs a correlation matrix analysis between acoustic features and each evaluation dimension based on large-scale user data. Through this analysis, the key feature groups under different dimensions can be clearly identified. For example, in the evaluation of bright and dark voices, sound intensity, fundamental frequency, harmonic noise ratio and frequency band energy ratio are identified as core features with significant influence. For the oral resonance dimension, the spectrum center frequency and energy distribution are found to be highly correlated with the evaluation results. For example, the evaluation of virtual voice and breath stability is highly dependent on the two parameters reflecting periodic perturbations, namely fundamental frequency perturbation and amplitude perturbation; chest resonance is affected by the combined influence of spectrum center frequency, sound intensity, amplitude perturbation and fundamental frequency; the dimensions of strong voice, weak voice, and strong and weak breath control are jointly dependent on features such as sound intensity, fundamental frequency and position changes of the first resonance peak.

[0056] The system uses the above analysis results to construct a feature engineering strategy, that is, for each evaluation item, it selects a combination of strongly correlated features and constructs a feature vector based on this. During the model training phase, the system conducts multiple rounds of iterative training and evaluation for each dimension based on more than 28,000 evaluation sample data annotated by professional voice teachers. During the training process, mainstream machine learning algorithm models such as support vector machines, random forests, gradient boosting trees, decision trees, logistic regression, and multi-layer perceptrons are introduced. Through cross-validation, a comprehensive evaluation of key performance indicators such as accuracy, recall rate, F1 score, and area under the curve (AUC) of each model is conducted. Ultimately, the system automatically selects the algorithm model with the best performance and applies it to each evaluation dimension.

[0057] The results show that different evaluation items are indeed well-suited to different algorithm models, with significant performance differences. For example, the evaluation items such as virtual voice, strong breath control, weak breath control, and voice darkness performed best under the support vector machine model, with accuracy rates of 82.45%, 88.62%, 88.33%, and 91.13%, respectively. The evaluation tasks of strong voice and chest resonance were more suitable for the random forest algorithm, with corresponding accuracy rates of 88.89% and 91%, respectively. The gradient boosting algorithm model achieved an evaluation accuracy of 83% for oral resonance. The decision tree model supported weak voice, with an accuracy of 82.64%. The logistic regression model performed best for voice brightness, with an accuracy of 94.35%. The real voice evaluation achieved an accuracy of 91.14% using the multilayer perceptron neural network model.

[0058] In the emotion and situational assessment section, the system also summarized the optimal implementation path through extensive experiments. Emotional assessment items are first divided into three categories: single emotion, situational interpretation, and role-playing. Single emotion assessment covers specific emotion types such as emotional fullness, joy, anger, impatience, coldness, fierceness, sadness, fear, love, smugness, ecstasy, questioning, calmness, humility, and begging. For the identification of emotional fullness, the system directly uses acoustic features for modeling and adopts a logistic regression algorithm for binary classification prediction, achieving an accuracy rate of 84.47%. In other single emotion assessments, the system combines prompt word templates to guide expression and uses a multimodal fusion model to input reference audio and target speech. After preprocessing and template tuning, the emotion matching accuracy is stable between 88% and 95%.

[0059] The core of the scenario-based interpretation assessment is to evaluate whether the user's voice matches the specified scenario and target emotion. The system uses a dynamic prompt generation mechanism, combined with the user's input scenario tags and target emotion, to automatically generate structured prompts, such as "Express excitement during a speech" or "Simulate nervous reading in an exam room," to guide the user's expression. The system then evaluates the voice's performance across three dimensions: emotional matching, scene reproduction, and emotional coherence, and outputs a comprehensive score. Field tests have shown that this module has an overall judgment accuracy of 87%.

[0060] The role-playing module further introduces role characteristic parameters such as age and personality based on the situational interpretation. For example, for tasks such as "simulating a young child expressing happiness" or "playing the role of a calm leader expressing concern", the system integrates the situational characteristics and role characteristics in the prompt word template to generate a composite template to guide the user to express the corresponding voice characteristics. The system then uses age characteristics such as speech speed, pitch, and tone length changes, and personality characteristics such as tone intensity and intonation jump amplitude to conduct a coupling matching analysis between the user's voice and the target role, and generates a role suitability score. In multiple scenario tests, the module's overall accuracy reached the range of 85% to 90%.

[0061] Through the above-mentioned modeling, training and verification strategies, the standard evaluation module 103 not only achieves high-accuracy, multi-dimensional speech ability evaluation, but also establishes a complete, closed-loop and scalable evaluation system based on scientific feature selection, classification algorithms and template scheduling strategies, laying a solid foundation for the engineering implementation of this system in the fields of speech teaching, expression training and intelligent speech evaluation.

[0062] In the sound standardization evaluation system for dynamic analysis of multi-dimensional acoustic parameters and emotional scenes described in the present invention, when the standard evaluation module performs the basic sound skills evaluation operation, in order to improve the evaluation accuracy and pertinence of each preset evaluation item (such as strong voice, false voice, chest resonance, etc.), a multi-stage, highly controllable acoustic feature selection mechanism is designed. This mechanism consists of three closely connected sub-steps to ensure that the features of the final input classification model have clear discriminative power and low interference risk, thereby effectively improving the system's fine-grained modeling capabilities for speech skills of different dimensions.

[0063] First, for each pre-defined assessment item, the system extracts historical user speech samples from the assessment database based on the corresponding scoring dimension. These samples are then grouped and paired with teacher samples annotated by professional voice instructors to construct a control sample set covering all scoring levels. For example, for the "breath stability" assessment item, the system groups samples that accurately express stable breath into one group, and samples that exhibit shaky or weak breath into another. Then, within these control samples, the system uses frame-level speech signal analysis to extract several acoustic parameters that are significantly correlated with vocal stability, spectral pattern changes, and energy distribution. These parameters include fundamental frequency perturbation (which measures the degree of fluctuation in the vocal cord vibration period), amplitude perturbation (which reflects the instability of sound energy output), spectral density center of gravity (which represents the resonance focus), and the amplitude of non-steady-state energy fluctuations (which describes short-term changes in speech intensity). These features together form the acoustic parameter set, providing the data foundation for subsequent differential modeling.

[0064] Next, the system performs inter-group difference enhancement processing on the above-mentioned acoustic parameter sets. This process does not use a simple statistical difference method, but introduces a shallow convolution structure constructed with a learnable weight kernel to model the response to parameter change trends between different level groups. Through training, the convolution structure can adaptively capture and enhance the feature gradients related to the performance level, and finally form a multi-dimensional feature response map. In this map, the system can accurately mark those feature indicators that are most sensitive to changes and have the clearest trends in high and low level performances. These feature change patterns constitute the "feature-sensitive subspace" of the corresponding evaluation items. This subspace is highly directional in the parameter dimension, ensuring that subsequent model training only focuses on feature channels that are truly discriminative.

[0065] Finally, within the obtained feature-sensitive subspace, the system further implements a feature conflict suppression strategy to avoid interference or ambiguity caused by overlapping features used in multiple evaluation tasks. The system introduces an "overlap suppression factor", which judges the interference potential of the current candidate feature by calculating the frequency of occurrence, sharing strength and feature direction consistency of the current candidate feature in other evaluation items. For features with high overlap, the system reduces their feature weights; for features that are unique and have outstanding performance, the system increases their proportion in the final feature combination. This strategy ensures that the target feature group finally screened out not only has good discrimination, but also has minimal impact on the prediction process of non-target evaluation items, thereby enhancing the specificity and anti-interference ability of the model.

[0066] This final feature set is fed into a trained classification model for the corresponding assessment item. This model, which can be a support vector machine, random forest, gradient boosting, or neural network, is used to predict and score newly input speech samples and compare them with the assessment grade standard, thereby generating an accurate basic vocal ability assessment result. Through this three-stage structured feature selection process, the present invention implements a speech feature extraction and modeling mechanism that is deeply bound to the assessment target, significantly improving the accuracy, stability, and adaptability of the system when evaluating multiple tasks in parallel.

[0067] In the implementation of this invention, to extract acoustic parameters closely related to voice basic skill assessment, the system pre-emphasizes each speech sample to enhance high-frequency components and suppress low-frequency noise interference. The speech signal is then divided into multiple overlapping short time frames, each approximately 20 milliseconds long, with a 10-millisecond interval between adjacent frames, to capture subtle changes in the speech signal over time.

[0068] For each frame of audio data, the system extracts multiple acoustic features, including key parameters such as fundamental frequency perturbation, amplitude perturbation, spectral density center of gravity, and non-steady-state energy fluctuation amplitude. Fundamental frequency perturbation measures the stability of the vocalization cycle. The system first uses the autocorrelation function of short-time frames to locate the main peak of the fundamental frequency cycle. The system then calculates the ratio of the standard deviation of the duration of adjacent cycles to the average period to determine the degree of fundamental frequency fluctuation. Amplitude perturbation reflects the stability of energy output during vocalization. The system quantifies its jitter by calculating the relative change in the maximum amplitude between adjacent cycles. The spectral density center of gravity is determined by performing a fast Fourier transform on each frame to obtain a frequency distribution map. A weighted average of the product of frequency and corresponding energy amplitude is then used to determine the location of spectral energy concentration, thereby inferring the resonance focus during vocalization. The non-steady-state energy fluctuation amplitude assesses the level of short-term energy fluctuation by statistically analyzing the energy changes between multiple consecutive frames. The resulting normalized fluctuation variance is then output as a parameter.

[0069] Through this feature extraction process, the system obtains a set of parameters that highly characterizes the stability and resonance structure of the speech signal, providing a reliable input foundation for subsequent score-based differential analysis, feature trajectory modeling, and multi-task interference control. This entire process is automated within the standard evaluation module, ensuring the stability, timeliness, and reproducibility of feature extraction. This provides the system with clear physical explanations and engineering feasibility for automated and personalized vocal skill assessment.

[0070] In a preferred embodiment of the present invention, when the standard evaluation module performs the basic voice skill evaluation operation, for each preset evaluation item (such as strong voice, virtual voice, and resonance indicators), the scoring level differential encoder is used to extract the acoustic feature trajectory that evolves with the score level. This module no longer uses the traditional static mean or variance features as input, but instead constructs a control sample set covering each score level, and guides a learnable differential neural network structure to generate a feature response map based on the difference relationship between the level embedding and the sample features. Through this structure, a trajectory feature vector with high discriminant ability can be obtained, which effectively maps the performance trend of the user sample in the target voice skill dimension, and supports the accuracy requirements of the evaluation model in the scenarios of subtle expression and graded judgment.

[0071] Furthermore, in order to improve the robustness of multiple evaluation tasks in parallel, the present invention designs a disturbance consistency resonance filtering mechanism. This mechanism introduces a linear kernel structure based on a shared disturbance channel to evaluate the degree of resonance triggered by the candidate features of the current evaluation item in other evaluation tasks, and calculates the interference risk of the features. According to the common risk of disturbances, the system dynamically adjusts the weight of the features in the final sensitive subspace, and gives priority to retaining the feature dimensions that are unique, highly discriminative, and not easily shared and interfered with by other tasks in this evaluation item. Compared with the existing methods based on information gain, variance screening, etc., this mechanism significantly enhances the practical effect of the system in task isolation, multidimensional modeling, and personalized training recommendations. Please refer to the following implementation code:

[0072] import numpy as np

[0073] import torch

[0074] import torch.nn as nn

[0075] import torch.nn.functional as F

[0076] #Build a scoring track differential encoder

[0077] class GradedContrastEncoder(nn.Module):

[0078] #Rating level differential encoder: simulates the process of users' ratings from low to high. The "evolution trajectory" of features is different from traditional static feature extraction, emphasizing the difference in rating levels rather than the mean.

[0079] def __init__(self, feature_dim):

[0080] super(GradedContrastEncoder, self).__init__()

[0081] self.delta_fc = nn.Linear(feature_dim, feature_dim)

[0082] self.score_embed = nn.Parameter(torch.randn(5, feature_dim)) # Assume 5 level labels

[0083] def forward(self, features, scores):

[0084] #features: [N, D], input feature vector

[0085] #scores: [N], score label corresponding to each sample (0-4)

[0086] # Get the corresponding rating level embedding

[0087] score_vecs = self.score_embed[scores]

[0088] # Differential representation: feature-level vector

[0089] delta = features - score_vecs

[0090] out = F.relu(self.delta_fc(delta))

[0091] return out # as a feature evolution response

[0092] # Perturbation consistent resonance filter

[0093] class PerturbationConsistencySuppressor(nn.Module):

[0094] Multi-task resonance risk assessment: avoids the "resonance" interference of selected features in other evaluation tasks, based on the perturbation commonality weight rather than simple cosine similarity

[0095] def __init__(self, feature_dim):

[0096] super(PerturbationConsistencySuppressor, self).__init__()

[0097] self.shared_filter = nn.Linear(feature_dim, 1, bias=False)

[0098] def forward(self, target_feature, other_features):

[0099] #target_feature: [D]

[0100] #other_features: [T, D], T is the feature center of other tasks

[0101] # Perturbation intensity: see each task's response to the shared perturbation channel

[0102] shared_risks = self.shared_filter(other_features) # [T, 1]

[0103] target_risk = self.shared_filter(target_feature.unsqueeze(0)) # [1,1]

[0104] # Calculate the "resonance" with the target disturbance

[0105] resonance_score = torch.mean(torch.sigmoid(shared_risks - target_risk))

[0106] # Suppression factor: The stronger the resonance → the greater the suppression

[0107] suppression_weight = torch.clamp(1 - resonance_score, min=0.2)

[0108] return target_feature suppression_weight

[0109] # Main process function: Integrate multiple modules to construct the final feature vector

[0110] def construct_inventive_feature_vector(feature_tensor, score_tensor, other_tasks_center_features):

[0111] # feature_tensor: basic features of all samples [N, D]

[0112] #score_tensor: score level of each sample [N]

[0113] #other_tasks_center_features: Center features of other evaluation tasks [T, D]

[0114] # Stage 1: Rating track encoding

[0115] encoder = GradedContrastEncoder(feature_dim=feature_tensor.shape[1])

[0116] trajectory_map = encoder(feature_tensor, score_tensor) # [N, D]

[0117] # Stage 2: Aggregate feature responses at different levels to obtain the sensitivity vector of the target evaluation item

[0118] trajectory_mean = torch.mean(trajectory_map, dim=0) # [D]

[0119] # Stage 3: Resonance filter suppresses overlap with other tasks

[0120] suppressor = PerturbationConsistencySuppressor(feature_dim=feature_tensor.shape[1])

[0121] optimized_vector = suppressor(trajectory_mean, other_tasks_center_features) # [D]

[0122] return optimized_vector # The final output can be fed into a dedicated classification model

[0123] # Construct test data simulation run

[0124] N = 10 # Sample size

[0125] D = 8 # feature dimension

[0126] T = 3 # Number of other tasks

[0127] torch.manual_seed(42)

[0128] features = torch.randn(N, D)

[0129] scores = torch.randint(0, 5, (N,)) # simulated score labels

[0130] other_task_feats = torch.randn(T, D) # Other task center features

[0131] # Execution process

[0132] final_feat = construct_inventive_feature_vector(features, scores,other_task_feats)

[0133] # Output the final feature vector

[0134] final_feat

[0135] Furthermore, when performing the sound basic skills evaluation operation, the standard evaluation module includes an acoustic channel weighting adjustment strategy based on characteristic interference suppression and discrimination enhancement, and the acoustic channel weighting adjustment strategy includes the following steps:

[0136] In the feature-sensitive subspace, for each dominant acoustic channel, a channel response scoring function is constructed , which is used to characterize its performance differentiation ability under the target evaluation item, is defined according to the following formula 1:

[0137] ;

[0138] in, Indicates channel The mean value in the high-scoring sample group; high-scoring samples refer to speech samples whose expert ratings are higher than the preset threshold; Indicates channel The mean value in the low-scoring sample group; low-scoring samples refer to speech samples whose expert ratings are lower than the preset threshold; , Represents channels Standard deviations in high and low scoring groups; A very small positive number introduced to prevent the denominator from being zero;

[0139] Based on the overlap between candidate channels in different evaluation items, a channel exclusive index is introduced , in order to control the channel interference level, which is calculated using the following formula 2:

[0140] ;

[0141] in, Indicates channel Whether it appears in the evaluation item ’s target feature group (1 if it appears, 0 if it does not appear); is the index of the current evaluation item, = is the total number of evaluation items; the calculation formula expresses the proportion of the current channel shared across all evaluation tasks other than the target evaluation item, thereby inferring its degree of exclusiveness. The closer the index is to 1, the less frequently the channel appears in other tasks, and the less likely it is to interfere. In this system, evaluation items refer to different dimensions of ability used to assess a user's vocal performance. For example, a common evaluation item might be breath stability, which determines whether the user maintains even breathing while speaking and whether the voice trembles or breaks. Another evaluation item might be chest resonance, which assesses the depth and penetration of the user's voice. Other evaluation items might include pitch control, which examines the natural transition between high and low notes during speech, and speech rhythm control, which measures whether the rhythm of speech is appropriate and whether there are inappropriate changes in speed. Furthermore, this might include whether emotional expression is appropriate to the specific context, such as whether the voice is low and the speed is slowed when expressing sadness, or whether the volume and tone rise when expressing anger.

[0142] Define the channel weighting coefficient according to the following formula 3: for:

[0143] ;

[0144] in, and is a hyperparameter used to adjust the balance between discrimination and exclusivity. The recommended value is ,This parameter ratio can be adjusted and optimized in the experiment to adapt to the vocal behavior characteristics of different user groups.

[0145] Through the above-mentioned weighting strategy, the system can optimize the weights of the dominant acoustic channels during the training phase, so that the final classification model input features are more focused on channels with high discriminability and low interference, thereby improving the model evaluation accuracy and task adaptability.

[0146] For example, in the task of assessing "chest resonance," the system analyzed multiple candidate acoustic channels and found that the change in the center of gravity of the spectrum differed significantly between high- and low-scoring users. Furthermore, this channel was used less frequently in other evaluation tasks, demonstrating strong discrimination and exclusivity. Meanwhile, while another channel, such as sound intensity, can also reflect vocal differences, it frequently appears in multiple tasks, such as "loud voice" and "breath control," easily introducing cross-interference.

[0147] Using this weighting strategy, the system assigns a higher weight to the spectral center of gravity channel and a relatively lower weight to the sound intensity channel. Later, during training, the classification model will prioritize changes in the former, prioritizing its patterns for distinguishing and judging during modeling. The resulting model demonstrates greater specificity and recognition accuracy when automatically evaluating chest resonance, significantly reducing the risk of misjudgment due to interference from other task features.

[0148] In the implementation of the present invention, in order to achieve dynamic binding of the optimal classification model for each preset basic sound skill evaluation item, the standard evaluation module has designed a three-stage execution process that combines acoustic channel analysis, model structure response adaptation and indicator-driven screening to ensure that the classification model can highly fit the performance characteristics of different evaluation tasks, thereby improving the overall evaluation accuracy and model stability.

[0149] First, the system constructs a set of scoring level control samples based on user samples and annotated samples covering all scoring levels of the evaluation item in the evaluation database. On this basis, the system calculates the response intensity and distribution trend of each acoustic feature channel (such as fundamental frequency, resonance peak, spectral center of gravity, energy change, etc.) in different scoring levels through frame-level feature extraction and channel response modeling. This process can calculate indicators such as frame mean, variance, time gradient, and response direction consistency after channel normalization to obtain a set of quantitative "response gradient maps". The system marks the channels in the map with the most significant changes between scoring levels and the clearest discrimination trends as the "dominant acoustic channel set", which represents the most representative acoustic performance path in the evaluation item.

[0150] Before binding the model structure for the basic vocal skills assessment, the system first constructs a set of scoring comparison samples for each assessment item. This set consists of speech samples covering all scoring levels for that assessment item and is divided into multiple subgroups based on the teacher's annotation information for each sample, such as "Excellent," "Good," "Fair," and "Needs Improvement." The system divides each sample into short time frames, using commonly used parameters of 20 milliseconds per frame length and 10 milliseconds frame shift. A fixed-dimensional acoustic feature channel is extracted from each frame. The system's preset channels include, but are not limited to, the following: fundamental frequency (which describes the period of vocal cord vibration), formant position (which reflects the shape of the oral cavity and pharynx), spectral density center of gravity (representing the location of sound energy concentration), energy intensity (reflecting changes in sound pressure), fundamental frequency perturbation (measuring frequency fluctuations), amplitude perturbation (measuring energy fluctuations), and harmonic-to-noise ratio (reflecting the clarity and hoarseness of the voice).

[0151] To ensure consistent feature scales across samples in subsequent analysis, the system normalizes the features of each channel across all samples within each scoring group. Specifically, the system calculates the mean and standard deviation of all frame values for each channel within each scoring group. The mean is then subtracted from each frame value within that channel and divided by the standard deviation to create a normalized channel matrix. This process ensures that feature variation trends across different scoring groups are more prominent.

[0152] Based on the normalized results, the system then calculates the characteristic variations of each acoustic channel across different scoring groups to generate a "channel response gradient map." The calculation of the channel response gradient map includes four types of metrics. The first is the normalized mean difference. For each channel, the difference between the mean values of the current and next groups is calculated across all scoring groups, and the maximum difference is recorded as the average response difference index for the channel. The second is the directional consistency index. This index checks whether the direction of change in the mean value of the channel across all scoring groups is consistent (consistently increasing or decreasing). If so, it is marked as "directionally stable," otherwise it is marked as "directionally inconsistent." The third is the channel response stability index. This index calculates the standard deviation of all frame values for the channel within each scoring group and compares the standard deviations of low- and high-level groups to determine the stability of the channel in high-quality vocalization. The fourth is the inter-frame change gradient index. For each channel, a sequence of adjacent differences of all frame values is calculated and averaged within each scoring group. This average inter-frame difference is then compared across groups to assess the vocal smoothness or controllability of the channel.

[0153] Each acoustic channel receives a set of quantitative scores across the four aforementioned metrics. The system assigns uniform weighting parameters to each channel, such as 40% for mean difference, 20% for directional consistency, 20% for response stability, and 20% for inter-frame gradient change. These weightings can be adjusted based on the preferences of the evaluation task. The system aggregates the weighted scores into an overall score and ranks all channels from highest to lowest based on their overall score.

[0154] After sorting, the system extracts a "dominant acoustic channel set" based on predefined channel filtering rules. These filtering rules can include selecting the top 30% of scoring channels or selecting all channels with scores above the average score of all channels plus one standard deviation. The channels included in this set are considered the most discriminative acoustic feature channels for the current evaluation item and will be used in subsequent model structure adaptation analysis and final evaluation.

[0155] After obtaining the set of dominant acoustic channels, the system enters the model structure adaptability evaluation stage. For multiple pre-set candidate model structures, including support vector machines, random forests, gradient boosting algorithms, multi-layer perceptrons, logistic regression and decision trees, the system analyzes the response stability of these models on the dominant acoustic channels one by one. The evaluation process includes but is not limited to the following criteria: the smoothness of the channel response surface of the model at different scoring levels, the sensitivity test to feature perturbations (such as the predicted offset amplitude after adding small noise), and whether the model's dependence intensity distribution on the dominant channel shows highly nonlinear fitting characteristics. For model types that show excessive sensitivity to channel drift, weak response gradient antagonism, or inconsistent feature directionality in the evaluation, the system will eliminate them from the candidate set and obtain the structural adaptability screening results.

[0156] To ensure the objectivity and reproducibility of the above evaluation, the system uses the following operational procedures to perform quantitative analysis of response stability in actual implementation. First, the system divides the score control sample set into multiple subsets, each subset contains the proportion distribution of samples of three levels: high, medium, and low. For each candidate model, the system trains and verifies the performance of the model on the dominant acoustic channel on each subset separately, and records the continuity of the model prediction output as the sample level changes and whether the discrimination interval remains stable. If the prediction boundary of the model between different subsets shifts significantly, or presents a broken distribution between high and low levels, it means that its response surface stability on the channel set is poor.

[0157] Furthermore, the system conducts perturbation experiments on the dominant channels one by one in each subset, that is, based on the original channel characteristics, small perturbations are introduced channel by channel, such as adding or subtracting an amplitude deviation of no more than 1% to each channel value, and observing the range of variation of the model output results. If the model's prediction results change dramatically after adding a very small perturbation, the model is too sensitive to channel noise and is not suitable for binding to the evaluation item. In addition, the system also calculates the feature importance or weight distribution of each dominant channel within each model. If there is a situation where the weight of a channel is extremely high and the weights of other channels are close to zero, and the high-weight channel is susceptible to external interference, it means that the model may have nonlinear overfitting and lack structural stability.

[0158] Through the above analysis path, the system ultimately eliminates models with unstable response structures and retains model types with stable performance, strong anti-interference ability and consistent directional trends on the dominant channel set as the source of structural adaptability screening results in the subsequent scoring process.

[0159] For example, when binding the evaluation model for the pre-set assessment item "breath stability," the system first constructs three subsets based on the score-based control sample set, corresponding to the teacher-labeled "stable," "moderately stable," and "unstable" samples, with the number of samples in each group remaining consistent. The system then extracts the dominant acoustic channel set, which includes three characteristic channels: fundamental frequency perturbations, amplitude perturbations, and short-term energy fluctuations.

[0160] For the candidate models—support vector machine (SVM), random forest (RF), multilayer perceptron (MLP), and logistic regression (LR)—the system sequentially trains and cross-validates each model on these three subsets. Taking the SVM as an example, after training each subset, the system compares the trend curves of its predicted probability as the score level changes. If the SVM's output probability distribution for high-grade samples highly overlaps with that for mid-grade samples, and the prediction boundary is concentrated at the edge of the sample, then its channel response surface has "flat areas" or "local jumps," indicating poor stability.

[0161] Subsequently, the system applies a perturbation test to each dominant channel. For example, all frame values in the fundamental frequency perturbation channel are adjusted up by 0.5%, and the original samples are input into each model again, and the difference in predicted scores before and after the perturbation is recorded. If the score change of the RF model on most samples is less than 5%, while the change of the MLP model exceeds 15%, it can be inferred that the RF model has better anti-interference stability for the channel. The system also analyzes the channel dependence strength of each model during the training process and finds that the LR model depends almost only on the amplitude perturbation channel, and the weight coefficients of the remaining channels are close to zero, suggesting that the model may be sensitive to feature deviations in practical applications and lack applicability.

[0162] The system eliminates the MLP and LR models, retaining only SVM and RF for final evaluation. It then uniformly scores the scoring accuracy, F1 score, and AUC to determine that the RF model performs best in the "breath stability" evaluation item, and uses it as the final binding model for subsequent user sample score prediction.

[0163] Finally, the system deploys the model structures retained in the structural adaptability screening results on the historical sample set of the evaluation item for cross-validation, constructing multiple rounds of evaluation experiments. The evaluation results for each model include multiple performance metrics such as precision, recall, F1 score, and area under the curve (AUC). The system weights these metrics to generate a unified performance evaluation score. The model with the highest score is considered the optimally adapted structure for the current evaluation item and is bound to that evaluation item for automatic scoring and grading of new user voice samples in the actual evaluation process.

[0164] Through the above-mentioned dynamic model selection mechanism, the standard evaluation module can not only achieve deep adaptation of the evaluation task and the model structure, but also ensure stable model performance, strong anti-interference ability, and consistent channel response. It effectively avoids the problems of insufficient model generalization and task deviation caused by static performance optimization in traditional model selection methods, thereby significantly enhancing the accuracy, professionalism and flexibility of the system in multiple voice skill evaluations.

[0165] When performing an emotion or situational assessment, the standard assessment module first calls upon the situational modeling component to generate a structured prompt word template that matches the user's settings. This template not only includes the semantic description of the target emotion (such as anger, sadness, and pleading), but also includes the scenario type (such as public speaking, role-playing, and family communication) and user-defined character traits (such as age, personality, and identity). It also internally encodes rhythm control markers to ensure a consistent time-aligned reference for subsequent fusion analysis. The system then converts this structured prompt word template into a sequence of semantic guide words. The fragment semantic embedding mechanism then segments this sequence into several fragments. Each fragment is embedded in a vector representation and bound to the time dimension, ultimately forming a time-aligned semantic encoding stream.

[0166] After obtaining the semantic encoding stream, the system simultaneously inputs it into the multimodal fusion engine along with the preprocessed audio data and reference demonstration audio. The core of the fusion engine is a computational framework built on a cross-modal attention mechanism. Within this engine, the system extracts acoustic features from each frame of the preprocessed audio data, including but not limited to pitch, energy, pitch curvature, speech rate boundaries, and rhythm changes. Simultaneously, the reference demonstration audio is processed into a frame structure of equal length to form a comparison baseline. Subsequently, the system establishes a correspondence between the two modalities by constructing a mutual attention weight matrix between the acoustic frame features and the semantic embeddings. This allows it to extract the degree of correlation between pitch and semantic keywords, the consistency between speech rate and rhythm and syntactic boundaries, and the dynamic coordination between pronunciation strength and emotional instructions. During this process, the system automatically identifies which acoustic variation patterns have a stable linkage relationship with semantic expressions, constructing a two-dimensional expression consistency map to visualize the structural linkage structure of emotional expression.

[0167] Based on this graph, the system further extracts three types of intermediate indicators. The first is a local inconsistency score, which identifies deviations in acoustic expression near key semantic units, such as a paradoxical area with accelerated tempo but a calm semantic meaning. The second is an emotional channel loss marker, which indicates insufficient emotional expression due to the lack of specific acoustic feature channels (such as a lack of intensity or pitch fluctuation). The third is semantic drift, which assesses whether the speech performance deviates from the intended direction set in the original prompt word template, such as a tone of threat instead of a pleading tone. The system jointly models these intermediate indicators with the global context parameters of the scene settings. Using a feature fusion and score aggregation strategy, it ultimately calculates quantitative indicators for three dimensions: emotional match (i.e., whether the target emotion is accurately expressed), scene fidelity (i.e., whether the vocal expression fits the set context), and emotional coherence (i.e., whether the emotional expression transitions naturally between the preceding and following sentences). The evaluation results are then returned to the integration processing module as structured scoring feedback for subsequent evaluation report generation.

[0168] The above process is automatically triggered every time a user submits an emotional or situational voice sample. It does not rely on the additional processing power of the user terminal, ensuring the real-time and accuracy of the evaluation process. It can also be personalized and extended to various scenarios and emotions through configuration templates and reference demonstration content. The following is the reference implementation code for the above-mentioned emotional and situational multimodal evaluation process:

[0169] import torch

[0170] import torch.nn as nn

[0171] import torch.nn.functional as F

[0172] # Assume that the maximum time step is T, the audio frame feature dimension is D_audio, and the semantic embedding dimension is D_text

[0173] T = 100

[0174] D_audio = 64

[0175] D_text = 128

[0176] class SemanticEmbeddingEncoder(nn.Module):

[0177] """Fragment semantic embedding module: fragments the structured prompt word template and converts it into a time-aligned embedding representation"""

[0178] def __init__(self, vocab_size, embedding_dim, hidden_dim):

[0179] super(SemanticEmbeddingEncoder, self).__init__()

[0180] self.embedding = nn.Embedding(vocab_size, embedding_dim)

[0181] self.gru = nn.GRU(embedding_dim, hidden_dim, batch_first=True,bidirectional=True)

[0182] def forward(self, template_seq):

[0183] # Input is the index sequence after semantic template segmentation [B, L]

[0184] embedded = self.embedding(template_seq) # [B, L, embedding_dim]

[0185] outputs, _ = self.gru(embedded) # [B, L, 2 hidden_dim]

[0186] return outputs

[0187] class AcousticFeatureExtractor(nn.Module):

[0188] """

[0189] Acoustic frame feature extractor: extracts features such as pitch, intensity, and speech rate boundaries from each frame

[0190] """

[0191] def __init__(self, input_dim, output_dim):

[0192] super(AcousticFeatureExtractor, self).__init__()

[0193] self.linear = nn.Linear(input_dim, output_dim)

[0194] def forward(self, acoustic_input):

[0195] # Assume that acoustic_input is the preliminary MFCC and other splicing features of each frame [B, T, input_dim]

[0196] return self.linear(acoustic_input) # [B, T, output_dim]

[0197] class CrossModalAttention(nn.Module):

[0198] """

[0199] Cross-modal attention mechanism module: realizing mutual attention between acoustic and semantic features

[0200] """

[0201] def __init__(self, dim_audio, dim_text, hidden_dim):

[0202] super(CrossModalAttention, self).__init__()

[0203] self.query_proj = nn.Linear(dim_audio, hidden_dim)

[0204] self.key_proj = nn.Linear(dim_text, hidden_dim)

[0205] self.value_proj = nn.Linear(dim_text, hidden_dim)

[0206] def forward(self, audio_feat, text_feat):

[0207] # audio_feat: [B, T, dim_audio], text_feat: [B, L, dim_text]

[0208] Q = self.query_proj(audio_feat) # [B, T, H]

[0209] K = self.key_proj(text_feat) # [B, L, H]

[0210] V = self.value_proj(text_feat) # [B, L, H]

[0211] attention_scores = torch.matmul(Q, K.transpose(1, 2)) / (Q.size(-1) 0.5) # [B, T, L]

[0212] attention_weights = F.softmax(attention_scores, dim=-1) # [B, T, L]

[0213] context = torch.matmul(attention_weights, V) # [B, T, H]

[0214] return context, attention_weights

[0215] class MultimodalFusionEngine(nn.Module):

[0216] """Multimodal fusion engine: Integrates acoustic features, semantic encoding, and reference audio to form a consistent graph"""

[0217] def __init__(self, dim_audio, dim_text, fusion_dim):

[0218] super(MultimodalFusionEngine, self).__init__()

[0219] self.attention = CrossModalAttention(dim_audio, dim_text, fusion_dim)

[0220] self.fusion_layer = nn.Linear(dim_audio + fusion_dim, fusion_dim)

[0221] self.out_layer = nn.Linear(fusion_dim, 3) # Output three evaluation indicators

[0222] def forward(self, audio_feat, text_feat):

[0223] # audio_feat: [B, T, dim_audio], text_feat: [B, L, dim_text]

[0224] context, weights = self.attention(audio_feat, text_feat) # context:[B, T, fusion_dim]

[0225] fusion = torch.cat([audio_feat, context], dim=-1) # [B, T, dim_audio+ fusion_dim]

[0226] fusion_out = torch.tanh(self.fusion_layer(fusion)) # [B, T, fusion_dim]

[0227] # Time dimension average pooling for final prediction

[0228] pooled = torch.mean(fusion_out, dim=1) # [B, fusion_dim]

[0229] output = self.out_layer(pooled) # [B, 3], corresponding to emotion matching, scene restoration, and emotional coherence

[0230] return output, weights

[0231] In the above implementation, the structured cue word template is first represented as a sequence of segmented indexes and then vectorized by the SemanticEmbeddingEncoder module. This module uses a bidirectional GRU to perform temporal modeling of the semantic guide words in each segment, generating a semantic encoding stream with time-aligned features, providing a precise semantic reference path for subsequent alignment analysis with acoustic frames.

[0232] After preprocessing audio data through the AcousticFeatureExtractor module, it outputs acoustic feature representations for each frame. These features can be composed of dimensions such as MFCC, pitch curvature, and rhythm boundaries, and are matched and fused after linear transformation. Subsequently, the audio frame and semantic encoding stream are jointly input into the CrossModalAttention module. By projecting them into query vectors and key-value vectors respectively, the system constructs a mutual attention weight matrix, extracts the linkage between audio clips and semantic keywords, and automatically identifies mapping patterns between speech rate and emotional semantics, and between pitch and keyword emphasis, forming an expression consistency map.

[0233] The fused context vector is fed into the MultimodalFusionEngine module, which concatenates the acoustic features and semantic attention outputs and performs nonlinear fusion. After global pooling and output layer, it generates three structured evaluation indicators: emotion matching, scene restoration, and emotional coherence.

[0234] In the audio standardization assessment system for dynamic analysis of multidimensional acoustic parameters and emotional scenarios described in this paper, the construction of a structured prompt word template is a critical prerequisite for emotional or situational assessment. Its core function is to provide a unified, temporally consistent, and contextually comprehensive semantic guidance benchmark for subsequent multimodal fusion analysis. After receiving the user-defined assessment objective, the standard assessment module first analyzes the user's input in three dimensions: emotion labels (e.g., "angry," "sad," "begging"), character identity parameters (e.g., "young woman," "serious middle-aged man," etc.), and scenario setting information (e.g., "apologizing in public" or "comforting in family conversations").

[0235] After parsing, the system semantically normalizes the input natural language descriptions, using entity recognition and semantic normalization to unify the expressions into a standard set of labels. Each evaluation objective is broken down into separate fields such as emotion category, pragmatic scenario type, character gender, age range, and social identity. This label set ensures the structure of the information and provides a basis for fine-grained content scheduling for downstream tasks.

[0236] Next, the system constructs a prompt word template with a clear hierarchical relationship based on the generated multi-field label set. This template structure typically consists of three layers: the first layer is the global semantic main sentence, which summarizes the semantic objective of the entire assessment task, such as "Please express the emotion of an elderly father gently persuading his family at a family gathering." The second layer is the contextual guide, which clearly expresses the situation and role background, such as "You play the role of a man around 60 years old, giving advice to relatives." The third layer is the local semantic segment, which is designed as a phrase or sentence unit that corresponds to the user's voice performance. The system embeds rhythm control markers (such as "[slow]," "[pause]," and "[emphasis]") and emotion regulation cues (such as "[low tone]," "[gentle tone]"). This information not only guides the user's performance intentions but also provides temporal synchronization anchors and semantic weighting indicators for the multimodal fusion module.

[0237] Once constructed, the system further converts the cue word template into a semantic encoding stream that can be aligned with the audio frame-level timeline. Specifically, the system uses pre-trained semantic embedding models (such as BERT, RoBERTa, etc.) to vectorize each semantic segment and applies a time encoding corresponding to its position in the original speech expression to each segment, thereby forming a semantic encoding stream tensor with time alignment properties. Through this time alignment mechanism, the multimodal fusion analysis engine in the subsequent analysis stage can realize frame-level interaction between acoustic features and semantic features, thereby achieving semantic-led alignment analysis and scoring reasoning when calculating key indicators such as emotional matching, scene restoration, and emotional coherence.

[0238] Through the above-mentioned specific processing flow, the structured prompt word template not only completely covers the target emotions and scenario settings of the evaluation task at the semantic level, but also provides a unified and schedulable representation method at the technical level, significantly improving the evaluation accuracy and interactive intelligence level of the system of the present invention in complex situations.

[0239] The integration processing module 104 is used to summarize the evaluation results generated by the standard evaluation module, generate a structured evaluation report including dimension scores, corresponding question prompts and personalized training suggestions, and transmit it back to the user terminal.

[0240] In this system, the integration and processing module 104 is responsible for integrating, analyzing, formatting, and providing feedback on evaluation results. It serves as the interface between the user and the evaluation system, ensuring a closed loop from input to feedback throughout the evaluation process. This module's core function is to process the multi-dimensional evaluation data output by the standard evaluation module 103, generating a complete, structured evaluation report based on a system-defined template, and accurately transmitting this report back to the user's terminal.

[0241] In its implementation, the integration processing module first receives data output from the standard assessment module. This data includes pronunciation accuracy, phoneme alignment results, and error location for the character-to-phonetic assessment; acoustic feature scores and corresponding classification labels for the basic skills assessment; and matching scores, scene restoration indicators, and coherence evaluation for the emotion and situational assessment. All of this assessment data is input in a structured format, including fields such as dimension identifiers, score values, comment template IDs, and model confidence. The integration processing module parses these structural fields, uniformly categorizes and combines the assessment dimensions, matches them with corresponding natural language explanations, and generates user-friendly explanatory content.

[0242] The system automatically assigns a quantitative grade to each evaluation dimension based on predefined scoring rules. For example, pronunciation accuracy above 90% is labeled "Excellent," 70%-90% is labeled "Fair," and below 70% is labeled "Needs Improvement." Similarly, for dimensions with a low degree of match in the emotion assessment, the system provides specific statements, such as "The tone of voice does not conform to the characteristics of a sad tone. We recommend slowing down the speech rate and increasing the use of low-frequency intonation." Each scoring item is accompanied not only by a numerical value but also an explanatory label to help users understand the deviation in the current voice characteristics.

[0243] To enhance the targeted nature of feedback, the integrated processing module draws on a built-in expert rule library and training suggestion library, combining each assessment score and its contextual relationship to generate personalized training suggestions. For example, if a user's "Oral Resonance" score is low and their "Voice Intensity" fluctuates significantly, the system might output a suggestion: "We recommend increasing mouth opening exercises, trying humming to strengthen front cavity resonance, and combining mirrored pronunciation with control of sound intensity stability." These suggestions are derived from a pre-built label-to-training suggestion mapping table, ensuring that the suggestions are actionable and measurable.

[0244] In addition, the integrated processing module supports multi-level structure output of reports. The system defaults to outputting a standard format report, which includes four parts: score overview, dimensional analysis, error prompts, and improvement suggestions, which is suitable for most users. At the same time, expert mode reports are provided for professional users or teaching scenarios, which include a complete feature index table, model score confidence, error distribution heat map, etc., to support in-depth analysis and comparison. In addition, the module also supports historical record comparison function, which can call the user's past evaluation data and generate a progress curve or trend chart, thereby realizing the tracking and quantification of the user's long-term voice training results.

[0245] Finally, the integration processing module encapsulates the completed assessment report into a visual data structure and transmits it back to the user's device via a local interface or remote terminal. For app or website users, the system presents the report in a combination of graphics and text, including bar charts, radar charts, text descriptions, and summary blocks of suggestions. For API users, report data can be exported in JSON or XML format, facilitating parsing and further processing by third-party systems. The module also supports exporting reports in PDF format for offline use or teacher review.

[0246] Therefore, the integrated processing module not only realizes the standardized processing and humanized presentation of multi-source evaluation data, but also builds a training guidance closed loop through the feedback generation mechanism, giving the entire evaluation system the ability from diagnosis to intervention, and providing users with a path to improve their voice expression ability with practical guidance.

[0247] In the voice standardization evaluation system for dynamic analysis of multi-dimensional acoustic parameters and emotional scenarios described in this paper, the integrated processing module not only organizes and reports the scoring results of various evaluation dimensions, but also designs a structured, executable, and logically closed-loop training feedback path. This path implements an assessment-tracking-training linkage mechanism aimed at improving user expression through precise analysis of current evaluation results, dynamic comparison of historical performance, and guided intervention from an expert knowledge base.

[0248] When the user completes a complete voice evaluation process, the standard evaluation module will output a structured evaluation result, which includes scoring information for multiple dimensions, scores for each item, comment prompts, and the types of expression problems detected by the system. The integration processing module first receives the structured evaluation result, automatically parses and filters out the labels of each dimension whose scores are lower than the set threshold. The threshold can be preset by the system or flexibly set by the user according to his or her goals, such as below 70 points or the lowest score in the top 10%. These dimension labels will constitute the "weakness indicator set" in the current evaluation, which will serve as the starting point for the training guidance mechanism.

[0249] The system will then retrieve historical evaluation records from the local user database and perform a time series comparison on the dimensions corresponding to the current weakness labels. During the comparison process, the system extracts the scoring trajectory of the same dimension in N historical evaluation cycles and generates a trend curve covering the cycle. To ensure the interpretability of the trend, the system will also calculate statistical indicators such as the scoring stability coefficient, fluctuation range, extreme deviation points, and regression slope of each dimension in the cycle, and output a visual trend map. The map will intuitively present the performance trend and the current score to help users understand whether the problem is long-term or a periodic fluctuation.

[0250] After obtaining the trend graph, the integration processing module will call the training suggestion rule library to perform structured matching on the weak dimensions. The rule library is constructed by speech teaching experts and is organized in a ternary structure, namely the mapping path of "dimension label-expression defect pattern-recommended training plan". For example, if a user's "speech speed rhythm" dimension score is low, and the trend graph shows that their score fluctuates frequently and has poor stability, the system will identify it as a "rhythm control disorder" mode and match the corresponding training nodes in the rule library, such as "breathing-stress labeling training", "rhythm recognition training segment", etc. This matching process not only takes into account the consistency of dimensions, but also comprehensively analyzes the slope characteristics of historical scoring trends, the amplitude of score fluctuations and the model confidence results to improve the accuracy and personalization of training suggestions.

[0251] Finally, the integrated processing module embeds the above training program nodes into the user's visual evaluation report in a structured form. The report will display the current weakness label, the specific performance description of the dimension, the typical errors or deficiencies detected by the system, the recommended training goals, and specific training task suggestions that can be directly executed in the form of pictures and texts. For example, if the problem of "unclear resonance position" is detected, the report will provide specific practice suggestions such as "single vowel strong reading training", "cavity control experimental exercises" and operating instructions. All content will be formatted and displayed on the user terminal to ensure that the user can clearly understand the root cause of the problem, the training direction and the execution path, thereby realizing a logical closed loop from evaluation and judgment to targeted training feedback.

[0252] In the voice standardization assessment system described in this article, the training suggestion rule library is one of the core knowledge components of the integrated processing module. After completing the structured assessment, its function is to provide targeted training path recommendations based on the user's performance deficiencies in specific dimensions. This rule library is constructed using a combination of expert experience and data-driven methods. Based on a pre-defined structure, it establishes a ternary binding mapping between assessment labels, expression deficiencies, and recommended training plans, thereby forming a callable, scalable, and inferential training suggestion system.

[0253] The original data sources of the training suggestion rule library include three aspects. First, the data is manually annotated by experts. That is, professionals with a background in speech teaching perform semantic attribution annotation on a large number of historical evaluation records to identify the possible expression defect patterns behind each low-scoring performance. Secondly, the user evaluation logs automatically archived by the system, including structured scoring results, acoustic feature distribution, expression consistency maps and other information. Finally, the training task template dataset collected by the platform covers common and effective voice training methods in the current industry, such as rhythm control training, breath control training, and emotional expression guidance.

[0254] Based on this data, the system first defines standardized evaluation dimension labels and establishes a dimensional index library, where each entry identifies a separate sub-ability of speech expression, such as "pause and continuity rhythm," "breath stability," and "oral resonance focus." Subsequently, historical records with scores below a preset threshold are clustered for defects. By comparing acoustic deviation characteristics with actual user performance and expert feedback, the system abstracts several reusable expression defect patterns, such as "unstable and persistent pronunciation," "insufficient sentence-ending sound," and "rhythm acceleration inconsistent with the target emotion."

[0255] The system pairs the aforementioned dimension labels with expression defect patterns. In the third phase, based on practical feedback from teaching, it associates each defect pattern with one or more training plan nodes. Each training plan node includes the following: the name of the recommended training method (e.g., "Breath Control Exercise A"), the training scenario (e.g., "Reading Complex Sentences with Weak Stress"), the execution method (e.g., "Pausing for 2-second breath control between sentences"), the recommended frequency and period (e.g., "5 times daily for 10 consecutive days"), and the evaluation criteria (e.g., "Rhythm stability score improved by at least 15% after retesting").

[0256] To improve system processing efficiency and the interpretability of rule calls, the aforementioned ternary relationships are stored in a structured format in a graph database. Each node is identified by a unique index, and edge relationships include association weights and applicable scope annotations. At runtime, the system performs shortest path inference based on the user's current weakness labels and matching paths in the defect graph. It selects the training solution nodes with the highest degree of fit as recommended output and presents them in a structured format in the evaluation report.

[0257] The training suggestion rule library described in this invention supports dynamic expansion and online expert updates, and all ternary binding structures are maintained under version management. This mechanism not only effectively enhances the interpretability and pertinence of the system's feedback guidance process, but also builds a stable and repeatable adaptive training closed loop for users, improving the efficiency and accuracy of speech expression improvement.

[0258] In a typical application scenario of the present invention, after the user completes the voice recording on the client, the system will automatically trigger the subsequent evaluation process, forming a complete execution chain from user input to analysis and scoring. In actual deployment, this chain is manifested as a linkage communication mechanism between the front-end, back-end and algorithm services, and calls multiple evaluation subsystems for collaborative processing to achieve a comprehensive evaluation of the user's audio. Please refer to Figure 2 It is a timing diagram of the user uploading voice and triggering the multi-path evaluation process in the sound standardization evaluation system for dynamic analysis of multi-dimensional acoustic parameters and emotional scenes described in the present invention. It shows the interactive logic and data call relationship between the user end, back-end, algorithm end, and the character and pronunciation evaluation system, emotion and scenario analysis system, and basic skills evaluation system.

[0259] First, the user records their voice and uploads it to the backend service. Upon receiving the audio data, the backend system immediately performs a duration check to determine if it exceeds the specified duration for the assessment task and whether there are significant omissions, silence, or blank passages. Compliant audio is temporarily stored and a unique resource locator (URL) is generated for subsequent algorithm calls.

[0260] Next, the backend sends the generated audio URL to the algorithm. Upon receiving the resource, the algorithm performs preliminary processing on the user's audio, including standardization and noise reduction, frame-level slicing, and audio integrity verification. This ensures that subsequent feature extraction and model inputs have consistent time and amplitude scales. During this processing phase, the system assesses whether there are any factors affecting the accuracy of the evaluation, such as recording interruptions, frame skipping, and abnormal background noise, and can provide re-recording recommendations if necessary.

[0261] Subsequently, the system will automatically analyze the type of item to be evaluated based on the user's evaluation configuration and the loaded sample audio content, and trigger the corresponding subsystem to perform special analysis:

[0262] If the content to be evaluated includes indicators related to Mandarin pronunciation, comprehensive pronunciation, or reading skills, the pronunciation evaluation system will be invoked. This system first uses a third-party speech recognition component (such as the iFlytek pronunciation analysis interface) to obtain preliminary phoneme recognition results of the user's pronunciation. It then uses a proprietary algorithm to perform a composite modeling of accuracy, pronunciation fluency, rhythm control, and other dimensions to output a multi-dimensional score.

[0263] If the assessment involves complex tasks such as emotional expression, role-playing, situational interpretation, or voiceover, the system invokes the emotion and situation analysis subsystem. This module, based on the multimodal fusion mechanism described in this invention, comprehensively analyzes the coupling between the acoustic parameters of the user's speech and the semantic information contained in the prompt word template. It then calculates scores across dimensions such as emotional matching, scene reproduction, and emotional coherence to determine whether the user's expression aligns with pre-set goals.

[0264] If the audio involves tasks related to fundamental vocal skills assessment, such as loudness, hollowness, breath stability, and chest resonance, the system invokes the fundamental skills assessment module to infer and judge the user's audio based on the trained classification model. The system extracts the dominant acoustic channel relevant to the task and applies the feature screening and weighting strategies proposed in this paper. Finally, the system calculates the output probability distribution of the classification model and combines it with the positive sample scoring criteria to generate a final score.

[0265] The evaluation results of all subsystems will be summarized in the integrated processing module and formed into a structured scoring report. After completing the multi-dimensional analysis of the user's voice input, it will be sent back to the front-end interface so that users can view the evaluation results, understand the deficiencies, and receive targeted training suggestions.

[0266] Through the orderly linkage and division of labor among the above modules, this system has built a highly scalable evaluation process framework, which not only realizes end-to-end voice input processing, but also reflects the professionalism, real-time and intelligent level of voice evaluation in multiple dimensions.

[0267] In the sound standardization evaluation system for dynamic analysis of multi-dimensional acoustic parameters and emotional scenes described in the present invention, in order to further improve the feature accuracy and model adaptability of the basic skill evaluation dimension, the system integrates a multi-stage analysis mechanism for acoustic signal features within the basic skill evaluation system. Its core process includes key steps such as signal preprocessing, fundamental frequency and resonance peak extraction, perturbation calculation, energy weighted analysis, and correlation matrix modeling. Please refer to Figure 3 This is a heat map of the correlation matrix between acoustic features in the sound standardization assessment system described in this invention. This map displays the correlation between various acoustic parameters (such as fundamental frequency, formant, perturbation, sound intensity, and spectral center) used in basic performance assessment. Colors range from blue to red, indicating the strength of the correlation, from negative to positive. This provides a reference for feature engineering and model feature selection.

[0268] First, the system pre-emphasizes the user's input audio signal to enhance high-frequency features and suppress low-frequency interference. It then performs framing, dividing the continuous speech into multiple 20-millisecond short frames and performing overlapping processing. Based on an autocorrelation algorithm, the system extracts the fundamental frequency of each frame. Combined with cepstral analysis, it extracts formant information and separates the source-filter model. Furthermore, the system calculates the relative difference in periodic variations between frames to determine fundamental frequency jitter and amplitude shimmer. Combined with time-domain energy fluctuations, this yields a complete set of acoustic parameters.

[0269] In order to analyze the relationship between various parameters in different evaluation dimensions, the system establishes a correlation matrix between multi-dimensional acoustic features and visualizes it as a heat map, such as Figure 3 This figure shows the pairwise correlation coefficients of various parameters, including fundamental frequency (f0), resonance peaks (f1, f2tr, etc.), energy, jitter, shimmer, spectral center (gravity), and comprehensive perceptual features (such as energy_ratio). It reflects the degree of coupling between features and the collinearity risk that may be caused during modeling.

[0270] By analyzing the correlation structure in the heat map, the system summarizes the core acoustic feature groups that different basic skill evaluation items rely on. For example:

[0271] The sound brightness / darkness evaluation depends on sound intensity, fundamental frequency, harmonic-to-noise ratio, and energy proportion;

[0272] Oral resonance evaluation is highly dependent on the center frequency and energy distribution of the spectrum;

[0273] The evaluation of virtual voice and breath stability is mainly based on fundamental frequency perturbation and amplitude perturbation;

[0274] The evaluation of chest resonance and loudness depends more on the position of formant, sound intensity and perturbation index;

[0275] The evaluation of speech rate control and rhythm smoothness requires comprehensive modeling based on short-term energy fluctuations and inter-frame frequency jumps.

[0276] After constructing the correlation features, the system employed a data-driven strategy to screen feature combinations and conduct multiple rounds of model training on over 28,000 training samples annotated by professional teachers. Mainstream machine learning algorithms, including support vector machines (SVMs), random forests, gradient boosting, logistic regression, multi-layer perceptrons (MLPs), and decision trees, were used for different evaluation items. The accuracy, recall, F1 score, and AUC metrics of each model were evaluated, and the optimal performing model structure was automatically selected for binding. The measured results are as follows:

[0277] Virtual voice, strong breath control, weak breath control, and dark voice: The support vector machine model performed best, with an accuracy range of 82.45%–91.13%;

[0278] Strong voice and chest resonance: using the random forest model, the accuracy rates were 88.89% and 91.00% respectively;

[0279] Oral resonance: Using gradient boosting algorithm, the accuracy rate reaches 83.00%;

[0280] Sound brightness: The logistic regression model achieved a high accuracy of 94.35%;

[0281] For real sound and weak sound, a multilayer perceptron and a decision tree were used, with accuracy rates of 91.14% and 82.64%, respectively.

[0282] In terms of emotion and situational analysis, the system further divides tasks into three categories: single emotion, situational interpretation, and role-playing. It also builds supporting prompt word template generation mechanisms, multimodal fusion modeling paths, and personalized evaluation algorithms for each.

[0283] The single emotion assessment covers emotional intensity and 15 specific emotions (such as joy, anger, sadness, and fear). Emotional intensity is classified using logistic regression based on acoustic features, achieving an accuracy rate of 84.47%. Other emotions are matched using a cue word template and multimodal audio comparison mechanism, achieving an accuracy rate between 88% and 95%.

[0284] The scenario interpretation assessment allows users to specify specific scenarios and target emotions, with the system dynamically generating structured prompt word templates. The fusion module outputs indicators across three dimensions: emotional matching, scene restoration, and emotional coherence, with an overall accuracy rate of 87%.

[0285] The role-playing assessment superimposes role setting parameters (age, personality, etc.) on the basis of scenario interpretation, introduces a feature coupling mechanism into the fusion model, combines voice and text to jointly model and evaluate role suitability, with an actual measured accuracy rate of 85%-90%.

[0286] Although the present application is disclosed as above with the preferred embodiments, it is not intended to limit the present application. Any person skilled in the art may make possible changes and modifications without departing from the spirit and scope of the present application. Therefore, the scope of protection of the present application shall be based on the scope defined by the claims of the present application.

Claims

1. A sound standardization evaluation system for dynamic analysis of multi-dimensional acoustic parameters and emotional scenes, characterized by: include: Audio acquisition module, used to collect user voice audio and generate raw audio data; An audio processing module, configured to pre-process the raw audio data to obtain pre-processed audio data; wherein the pre-processing includes noise reduction, sampling rate adjustment, pre-emphasis, framing, and slicing operations; The standard evaluation module is configured to determine an evaluation type according to a user setting; receive the pre-processed audio data and perform an evaluation operation corresponding to the evaluation type, wherein the evaluation operation includes at least one of the following operations: Recognizing the speech units in the preprocessed audio data, calculating the accuracy of the linguistic features of initials, finals, and speech flow sound changes, and generating a word pronunciation evaluation result; Extracting acoustic features from the preprocessed audio data, including fundamental frequency, formant, sound intensity, fundamental frequency perturbation, and amplitude perturbation, and inputting the features into machine learning models trained for different evaluation items for classification and prediction to generate sound basic skills evaluation results; Based on the user's emotions or situational information, a prompt word template is constructed, and a multimodal fusion analysis is performed on the pre-processed audio data and the situational text, outputting emotion matching, scene restoration, and emotion coherence indicators to generate an emotion evaluation result; The integration processing module is used to summarize the evaluation results generated by the standard evaluation module, generate a structured evaluation report, which includes dimension scores, corresponding question prompts and personalized training suggestions, and transmit it back to the user terminal.

2. The sound standardization evaluation system for multi-dimensional acoustic parameters and emotional scene dynamic analysis according to claim 1 is characterized in that: When the standard evaluation module performs the basic voice evaluation operation, it performs the following feature selection process for each preset evaluation item: Based on the scoring dimensions of the evaluation items, the collected user samples and the labeled teacher samples are grouped and processed to construct a control sample set covering performance at all levels. Based on the control sample set, a set of acoustic parameters related to vocal stability, spectral variation pattern, and energy distribution structure are extracted. The acoustic parameter set includes: fundamental frequency perturbation, amplitude perturbation, spectral density center of gravity position, and non-steady-state energy fluctuation amplitude; Performing inter-group difference enhancement analysis on the acoustic parameter set, generating feature response maps using a convolutional structure based on a learnable weight kernel, calibrating feature change patterns that have significant discriminatory power for different levels of performance, and constructing feature-sensitive subspaces corresponding to the evaluation items; In the feature-sensitive subspace, the interference risk of candidate features relative to feature overlap in other evaluation items is evaluated, and the feature weights are adjusted by introducing an overlap suppression factor to screen out a target feature group with high discrimination and minimal interference, which is input as the final feature corresponding to the evaluation item into the trained exclusive classification model for evaluation and judgment.

3. The sound standardization evaluation system for dynamic analysis of multi-dimensional acoustic parameters and emotional scenes according to claim 1 is characterized in that: When performing the basic voice evaluation operation, the standard evaluation module sequentially performs the following steps for each preset evaluation item to dynamically determine the optimal classification model bound to the evaluation item: Based on a set of control samples covering all rating levels of the evaluation item, extract the distribution response map of the acoustic feature channels and analyze the channel response gradients between different levels to obtain the dominant acoustic channel set corresponding to the evaluation item; Based on the dominant acoustic channel set, the response stability of each candidate model structure on the channel set is evaluated, and model types with nonlinear overfitting tendencies or channel drift sensitivity on the dominant channel are eliminated to form a structural adaptability screening result; The models retained in the structural adaptability screening results are deployed on the historical sample set of the evaluation item for cross-validation, and a unified evaluation index is calculated based on the accuracy, recall rate, F1 score and area under the curve. The model with the highest evaluation index is used as the final classification model and bound to the evaluation item for subsequent sample evaluation and scoring.

4. The sound standardization evaluation system for dynamic analysis of multi-dimensional acoustic parameters and emotional scenes according to claim 1 is characterized in that: When the standard evaluation module performs the sentiment evaluation operation, it sequentially performs the following steps to implement multimodal fusion path construction and evaluation calculation: Synchronously aligning the pre-processed audio data with a structured prompt word template, where the structured prompt word template is automatically generated based on the target emotion, scene description, or character setting selected by the user and includes a semantic guide word sequence and rhythm control markers. The system converts the structured prompt word template into a time-aligned semantic encoding stream using a segment semantic embedding mechanism; The semantic encoding stream, reference demonstration audio, and pre-processed audio data are input into a multimodal fusion engine. The fusion engine uses a cross-modal attention mechanism to extract the dynamic coupling pattern between pronunciation intensity, pitch change, speaking rate rhythm, and keyword semantic emphasis based on the mutual attention weight matrix between acoustic frame features and semantic embedding sequences, and construct an expression consistency map. Based on the expression consistency map, multidimensional intermediate indicators including local inconsistency scores, emotional channel missing marks and semantic drift degrees are extracted, and the global scene constraint information is integrated to calculate the final evaluation results of the three dimensions of emotional matching, scene restoration and emotional coherence, which are used to form structured scoring feedback under the emotional dimension.

5. The sound standardization evaluation system for dynamic analysis of multi-dimensional acoustic parameters and emotional scenes according to claim 1 is characterized in that: When performing the sentiment evaluation operation, the standard evaluation module performs the following steps to construct a structured prompt word template based on the evaluation target set by the user: Receive and parse the target emotion tag, character identity parameters, and scene setting information input by the user, and map them into a multi-field tag set through semantic normalization. The tag set includes emotion category, pragmatic scene type, character gender, age range, and social identity. Based on the multi-field tag set, a structured prompt word template with a hierarchical nested structure is constructed. The structured prompt word template includes: a global semantic main sentence defining the task goal, a contextual guide carrying role attribute constraints, and a local semantic segment for indicating expression details; wherein, the local semantic segment is embedded with rhythm control markers and emotion regulation prompts to support multimodal alignment processing; The structured prompt word template is encoded into a time-aligned semantic coding stream, and is bound to the target speech sample in the frame-level time dimension through a position embedding mechanism, so that it can be called by the multimodal fusion analysis engine, thereby supporting the associative reasoning of expression consistency judgment and context indicator scoring.

6. The sound standardization evaluation system for dynamic analysis of multi-dimensional acoustic parameters and emotional scenes according to claim 1 is characterized in that: In the process of generating a structured evaluation report, the integration processing module sequentially performs the following steps for each evaluation dimension involved in the current evaluation result to construct a training feedback closed-loop path: Receive and parse the current structured evaluation results output by the standard evaluation module, extract all dimension labels with scores below the preset threshold, form a set of targeted training indicators, and compare them item by item at the dimension level with the historical evaluation records stored in the local database; Based on the comparison results, the progress archiving module is called to generate a quantitative training curve covering the last N evaluation cycles. The training curve reflects the time series score change trend of each indicator. At the same time, the stability coefficient and fluctuation range of each dimension are calculated to form a visual user performance trend map; Matching the trend graph with a training suggestion rule library, which is built based on the experience of speech teaching experts and adopts a three-way binding mechanism of "dimension label - expression defect pattern - recommended training plan". Through graph structure alignment and defect pattern recognition, it locates the expression shortcomings of the current user that are most worth improving and selects several personalized training plan nodes with the highest matching degree. The training program node is embedded in the final evaluation report in a structured manner, and the five-dimensional feedback results including the current weakness label, specific performance description, typical problem prompts, recommended training goals and specific training task suggestions are displayed on the user terminal, thus forming a complete feedback closed-loop path from evaluation judgment, history tracking, trend modeling to training guidance.

Images