Voice quality objective evaluation method based on nonlinear regression improvement

Abstract

The invention relates to a voice quality objective evaluation method based on nonlinear regression improvement, and the method comprises the steps: carrying out the preprocessing of an original voice signal and a distorted voice signal, and carrying out the time alignment; performing auditory conversion on the aligned signals to obtain auditory feature representation, and performing jitter processing; inputting the features subjected to jitter processing into a cognitive model to obtain a preliminary voice quality score; wherein the cognitive model is constructed based on an auditory psychology experiment result; and performing nonlinear regression correction on the initial voice quality score to obtain a final voice quality score. According to the method, a noise reconstruction and nonlinear regression module is added on the basis of the PESQ model, the predicted voice quality is remapped, experiments are performed on a plurality of data sets with or without noise, compared with an original PESQ model, the prediction score accuracy is improved by 82.5% at most, an experiment result proves that the model can effectively improve the prediction quality, and the prediction efficiency is improved. And the voice synthesis work can be more effectively assisted.

Description

Technical Field

[0001] The present invention relates to the technical field of speech quality assessment, and particularly to an objective speech quality assessment method improved based on non-linear regression. Background Art

[0002] Speech, as one of the most important communication methods, has always been the focus of attention in the field of human-computer interaction. At the same time, most of the multimodal human-computer interaction systems on the market use user speech information as operation instructions. Different from pictures and statements, speech quality cannot be directly distinguished, so it is necessary to quantify speech to obtain evaluation indicators. And in the whole chain from laboratory to industrial implementation, speech evaluation is an important criterion. Speech Quality Assessment (SQA) has also become a core issue in technologies such as communication and speech generation. For example, in an online meeting, the speech quality (such as latency, noise suppression effect) can be monitored in real time and the codec parameters can be dynamically adjusted. Similarly, for the performance optimization of speech synthesis (TTS), voice conversion (VC) and other systems, automatic evaluation of the naturalness of the generated speech is required. On the other hand, for hearing aids, noise-canceling headphones, and the repair of old audio, speech evaluation is also needed to balance and optimize algorithms and user experience. Currently, the most recognized speech evaluation indicator is the Mean Opinion Score (MOS) of speech naturalness. However, MOS scores are usually obtained by multiple professionals or testers evaluating the quality of audio, video, or other multimedia content based on subjective feelings under the same conditions. These professionals generally include audio engineers, video engineers, telecommunications experts, etc. They will consider various factors when scoring. For example, when evaluating audio quality, factors such as clarity, noise, and echo will be considered, and when evaluating video quality, factors such as clarity, contrast, brightness, and color will be considered. Moreover, a piece of audio will be scored by multiple experts, and then the average score will be taken, mapping low-level audio features to different score labels. The MOS has the following scoring criteria: 5 means excellent naturalness, clear, almost no latency; 4 means slightly poor naturalness, audible, small latency; 3 means acceptable naturalness, basically audible, with latency; 2 means unsmooth naturalness, not very audible, large latency; 1 means extremely poor naturalness, inaudible, very large latency. Obviously, such a process will seriously slow down the experimental development progress and have high economic costs, and at the same time, there are deficiencies such as spending a lot of energy and having a lot of subjective evaluation components. Therefore, an effective, high-speed, objective and non-invasive indicator is very necessary.

[0003] Speech quality evaluation can be divided into two main categories from the perspective of the evaluation subject: subjective evaluation and objective evaluation. Subjective evaluation methods are time-consuming and laborious, but have high reliability. Objective evaluation methods are simple and fast, but have relatively low accuracy. Subjective evaluation is represented by MOS. The ultimate goal of objective evaluation methods is to make the evaluation results output by machines close to the subjective MOS value of speech quality. At the same time, from a technical architecture perspective, it can also be divided into: non-intrusive metrics and intrusive metrics. Intrusive speech scoring models require the input of the speech to be scored and a clean reference speech, while non-intrusive speech scoring models only require the input of the speech to be scored. Each of these two methods has its own advantages and disadvantages.

[0004] Intrusive metrics can also be called reference-based evaluation metrics. Typically, it is to predict speech scores based on the input and output speech signals. Among them, PESQ is based on this aspect and is combined with a psychoacoustic model to predict the end-to-end perceptual speech quality. PESQ uses an original speech signal and a noisy speech signal adjusted to a specific reference level, and aligns the time series at the same time, and then outputs it to a model that simulates the auditory characteristics of the human ear for scoring. Unfortunately, the latest algorithm of ITU is not open source, and the current PESQ effect is already a bit too backward with low accuracy.

[0005] This invention is an improvement based on the intrusive metric PESQ. Aiming at the defect of the existing PESQ with excessive errors, a method for objectively evaluating speech quality based on non-linear regression improvement is proposed. Summary of the Invention

[0006] The purpose of this invention is to propose a method for objectively evaluating speech quality based on non-linear regression improvement to solve the problems existing in the above-mentioned prior art. This invention proposes a model PESQ_PRE based on PESQ. This model first conducts noise model research on the input dual-channel audio of the original PESQ perceptual speech quality evaluation model. Through experiments, it is proved that the prediction accuracy of PESQ for different noises is different. Therefore, noise reconstruction is performed on the dual-channel speech of PESQ. At the same time, based on the output result, a non-linear regression module is added to better estimate the input speech quality.

[0007] To achieve the above purpose, this invention provides the following solutions:

[0008] A method for objectively evaluating speech quality based on non-linear regression improvement, including:

[0009] After preprocessing the original speech signal and the distorted speech signal respectively, perform time alignment;

[0010] Convert the aligned signals into auditory feature representations through auditory conversion respectively, and perform jitter processing;

[0011] Input the jitter-processed features into the cognitive model to obtain a preliminary speech quality score; wherein, the cognitive model is constructed based on the results of auditory psychology experiments;

[0012] Correct the preliminary speech quality score to obtain the final speech quality score.

[0013] Optionally, the preprocessing of the original speech signal includes:

[0014] Adjust the level of the original speech signal and perform filtering processing.

[0015] Optionally, the preprocessing of the distorted speech signal includes:

[0016] Perform noise reconstruction on the distorted speech signal to generate noise related to the original speech signal;

[0017] Add the noise to the original speech signal to generate a noisy audio signal;

[0018] Adjust the level and perform filtering processing on the noisy audio signal;

[0019] The noise reconstruction is:

[0020] y = f × max(|x(t)|)

[0021] wherein, x(t) is the input audio signal, max is to take the maximum value in the audio signal, y represents the noise level, and f is the set percentage.

[0022] Optionally, the time alignment of the preprocessed signal includes:

[0023] Detect the silent time in the preprocessed signal;

[0024] Using the envelope correlation method, calculate the rough delay estimate of the signal after detecting the silent time, and eliminate the rough delay;

[0025] Using the weighted histogram method, perform fine delay and confidence estimation on the signal after eliminating the rough delay, and use the methods of fixed delay alignment and dynamic alignment to align the signal.

[0026] Optionally, the auditory conversion of the aligned signal includes:

[0027] Perform fast Fourier transform on the aligned signal, group it according to the Bark scale, and extract spectral features; map the frequency of the speech signal to a non-linear pitch scale that conforms to the human ear perception characteristics;

[0028] Compare the spectral differences between the original frame and the degraded frame; convert the physical intensity of the speech signal into a non-linear scale that conforms to the subjective loudness perception of the human ear to obtain an auditory feature representation.

[0029] Optionally, inputting the jitter-processed features into the cognitive model includes:

[0030] Determine and count the bad intervals of the jitter-processed features to obtain a speech interference value;

[0031] Realign the bad intervals;

[0032] Quantify the local degradation degree of the speech quality by introducing the number of bad intervals;

[0033] Aggregate the speech interference value using the L6 norm;

[0034] Segment the aggregated speech signal into preset short-time windows, and within the second-minute interval of each preset short-time window, aggregate the frame-level interference values using the L2 norm to generate a second-minute interference index;

[0035] Combine the second-minute interference index and the aggregated speech signal to generate a score through linear combination.

[0036] Optionally, the method for correcting the preliminary speech quality score is: using a non-linear regression equation.

[0037] The beneficial effects of the present invention are:

[0038] The present invention proposes an objective speech quality evaluation method based on non-linear regression improvement. First, after preprocessing the original speech signal and the distorted speech signal respectively, perform time alignment; secondly, perform auditory conversion on the aligned signals respectively to convert them into auditory feature representations and perform jitter processing; then input the jitter-processed features into the cognitive model to obtain a preliminary speech quality score; wherein, the cognitive model is constructed based on the results of auditory psychology experiments; finally, perform non-linear regression correction on the preliminary speech quality score to obtain the final speech quality score. The present invention adds two parts, noise reconstruction and non-linear regression, to the PESQ model, remaps the predicted speech quality, and experiments with and without noise and on multiple datasets. Compared with the original PESQ model, the prediction score accuracy is improved by up to 82.5%. The experimental results prove that this model can effectively improve the prediction quality and can more effectively assist speech synthesis work. Description of the Drawings

[0039] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the accompanying drawings required in the embodiments. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can also be obtained based on these drawings.

[0040] Figure 1 PESQ prediction score and MOS for the embodiment of the present invention;

[0041] Figure 2 PESQ flow chart for the embodiment of the present invention;

[0042] Figure 3 Perceptual model flow chart for the embodiment of the present invention;

[0043] Figure 4 Cognitive model flow chart for the embodiment of the present invention;

[0044] Figure 5 Noise level and MOS for the embodiment of the present invention;

[0045] Figure 6 MOS value and error of PESQ distance from MOS for the embodiment of the present invention;

[0046] Figure 7 Cubic fitting result for the embodiment of the present invention;

[0047] Figure 8 An objective speech quality evaluation method based on improved non - linear regression for the embodiment of the present invention. Specific embodiments

[0048] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, rather than all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0049] To make the above - mentioned objects, features, and advantages of the present invention more obvious and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0050] In this embodiment, since generative models are very popular nowadays, for example, large models such as ChatGPT and DeepSeek are text-generation large models. Similarly, large speech synthesis models play a very important role in both human-computer interaction and audio-visual entertainment. However, the MOS of speech quality metrics requires a large number of professionals to listen and score, so there is an urgent need for effective objective speech evaluation metrics. In response to this situation, a noise reconstruction and non-linear regression module is added to the PESQ model to remap the predicted speech quality. Experiments are conducted with and without noise and on multiple datasets. Compared with the original PESQ model, the prediction score accuracy has increased by up to 82.5%. The experimental results prove that this model can effectively improve the prediction quality and can more effectively assist speech synthesis work.

[0051] PESQ (Perceptual Evaluation of Speech Quality) is a standardized objective evaluation algorithm for speech quality proposed by the International Telecommunication Union ITU-T in 2001. It belongs to the subsequent version of ITU-T P.862 and aims to simulate the human ear's perception of speech quality through an algorithm. It receives a reference signal and a distorted signal and then outputs a score to replace the traditional time-consuming subjective test MOS. Compared with traditional speech quality evaluation methods such as SNR and STOI, it has significant advantages in simulating human subjective auditory perception. The PESQ algorithm is improved based on the psychoacoustic model. The research team found during the optimization process that traditional frequency-domain distortion metrics are not sensitive enough to complex speech impairments such as codec artifacts and intermittent noise. Therefore, by introducing auditory masking effect modeling and time-frequency domain alignment technology, the consistency between the evaluation results and the artificial subjective score MOS has been significantly improved. The improved PESQ algorithm has better scoring accuracy and stability than the early models in scenarios such as speech communication and noise reduction algorithms.

[0052] The main process of the PESQ model is as follows:

[0053] After inputting the original reference speech signal (original input) and the degraded speech signal (degraded output) into the Perceptual Model, preprocessing, time alignment, and amplitude normalization are first performed to eliminate delay differences (delay estimates) and amplitude offsets, detect silent segments, and retain the effective speech region to ensure that the signals are on the same basis. Then, the core algorithm module is input. In the model, the original input and the degraded output will be converted into the internal representation of the auditory system (Internal representation of original). Then, the difference calculation (Difference in Internal Representation) is performed internally to quantify the differences in perceptual features between the original signal and the degraded signal, such as time-domain differences and frequency-domain differences.

[0054] Finally, the Cognitive model will simulate the comprehensive judgment of human beings on speech quality, map the perceptual differences into the final quality score (quality). The higher the score, the better the speech quality and the closer it is to the original signal. The overall process is as Figure 2 shown.

[0055] Perceptual model:

[0056] The perceptual model is constructed based on psychoacoustic principles. As Figure 3 shown, the innovation of PESQ lies in: mapping the physical signal differences to the human auditory sensitivity, such as higher weights for low-frequency distortions; simulating the short-term memory characteristics of the auditory system through sliding windows and non-linear accumulation; and considering both spectral distortion and temporal continuity, making it closer to the actual listening experience. The following are the core algorithms and steps:

[0057] 1. Time-frequency analysis and frame-level distortion calculation: The Bark scale shows that the frequency resolution of the human ear is finer in the low-frequency band than in the high-frequency band. Therefore, the fast Fourier transform is performed on each frame of the preprocessed speech signal, grouped according to the Bark scale, and then the spectral features are extracted; the frequency of the speech signal is mapped to a non-linear pitch scale that conforms to the human ear's perceptual characteristics to ensure uniform perception. Secondly, the spectral differences between the original frame and the degraded frame are compared; then the physical intensity (such as decibels) of the speech signal is converted into a non-linear scale that conforms to the subjective loudness perception of the human ear. This process ensures that the speech quality assessment is closer to the subjective listening experience by simulating the non-linear response of the human auditory system to sound intensity.

[0058] 2. Asymmetry processing: Apply symmetry constraints to the time domain and frequency domain of the signal to improve the robustness of the processing results.

[0059] Emphasizing Silent Parts: Deliberately enhance the salience of silent or low-energy segments to make it easier to detect whether the segments are natural and for analysis.

[0060] The cognitive model is as Figure 4 shown;

[0061] The cognitive model is also designed based on the results of auditory psychology experiments. It receives the audio signals output by the perception model, and then performs bad interval determination and counting. This step detects consecutive highly interfering frames and marks them as bad intervals to reduce latency (set the frame interference value to zero) and realign the bad intervals. Then, the degree of local degradation of speech quality is quantified by introducing the number of bad intervals. After that, the L6 norm is used to aggregate the speech interference values. Secondly, the speech signal is segmented into short-time windows, which are 320 ms long with 50% overlap, to mimic the short-term memory characteristics of the human ear for instantaneous distortion. At the same time, within each second-minute interval, the L2 norm is used to aggregate the frame-level interference values to generate the second-minute interference index. Finally, combining the second-minute and global aggregation results, the PESQ score is generated through linear combination.

[0062] The realignment of bad intervals includes:

[0063] Local cross-correlation calculation: Recalculate the maximum cross-correlation coefficient between the absolute original signal and the degraded signal within the bad interval to estimate the new delay value.

[0064] Delay compensation: Apply the new delay value to perform time shift compensation on the degraded signal:

[0065] After the realignment of bad intervals:

[0066] Recalculate frame interference: If the maximum cross-correlation coefficient meets the standard (higher than the threshold), recalculate the frame interference value (D) for each frame within the bad interval.

[0067] Replacement strategy: If the new frame interference value is less than the original value, replace the original frame interference value to generate the optimized D' and DA' (the frame interference parameters after alignment).

[0068] This embodiment improves the PESQ algorithm as follows:

[0069] This embodiment makes improvements based on the intrusive metric PESQ. Aiming at the defect of excessive error in the existing PESQ, it is found that there is a certain non-linear relationship between its error and MOS. Using the strategy of remapping with a non-linear regression algorithm improves the accuracy of the algorithm results. And PESQ requires the input of a degraded signal, and this degraded signal has various parameters, among which the length of the noise frame, the noise level, etc. are relatively important. Through experiments, it is found that the intensity of the noise signal will affect the prediction score in a non-linear law. For example Figure 2As shown, an audio and its degraded signal are selected. The score predicted by PESQ is significantly affected by the noise level and there is a non-linear relationship. The built-in cognitive model of PESQ has an obvious bias in the processing and recognition of noise. It is not the case that the better the prediction is when the noise is either greater or smaller. On the contrary, if the original signals are input at both ends, PESQ will be unable to extract the features of the noise, resulting in poor robustness and prediction effect. In this case, the predicted score will fluctuate around 4.6, which may be due to the lack of detailed analysis of a certain type of noise, leading to insufficient generalization ability of the model and the easy loss of effective signals when removing noise data. On the contrary, under extremely strong noise interference, it will also give an overly high speech prediction score.

[0070] Through research, the model can greatly improve the accuracy at the cost of a small amount of time. At the same time, compared with other neural network prediction models, there are still significant advantages in terms of speed and deployment cost. Moreover, thanks to the design of extracting signal features at both ends, it can effectively learn the noise features and has good anti-noise ability. The following is the model and its implementation principle. The optimized model process is as Figure 8 shown.

[0071] An objective speech quality evaluation method based on non-linear regression improvement proposed in this embodiment specifically includes:

[0072] After preprocessing the original speech signal and the distorted speech signal respectively, perform time alignment;

[0073] Convert the aligned signals into auditory feature representations through auditory conversion respectively, and perform dithering processing;

[0074] Input the features after dithering processing into the cognitive model to obtain the preliminary speech quality score; wherein, the cognitive model is constructed based on the results of auditory psychology experiments;

[0075] Perform non-linear regression correction on the preliminary speech quality score to obtain the final speech quality score.

[0076] Further, the preprocessing of the original speech signal includes:

[0077] Adjust the level of the original speech signal and perform filtering processing.

[0078] Further, the preprocessing of the distorted speech signal includes:

[0079] Perform noise reconstruction on the distorted speech signal to generate noise related to the original speech signal;

[0080] Add the noise to the original speech signal to generate a noisy audio signal;

[0081] Perform level adjustment and filtering on the noisy audio signal;

[0082] The noise is reconstructed as:

[0083] y = f × max(|x(t)|)

[0084] where x(t) is the input audio signal, max is to extract the maximum value in the audio signal, y represents the noise level, and f is the set percentage.

[0085] Specifically, in this embodiment, the distorted speech signal is subjected to noise reconstruction as Figure 5 shown;

[0086] The core of the intrusive metric PESQ design lies in the dual - end input audio file, and it also has a model built - in with anthropomorphic ear auditory characteristics. Degraded audio always contains noise, and solving the noise problem has always been an important challenge. PESQ chooses to learn features from the noise to obtain strong robustness in a noisy environment. However, this does not mean that the impact of noise can be avoided. As Figure 2 shown, under different noise level conditions, the scores predicted by PESQ show a non - linear function curve. Certain conditions are required to obtain an ideal prediction result.

[0087] Therefore, a noise reconstruction module is added. By analyzing the audio file, the amplitude of the generated noise is scaled proportionally to the maximum amplitude of the audio signal. Then it is superimposed on the original audio file.

[0088] The following is the method for generating noise.

[0089] y = f × max(|x(t)|)(1)

[0090] where x(t) is the input audio signal, max is to extract the maximum value in the audio signal, y represents the noise level, and f is the set percentage.

[0091] It is calculated from the maximum value of the input audio signal and the percentage, and is used to control the intensity of the noise. After generating the noise, by adjusting the noise amplitude and adding it to the original audio signal, a noisy audio signal is finally obtained. This method is often used in noise addition experiments or tests in audio processing.

[0092] Furthermore, for the pre - processed signal, time alignment includes:

[0093] Detect the silent time in the pre - processed signal;

[0094] Using the envelope - related method, calculate the rough delay estimate of the signal after detecting the silent time, and eliminate the rough delay;

[0095] Using the weighted histogram method, perform fine delay and confidence estimation on the signal after eliminating the coarse delay, and adopt the methods of fixed-delay alignment and dynamic alignment for signal alignment.

[0096] Among them, the silent segment is used as the time anchor point. For example, speech segment a + silence + speech segment b and speech segment a + cut + speech segment b are not detected, which may lead to incorrect matching of speech segments a and b. At the same time, in the perception model, the perception weight of the non-silent segment will be increased.

[0097] Among them, fixed-delay alignment: If the confidence is high, directly move the whole signal. Dynamic alignment: If the delay changes over time (such as signal segment estimation), use interpolation or dynamic time warping (DTW) for per-segment alignment.

[0098] Specifically, in this embodiment, for time alignment, first define the silent interval:

[0099] For the purpose of time alignment, a utterance is defined as a continuous speech segment with a duration of at least 300 ms, which does not contain more than 200 ms of silent time. This recognition is performed on the reference signal by a voice activity detector (VAD), which has an adaptive threshold to make the speech / non-speech decision robust to noise.

[0100] Then use the envelope correlation method to calculate the coarse delay estimate of the whole signal. This allows large time offsets, which are eliminated due to continuous delay or poor synchronization during the measurement. Then, the envelope correlation method is applied to the corresponding parts of the reference file and the degraded file for each utterance to identify the coarse delay of each utterance.

[0101] Envelope: Calculate the decimated envelopes of the reference signal and the degraded signal by measuring the power of the filtered signal on 4-ms non-overlapping frames. Take the position of the maximum cross-correlation between the envelopes as the rough estimate of the delay.

[0102] Adopt the weighted histogram method for fine delay and confidence estimation. Finally, the utterance boundaries are placed in the middle of each silent period, or, for the first and last utterances, at the start and end of the file. Since each utterance is aligned separately, this method directly takes into account the delay changes during the silence period. Weighted Histogram Method

[0103] Frame processing: The signal is divided into 64-ms, 75% overlapping Hanning window frames.

[0104] Per-frame delay estimation: Calculate the cross-correlation of the reference and distorted signals for each frame, and record the delay value of the maximum correlation point.

[0105] Weighted histogram construction: The delay estimation weight for each frame is the 0.125th power of the maximum correlation value (emphasizing high-energy frames). The delay distributions of all frames are statistically analyzed to generate a weighted histogram.

[0106] Smoothing and peak detection: After the histogram is smoothed with a triangular kernel, the peak is taken as the final delay estimate.

[0107] The delay estimation for the entire utterance cannot account for the delay variations during the speech process, i.e., the delay of the entire speech segment is dynamically changing.

[0108] The utterance alignment method is extended as follows:

[0109] Each utterance is divided into two parts, and each part goes through the same coarse / fine delay estimation stages as before. This is repeated at a large number of division points.

[0110] If there is evidence of delay variation - an increase in confidence and an absolute change in delay greater than or equal to 4 milliseconds - select the division that yields the maximum confidence.

[0111] In this case, the utterance is divided into two parts at this point. Since multiple delay variations may occur within each utterance, the process is recursively applied to each new half before proceeding with the remaining utterance.

[0112] Furthermore, the auditory transformation of the aligned signal includes:

[0113] Performing a fast Fourier transform on the aligned signal, grouping it by the Bark scale, and extracting spectral features; mapping the frequency of the speech signal to a non-linear pitch scale that conforms to the human ear's perception characteristics;

[0114] Comparing the spectral differences between the original frame and the degraded frame; converting the physical intensity of the speech signal to a non-linear scale that conforms to the human ear's subjective loudness perception to obtain an auditory feature representation.

[0115] Mapping the aligned signal to the human ear-sensitive perceptual domain, which is core to simulating the non-linear characteristics of the auditory system through Bark band analysis, loudness compression, and time-frequency masking. The transformed signal is represented as a two-dimensional time-frequency matrix.

[0116] Furthermore, inputting the features after jitter processing into the cognitive model includes:

[0117] Determining and counting the bad intervals of the features after jitter processing;

[0118] Quantifying the local degradation degree of the speech quality by introducing the number of bad intervals;

[0119] Using the L6 norm to globally aggregate the speech interference values;

[0120] The speech signal is segmented into short-time windows. Within each window interval, the frame-level interference values are aggregated using the L2 norm to generate a second-minute interference metric. Finally, combining the window interval and the global aggregation results, a score is generated through linear combination.

[0121] Among them, constructing a cognitive model based on the results of auditory psychology experiments includes:

[0122] Integrate the physiological perception characteristics of the human ear for sound and the psychological subjective evaluation data, and transform complex auditory phenomena into computable mathematical representations;

[0123] Through masking experiments (such as pure tones masked by narrowband noise), it is found that the human ear divides frequencies into 24 critical bands (Bark scale), and the signals within each band interfere with each other. PESQ adopts the Bark band division to simulate the differences in the distortion sensitivity of the human ear to different frequency bands.

[0124] Furthermore, the specific implementation of the non-linear regression correction includes:

[0125] During the experiment, it was found that there is a certain mathematical relationship between the error of the PESQ distance from MOS and MOS. The specific mathematical relationship is as Figure 6 shown, which is relatively close to the quadratic function in the non-linear function, that is, y = ax 2 + bx. Therefore, it can be assumed that the y values represented by MOS and the error of the PESQ distance from MOS can satisfy the conditions of the non-linear regression model, that is, non-linear fitting can be performed on them by the least squares method to find the estimated values of the coefficients and then obtain the non-linear regression equation between the parameters.

[0126] Let z be the error of the PESQ distance from MOS, x1 be the MOS score, x2 be the PESQ_PRE score, and m be the PESQ score. Then the following formula is obtained:

[0127] z = (x1 - m) 2 (2)

[0128]

[0129] After fitting using the least squares method, the non-linear regression equation obtained is as follows:

[0130] y = 1.8478x 2 + 9.6913x + 14.1911(4)

[0131] In addition, there is a certain non-linear relationship between MOS and the error of PESQ. Utilize the advantage of the Marquardt method that has lower requirements for the initial values of simple non-linear models to establish a non-linear model. From the experimental results Figure 7 it can be seen that there may be a situation where the cubic term function is more optimal, but from the experimental results Figure 7Judging from the results verified by the large datasets in Table 1, it is only a trap of local optimality.

[0132] Table 1

[0133]

[0134] The experimental analysis of this embodiment is introduced as follows:

[0135] To verify the effectiveness and generalization ability of the model, as well as whether the added improvement modules have a good effect, the following two sets of open-source datasets were selected; namely, the datasets from the VCC 2018 speech challenge and the dataset created by Ryan Dhimas E. Zezario et al.: Wall_street.

[0136] The VCC 2018 dataset contains a total of 20,580 English speech recordings, where real people speak a complete sentence or a noun with an adjective. The speaking speed is relatively fast, with a duration between 3 seconds and 5 seconds, an average of 3 words per second, and both male and female voices.

[0137] Wall_street contains a total of 14,915 Chinese speech recordings, where real people speak a complete sentence, and its length is generally controlled within 5 seconds, mostly 3 seconds. The average number of characters in each data is ten. It comes with a dataset containing noise, and both male and female voices. According to the requirements of PESQ, all audio files are set to mono, and the sampling rate is set to 16 kHz. The input of the model is to simultaneously input the audio files with noise and the original ones.

[0138] The evaluation metrics in this embodiment are as follows:

[0139] To adjust the training of the model and evaluate the final prediction effect of the model, the Mean Squared Error (MSE) is used as the loss function and the verification metric. The formula is as follows:

[0140]

[0141] Where is the speech prediction score value, x i is the MOS score value. n represents the dataset size.

[0142] In terms of speed, all datasets are input to the model, and the start and end times are recorded. Due to possible errors, the average value is obtained by running multiple times when there is no background program running on the computer.

[0143]

[0144] Among them, t is the total time, t n1 is the end time, t n2is the start time, and n is the number of runs.

[0145] The experimental configuration of this embodiment is as follows:

[0146] To verify the feasibility of low-cost operation and easy deployment, the following introduces the experimental configuration used:

[0147] This experiment was completed on a computing platform based on an Intel Core i9-14900K processor. The platform was selected based on the following: The processor has 24 cores and 32 threads (maximum turbo frequency 6.0 GHz), which can meet the parallel computing requirements for large-scale speech feature extraction. The measured time-consuming for PESQ prediction of 100 samples in a single batch is less than 10 seconds.

[0148] A dataset annotated by the VCC 2018 speech challenge was used for non-linear regression fitting. A second-degree fit was selected for polynomial fitting. All the work was completed using the CPU throughout the process.

[0149] Ablation experiment:

[0150] In addition, to verify the effectiveness of the used noise reconstruction module and the actual improvement effect of the proposed non-linear regression module on the network, this embodiment conducted an ablation experiment. First, in the case of only using the noise reconstruction module, the selected noise level fluctuation range was obtained from the situation of obtaining the MOS score by adjusting the noise level in the dataset. The noise level effectively affected the accuracy of PESQ, and the most ideal situation could reduce the MSE to 1.0717.

[0151] Second, in the case of only using the non-linear regression module, the experiment proved that the non-linear regression module comprehensively and effectively improved the accuracy of PESQ at different stages, with a maximum improvement of 82.5%. And when the influence degree of the noise level was increased equally, the accuracy of PESQ would fluctuate to a large extent. For example, when the noise level was increased by 20%, the PESQ error increased by 34%, while the error of the non-linear regression module only increased by 1.4%, which was more stable and effectively suppressed the fluctuation.

[0152] The experimental results proved that both the noise reconstruction module and the non-linear regression module played a role in improving the accuracy. Among them, the non-linear regression module had the most significant effect and effectively reduced the MSE. However, unfortunately, the situation where PESQ_PRE_MSE reached the minimum value was not the time when PESQ_MSE was the minimum. This indicates that there is a complex relationship between these two indicators, and other variables may need to be added for optimization to balance.

[0153] Experimental results and analysis:

[0154] This embodiment systematically studied the performance of PESQ_PRE and various factors affecting the final result.

[0155] In terms of model training and deployment, both model training and prediction are tasks completed using the CPU. Moreover, for example, MOSNet must operate in a Linux environment, but this embodiment avoids such trouble and provides multi-platform support. It also offers a more lightweight and simple deployment, enabling even non-professionals to easily get started.

[0156] On different datasets, compared with MOSNet, the speed of PESQ_PRE increased by 82.5% and 71.9% respectively. In the case where the detected samples are mixed with noise, the accuracy of PESQ_PRE increased by 68.8%, exceeding the performance of MOSNet. And it can improve the accuracy of PESQ by up to 89.1%.

[0157] The main purpose of this embodiment is to provide voice score prediction for the laboratory. It selects a wideband mode that is closer to the human auditory range, can capture details such as consonants and voiceless segments of speech, and improve the accuracy of speech evaluation.

[0158] PESQ is an open-source and highly recognized algorithm model in intrusive voice score prediction. Considering that when detecting speech quality or evaluating the quality of synthetic speech, both speed and accuracy need to be taken into account. Therefore, based on PESQ, this paper proposes a voice score prediction method based on improved PESQ. The experimental results of its original model's wideband mode on the open-source dataset are as Figure 1 shown. Aiming at the situation where the model has low accuracy in predicting voice scores, the noise of the distorted signal is reconstructed, and a non-linear score mapping enhancement module is introduced at the end of the model for score calibration. The experimental results show that: the improved algorithm in this embodiment has high accuracy in voice score prediction, and at the same time meets the requirements of the prediction speed in terms of detection speed.

[0159] The embodiments described above are only descriptions of the preferred embodiments of the present invention, and do not limit the scope of the present invention. Without departing from the design spirit of the present invention, various deformations and improvements made by those of ordinary skill in the art to the technical solutions of the present invention should fall within the protection scope determined by the claims of the present invention.

Claims

1. An objective evaluation method for speech quality improved based on non - linear regression, characterized in that, Including: After preprocessing the original speech signal and the distorted speech signal respectively, perform time alignment; Convert the aligned signals into auditory feature representations through auditory transformation respectively, and perform dithering processing; Input the features after dithering processing into the cognitive model to obtain the preliminary speech quality score; wherein, the cognitive model is constructed based on the results of auditory psychology experiments; Correct the preliminary speech quality score to obtain the final speech quality score.

2. The objective speech quality evaluation method based on non - linear regression improvement according to claim 1, characterized in that, The preprocessing of the original speech signal includes: Perform level adjustment and filtering on the original speech signal.

3. The objective speech quality evaluation method improved based on non-linear regression according to claim 1, characterized in that, The preprocessing of the distorted speech signal includes: Perform noise reconstruction on the distorted speech signal to generate noise related to the original speech signal; Add the noise to the original speech signal to generate a noisy audio signal; Perform level adjustment and filtering on the noisy audio signal; The noise reconstruction is: y = f × max(|x(t)|) wherein, x(t) is the input audio signal, max is to take the maximum value in the audio signal, y represents the noise level, and f is the set percentage.

4. The objective speech quality evaluation method improved based on non - linear regression according to claim 1, characterized in that, Performing time alignment on the preprocessed signals includes: Detect the silent time in the preprocessed signals; Use the envelope correlation method to calculate the rough delay estimate of the signal after detecting the silent time, and eliminate the rough delay; Use the weighted histogram method to perform fine delay and confidence estimation on the signal after eliminating the rough delay, and use the methods of fixed delay alignment and dynamic alignment to perform signal alignment.

5. The objective speech quality evaluation method improved based on non - linear regression according to claim 1, characterized in that, Performing auditory transformation on the aligned signals includes: Perform fast Fourier transform on the aligned signals, group them according to the Bark scale, and extract spectral features; map the frequency of the speech signal to a non-linear pitch scale that conforms to the human ear perception characteristics; Compare the spectral differences between the original frame and the degraded frame; convert the physical intensity of the speech signal to a non-linear scale that conforms to the subjective loudness perception of the human ear to obtain the auditory feature representation.

6. The objective speech quality evaluation method improved based on nonlinear regression according to claim 1, characterized in that Inputting the features after dithering processing into the cognitive model includes: Determine and count the bad intervals of the features after dithering processing to obtain the speech interference value; Realign the bad intervals; Quantify the local degradation degree of speech quality by introducing the number of bad intervals; Aggregate the speech interference value using the L6 norm; Divide the aggregated speech signal into preset short-time windows, and within the second-minute interval of each preset short-time window, use the L2 norm to aggregate the frame-level interference values to generate the second-minute interference index; Combine the second-minute interference index and the aggregated speech signal to generate a score through linear combination.

7. The objective speech quality evaluation method improved based on non - linear regression according to claim 1, characterized in that, The method for correcting the preliminary speech quality score is: adopt a non-linear regression equation.

Images