Voice quality evaluation method and device, model training method and device and storage medium
By performing objective quality evaluation and feature extraction of speech data without reference sound, combined with neural network model, the problem of low accuracy of objective voice quality evaluation in the prior art is solved, and more efficient speech quality evaluation is achieved.
Patent Information
- Application Number
- CN202411540430.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-30
- Publication Date
- 2025-02-11
AI Technical Summary
In the prior art, the accuracy of objective voice quality evaluation is low, and it is difficult to meet the needs of large-scale voice quality evaluation.
By performing objective quality evaluation of speech data without reference sound, frequency domain features and time domain features are extracted, and these features are input into a preset neural network model to obtain more accurate objective score values.
It improves the accuracy of objective voice quality evaluation and can conduct large-scale voice quality evaluation more effectively.
Smart Images
Figure CN119418721A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of sound quality assessment, and in particular to a speech quality assessment method, a model training method, a device and a storage medium. Background Art
[0002] In speech signal processing tasks, such as beamforming, speech noise reduction, speech encoding and decoding, speech synthesis, speech transmission, etc., speech quality evaluation indicators are usually used to illustrate the effect of processing. Speech evaluation is generally divided into subjective evaluation and objective evaluation.
[0003] Subjective evaluation relies on the experience and standards of the evaluators, and there are large differences. The scoring process is time-consuming and requires a lot of manual resources. It is not suitable for large-scale speech quality evaluation, and the scoring results may be affected by environmental noise and other irrelevant factors, which limits the accuracy. The current objective evaluation generally conducts objective evaluation without reference sound, which has low accuracy. Summary of the invention
[0004] The present application provides a speech quality assessment method, a model training method, a device and a storage medium to solve the problem of how to improve the accuracy of objective speech quality assessment.
[0005] In a first aspect, the present application provides a method for evaluating speech quality, the method comprising:
[0006] Conduct objective quality assessment on speech data without reference sound to obtain a preliminary score value;
[0007] Performing frequency domain feature extraction on the voice data to obtain frequency domain features of the voice data, and performing time domain feature extraction on the voice data to obtain time domain features of the voice data;
[0008] The preliminary scoring value, the frequency domain features and the time domain features are input into a preset neural network model to obtain an objective scoring value of the speech data.
[0009] Optionally, the frequency domain feature includes a Mel frequency cepstrum coefficient, and frequency domain feature extraction is performed on the speech data to obtain the frequency domain feature of the speech data, including:
[0010] Performing pre-emphasis processing on the voice data using a high-pass filter to obtain a target voice signal;
[0011] Performing windowing processing on the target speech signal based on a window function to obtain M frames of windowed signals;
[0012] Performing time-frequency domain conversion on each frame of the windowed signal to obtain target spectrum data;
[0013] The Mel-frequency cepstrum coefficients of the target spectrum data are extracted based on the Mel filter bank.
[0014] Optionally, the time domain feature includes a linear prediction coefficient, and performing time domain feature extraction on the speech data to obtain the time domain feature of the speech data includes:
[0015] Performing frame division and windowing processing on the speech data to obtain N frames of frame division and windowing signals;
[0016] Performing signal autocorrelation processing on the N frame windowed signals to determine a linear relationship between the frame windowed signals;
[0017] The linear prediction coefficients are determined based on the linear relationship.
[0018] Optionally, the preliminary scoring value, the frequency domain feature and the time domain feature are input into a preset neural network model to obtain an objective scoring value of the speech data, including:
[0019] Combining the preliminary score value, the frequency domain feature and the time domain feature into a multidimensional vector;
[0020] The multidimensional vector is input into the neural network model to obtain the objective score value.
[0021] Optionally, the neural network model is a recurrent neural network model.
[0022] In a second aspect, the present application provides a method for training a speech quality assessment model, the method comprising:
[0023] Acquire training data and a target training score corresponding to the training data; wherein the target training score is a subjective score value obtained based on the training data;
[0024] Performing objective quality assessment on the training data without reference sound to obtain a preliminary score value;
[0025] Performing frequency domain feature extraction on the training data to obtain frequency domain features of the training data, and performing time domain feature extraction on the training data to obtain time domain features of the training data;
[0026] Inputting the preliminary score value, the frequency domain feature and the time domain feature into the model to be trained to obtain an objective score value of the training data;
[0027] Determine whether the difference data obtained by subtracting the objective score value from the target training score is less than a preset difference; if not, feed back the difference data to the model to be trained so that the model to be trained iterates the training parameters, and after the iteration of the training parameters, repeatedly execute the step of inputting the preliminary score value, the frequency domain features and the time domain features into the model to be trained to obtain the target training score of the training data, until the step of determining whether the difference data obtained by subtracting the objective score value from the target training score is less than a preset difference, until the difference data is less than the preset difference; if so, use the model to be trained as the neural network model obtained by training as described in any one of the first aspects.
[0028] In a third aspect, the present application provides a speech quality assessment device, the device comprising:
[0029] The first evaluation module is used to perform objective quality evaluation on the speech data without reference sound to obtain a preliminary score value;
[0030] A first extraction module is used to perform frequency domain feature extraction on the voice data to obtain the frequency domain features of the voice data, and to perform time domain feature extraction on the voice data to obtain the time domain features of the voice data;
[0031] The second evaluation module is used to input the preliminary scoring value, the frequency domain features and the time domain features into a preset neural network model to obtain an objective scoring value of the speech data.
[0032] In a fourth aspect, the present application provides a speech quality assessment model training device, the device comprising:
[0033] An acquisition module, used to acquire training data and a target training score corresponding to the training data; wherein the target training score is a subjective score value obtained based on the training data;
[0034] A third evaluation module is used to perform an objective quality evaluation on the training data without reference sound to obtain a preliminary score value;
[0035] A second extraction module is used to perform frequency domain feature extraction on the training data to obtain the frequency domain features of the training data, and to perform time domain feature extraction on the training data to obtain the time domain features of the training data;
[0036] A fourth evaluation module, used for inputting the preliminary score value, the frequency domain feature and the time domain feature into the model to be trained to obtain an objective score value of the training data;
[0037] A training module, used to determine whether the difference data obtained by subtracting the objective score value from the target training score is less than a preset difference; if not, the difference data is fed back to the model to be trained so that the model to be trained iterates the training parameters, and after the iteration of the training parameters, repeatedly execute the step of inputting the preliminary score value, the frequency domain features and the time domain features into the model to be trained to obtain the target training score of the training data, to the step of determining whether the difference data obtained by subtracting the objective score value from the target training score is less than a preset difference, until the difference data is less than the preset difference; if so, use the model to be trained as the neural network model obtained by training as described in any one of the first aspects.
[0038] In a fifth aspect, the present application provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other via the communication bus;
[0039] Memory, used to store computer programs;
[0040] The processor is used to implement the speech quality assessment method described in any one of the embodiments of the first aspect or the speech quality assessment model training method described in any one of the embodiments of the second aspect when executing the program stored in the memory.
[0041] In a sixth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the speech quality assessment method as described in any one of the embodiments of the first aspect or the speech quality assessment model training method as described in any one of the embodiments of the second aspect.
[0042] The above technical solution provided by the embodiment of the present application has the following advantages over the prior art: the method provided by the embodiment of the present application performs an objective quality assessment on the speech data without reference sound to obtain a preliminary score value; performs frequency domain feature extraction on the speech data to obtain the frequency domain features of the speech data, and performs time domain feature extraction on the speech data to obtain the time domain features of the speech data; inputs the preliminary score value, the frequency domain features and the time domain features into a preset neural network model to obtain the objective score value of the speech data. The method first performs an objective quality assessment on the speech data without reference sound to obtain a preliminary score value, then performs feature extraction on the speech data to obtain frequency domain features and time domain features respectively, inputs the preliminary score value, frequency domain features and time domain features into a pre-trained neural network model to obtain the objective score value of the speech data output by the neural network model, which can improve the accuracy of the objective assessment. BRIEF DESCRIPTION OF THE DRAWINGS
[0043] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.
[0044] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.
[0045] One or more embodiments are exemplarily described by pictures in the corresponding drawings, and these exemplified descriptions do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings represent similar elements, and unless otherwise stated, the figures in the drawings do not constitute proportional limitations.
[0046] Figure 1 A system architecture diagram of a method for evaluating speech quality provided by one embodiment of the present application;
[0047] Figure 2 A flowchart of a method for evaluating speech quality provided by an embodiment of the present application;
[0048] Figure 3 A schematic diagram of a frequency domain MFCC coefficient extraction process provided for one embodiment of the present application;
[0049] Figure 4 A schematic diagram of a time domain LPC coefficient extraction process provided by an embodiment of the present application;
[0050] Figure 5 A flowchart of a method for training a speech quality assessment model provided by one embodiment of the present application;
[0051] Figure 6 A training block diagram of an evaluation model provided for one embodiment of the present application;
[0052] Figure 7 A schematic diagram of an application of a method for evaluating speech quality provided by an embodiment of the present application;
[0053] Figure 8 A schematic diagram of the structure of a speech quality assessment device provided by one embodiment of the present application;
[0054] Fig. 9 A schematic diagram of the structure of a speech quality assessment model training device provided in one embodiment of the present application;
[0055] Fig.10 A schematic diagram of the structure of an electronic device provided in accordance with an embodiment of the present application. DETAILED DESCRIPTION
[0056] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.
[0057] The disclosure below provides many different embodiments or examples to realize the different structures of the present application. In order to simplify the disclosure of the present application, the parts and settings of specific examples are described below. Of course, they are only examples, and the purpose is not to limit the present application. In addition, the present application can repeat reference numbers and / or letters in different examples. This repetition is for the purpose of simplification and clarity, and does not itself indicate the relationship between the various embodiments and / or settings discussed.
[0058] In speech signal processing tasks, such as beamforming, speech noise reduction, speech encoding and decoding, speech synthesis, speech transmission, etc., speech quality evaluation indicators are usually used to illustrate the effect of processing. Speech evaluation is generally divided into subjective evaluation and objective evaluation.
[0059] The subjective scoring method obtains the average subjective score MOS (Mean Opinion Score) by asking a certain number of artificial listeners to score the auditory quality of speech samples, and then calculating the average of all listeners' scores. The MOS score is usually between 1 and 5, with the scores being: 1-very poor, almost unusable; 2-poor, the quality is obviously impaired and the application is limited; 3-average, can be used for some applications; 4-good, can be used for most applications; 5-very good, comparable to the quality of high-quality standard telephones. The advantage of MOS scoring is that it is intuitive and simple, corresponds to human auditory perception, and can effectively evaluate the subjective hearing experience of human listeners. However, it also has certain disadvantages:
[0060] 1. The accuracy of the scoring depends on the experience and standards of the evaluators and there is a large difference.
[0061] 2. The scoring process is time-consuming and requires a lot of manual resources, which is not suitable for large-scale speech quality assessment.
[0062] 3. The scoring results may be affected by environmental noise and other irrelevant factors, and their accuracy may be limited.
[0063] In the large-scale evaluation of algorithm development, products and services, more objective and automated evaluation indicators are used.
[0064] Objective scoring methods can be divided into two categories: with reference and without reference. Typical objective scoring with reference is Perceptual Evaluation of Speech Quality (PESQ), which takes clean speech and processed speech as input, compares the difference between the two, and then gives a score value. Methods with reference require clean speech as a reference for scoring, which is often difficult to obtain in practical applications, such as obtaining the voice quality of the remote terminal in a conference speaker. Therefore, in recent years, the industry and academia have been exploring the use of objective scoring methods for speech quality without reference. For example, the International Telecommunication Union (ITU) proposed a non-reference objective sound quality scoring method ITU-T p.563 using traditional signal processing, but according to relevant literature evaluation, the correlation with subjective scoring is relatively low. In addition, there are some solutions that evaluate the sound quality after speech enhancement, which are relatively complex and not conducive to real-time evaluation.
[0065] In order to solve the technical problem of how to improve the accuracy of objective speech quality assessment in the prior art, the present application provides a speech quality assessment method, a model training method, a device and a storage medium, which can combine preliminary scoring values, frequency domain features and time domain features, and output objective scoring values through a neural network model to improve the accuracy of objective assessment.
[0066] The first embodiment of the present application provides a method for evaluating speech quality. The method can be applied to Figure 1 The system architecture shown in the figure includes at least a traditional evaluation module 101, a feature extraction module 102 and a neural network model 103. The speech data is respectively passed through the traditional evaluation module 101 to obtain a preliminary score value, and passed through the feature extraction module 102 to obtain frequency domain features and time domain features. The preliminary score value, frequency domain features and time domain features are input into the neural network model 103 to output an objective score value.
[0067] Next, based on the system architecture, the speech quality evaluation method is described in detail. Figure 2 , the speech quality assessment method comprises:
[0068] Step 201 : objectively evaluate the quality of speech data without reference sound to obtain a preliminary score.
[0069] The speech data may be subjected to objective quality evaluation without reference sound through a traditional evaluation module to obtain a preliminary score value. The traditional evaluation module may be a P.563 module without limitation. P.563 is a single-ended evaluation standard for objective speech quality applied to narrowband communications.
[0070] Step 202: extract frequency domain features from the speech data to obtain frequency domain features of the speech data, and extract time domain features from the speech data to obtain time domain features of the speech data.
[0071] Frequency domain feature extraction may be to extract Mel-frequency cepstral coefficients of speech data, where the Mel-frequency cepstral coefficients are MFCC (mel-frequency cepstral coefficients).
[0072] In one embodiment, frequency domain features are extracted from speech data to obtain frequency domain features of the speech data, including: pre-emphasis processing is performed on the speech data using a high-pass filter to obtain a target speech signal; windowing is performed on the target speech signal based on a window function to obtain M frames of windowed signals; time-frequency domain conversion is performed on each frame of the windowed signal to obtain target spectrum data; and Mel-frequency cepstrum coefficients of the target spectrum data are extracted based on a Mel filter group.
[0073] In this embodiment, the frequency domain MFCC coefficient extraction process diagram is as follows: Figure 3 , a high-pass filter can be used to pre-emphasize the voice data to obtain the target voice signal. The purpose of pre-emphasis is to enhance the high-frequency part of the voice data to compensate for the high-frequency attenuation that may occur during the transmission or recording of the audio signal. Pre-emphasis can be achieved through a high-pass filter, and its differential equation can be:
[0074] s[n]=s[n]-0.95*s[n-1]; where n is the time domain frame number;
[0075] The target speech signal is windowed based on the window function to obtain M frames of windowed signals. The windowing process can use the Hamming window:
[0076] W[n]=0.54-0.46*cos(2*pi*n / M); wherein n is the time domain frame number and M is the frame length; wherein, to ensure that the window function is correctly normalized at the frame boundary, the M in the denominator may also be M-1 without restriction.
[0077] The time-frequency domain conversion can use fast Fourier transform (FFT) to obtain the target spectrum data, and then process the target spectrum data based on the Mel filter group. For example, the number of filters is selected according to the bandwidth, the target spectrum data is filtered, and then logarithmic operation and discrete cosine transform (DCT) are performed to obtain 13-dimensional MFCC coefficients.
[0078] In one embodiment, the time domain feature may be a linear prediction coefficient, i.e., an LPC (Linear prediction coefficients) coefficient. Time domain feature extraction is performed on speech data to obtain the time domain feature of the speech data, including: framing and windowing the speech data to obtain N frames of framed and windowed signals; performing signal autocorrelation processing on the N frames of framed and windowed signals to determine the linear relationship between the framed and windowed signals; and determining the linear prediction coefficient based on the linear relationship.
[0079] In this embodiment, the schematic diagram of the time domain LPC coefficient extraction process is as follows: Figure 4 , the speech data is framed, windowed, signal autocorrelated and LPC analyzed. The LPC analysis can use the Levinson-Durbin recursive algorithm to finally obtain 11-dimensional LPC coefficients (the first constant 1 is not used to save calculations).
[0080] Step 203: Input the preliminary score value, frequency domain features and time domain features into a preset neural network model to obtain an objective score value of the speech data.
[0081] The method firstly performs an objective quality assessment on speech data without reference sound to obtain a preliminary score value, then performs feature extraction on the speech data to obtain frequency domain features and time domain features respectively, and inputs the preliminary score value, frequency domain features and time domain features into a pre-trained neural network model to obtain an objective score value of the speech data output by the neural network model, thereby improving the accuracy of the objective assessment.
[0082] In one embodiment, the neural network model is a recurrent neural network model.
[0083] Since speech signals are similar to language and text in that they have temporal correlation, and a recurrent neural network (RNN) happens to be a type of network with short-term memory, this embodiment uses a recurrent neural network model to process speech data. The recurrent neural network can use a long short-term memory network (Long Short-Term Memory, referred to as LSTM) or a gated recurrent unit (Gated Recurrent Unit, referred to as GRU). Since the design concept of GRU is similar to that of LSTM, and compared with LSTM, GRU has one less gate inside, that is, the parameters required for using GRU are less than those required for using LSTM, and using GRU can also approach or achieve performance comparable to that of LSTM, so GRU can be used to build a model.
[0084] In one embodiment, a preliminary scoring value, frequency domain features, and time domain features are input into a preset neural network model to obtain an objective scoring value for the speech data, including: merging the preliminary scoring value, frequency domain features, and time domain features into a multidimensional vector; and inputting the multidimensional vector into the neural network model to obtain an objective scoring value.
[0085] In this embodiment, the preliminary scoring value, frequency domain features and time domain features are combined into a multi-dimensional vector. For example, the preliminary scoring value is a 1-dimensional parameter, the MFCC coefficient is a 13-dimensional parameter, and the LPC coefficient is an 11-dimensional parameter, that is, they are combined into a 25-dimensional feature vector. The 25-dimensional feature vector is input into the neural network model to obtain the objective scoring value output by the model.
[0086] In one embodiment, before inputting the preliminary score value, frequency domain features and time domain features into a preset neural network model to obtain an objective score value of the speech data, the method also includes: obtaining a neural network model; wherein the training process of the neural network model is described in the second embodiment of the present application and is not repeated here.
[0087] Based on the same technical concept, the second embodiment of the present application provides a method for training a speech quality assessment model, such as Figure 5 , methods include:
[0088] Step 501, obtaining training data and a target training score corresponding to the training data; wherein the target training score is a subjective score value obtained based on the training data;
[0089] Step 502, objective quality evaluation is performed on the training data without reference sound to obtain a preliminary score value;
[0090] Step 503, extracting frequency domain features from the training data to obtain frequency domain features of the training data, and extracting time domain features from the training data to obtain time domain features of the training data;
[0091] Step 504, inputting the preliminary score value, frequency domain features and time domain features into the model to be trained to obtain an objective score value of the training data;
[0092] Step 505, determine whether the difference data obtained by subtracting the objective score value from the target training score is less than the preset difference; if not, feed the difference data back to the model to be trained so that the model to be trained iterates the training parameters, and after the iteration of the training parameters, repeatedly input the preliminary score value, frequency domain features and time domain features into the model to be trained to obtain the target training score of the training data, until the step of determining whether the difference data obtained by subtracting the objective score value from the target training score is less than the preset difference, until the difference data is less than the preset difference; if so, use the model to be trained as the trained neural network model.
[0093] The training method first obtains the training data and the target training score corresponding to the training data, that is, the subjective score value, performs objective quality evaluation on the training data without reference sound to obtain a preliminary score value, performs feature extraction on the training data to obtain frequency domain features and time domain features, and then inputs the preliminary score value, frequency domain features and time domain features into the model to be trained to obtain the objective score value of the training data, and judges whether to continue training the model to be trained based on whether the difference data obtained by subtracting the objective score value from the target training score is less than a preset difference value, until the difference data is less than the preset difference value, and the training is completed. Since the subjective score value is used as a reference during training, the objective score value output by the trained neural network model when used is closer to the subjective score, which can improve the accuracy of the objective evaluation, and the amount of calculation is small, which is convenient for online evaluation.
[0094] In this embodiment, the training block diagram of the evaluation model can be as follows: Figure 6 The model to be trained can be a deep neural network model (Deep Neural Network, DNN for short). First, prepare the training corpus (the training expectation is also speech data. To distinguish it from the above embodiment, the training expectation is referred to as training data here) and the real MOS value corresponding to the training data (i.e., the target training score, which can be called MOS_TRUE). The training data is processed by a traditional evaluation module (such as a P.563 module) and feature extraction and then concatenated into a 25-dimensional input vector. After the vector passes through the DNN model, the inferred MOS value (objective score value, which can be called MOS_EXT) is output. The difference between MOS_EXT and MOS_TRUE is fed back to the DNN model, and then the weight coefficient W and the bias b are iterated. After several iterations, the training ends when the difference between MOS_EXT and MOS_TRUE is less than a given value.
[0095] It should be noted that, in this embodiment, the above network training process can be specifically completed in advance using framework programming such as tensorflow, keras, Mxnet, etc.
[0096] A schematic diagram of an application of a speech quality assessment method is shown in FIG. Figure 7 After the MOS_EST module of the remote voice terminal product obtains the downlink voice data, it uses the voice quality evaluation method proposed in the present application to obtain the objective score value estimated by the neural network model. In addition, the volume playback size information LEVEL_index is obtained through the power amplifier module (PA). The voice quality of the downlink playback is estimated to be good (1) or bad (0) through the objective score value and the volume playback size information, and the flag bit FLAG is used to mark it and transmit it to the near end through the uplink.
[0097] Based on the same technical concept, the present application embodiment provides a speech quality assessment device, such as Figure 8 , the device comprises:
[0098] The first evaluation module 801 is used to perform objective quality evaluation on the speech data without reference sound to obtain a preliminary score value;
[0099] The first extraction module 802 is used to perform frequency domain feature extraction on the voice data to obtain the frequency domain features of the voice data, and to perform time domain feature extraction on the voice data to obtain the time domain features of the voice data;
[0100] The second evaluation module 803 is used to input the preliminary scoring value, the frequency domain features and the time domain features into a preset neural network model to obtain an objective scoring value of the speech data.
[0101] The device firstly performs an objective quality evaluation on speech data without reference sound to obtain a preliminary scoring value, then performs feature extraction on the speech data to obtain frequency domain features and time domain features respectively, inputs the preliminary scoring value, frequency domain features and time domain features into a pre-trained neural network model, obtains an objective scoring value of the speech data output by the neural network model, and can improve the accuracy of the objective evaluation.
[0102] The present application embodiment provides a speech quality assessment model training device, such as Fig. 9 , the device comprises:
[0103] An acquisition module 901 is used to acquire training data and a target training score corresponding to the training data; wherein the target training score is a subjective score value obtained based on the training data;
[0104] A third evaluation module 902 is used to perform an objective quality evaluation on the training data without reference sound to obtain a preliminary score value;
[0105] The second extraction module 903 is used to perform frequency domain feature extraction on the training data to obtain the frequency domain features of the training data, and to perform time domain feature extraction on the training data to obtain the time domain features of the training data;
[0106] A fourth evaluation module 904 is used to input the preliminary score value, the frequency domain feature and the time domain feature into the model to be trained to obtain an objective score value of the training data;
[0107] The training module 905 is used to determine whether the difference data obtained by subtracting the objective score value from the target training score is less than a preset difference; if not, the difference data is fed back to the model to be trained so that the model to be trained iterates the training parameters, and after the iteration of the training parameters, the step of inputting the preliminary score value, the frequency domain features and the time domain features into the model to be trained is repeated to obtain the target training score of the training data, until the step of determining whether the difference data obtained by subtracting the objective score value from the target training score is less than a preset difference, until the difference data is less than the preset difference; if so, the model to be trained is used as the trained neural network model.
[0108] The training device first obtains training data and a target training score corresponding to the training data, i.e., a subjective score value, performs an objective quality assessment on the training data without reference sound to obtain a preliminary score value, performs feature extraction on the training data to obtain frequency domain features and time domain features, and then inputs the preliminary score value, frequency domain features, and time domain features into the model to be trained to obtain an objective score value of the training data, and determines whether to continue training the model to be trained based on whether the difference data obtained by subtracting the objective score value from the target training score is less than a preset difference value, until the difference data is less than the preset difference value, and the training is completed. Since the subjective score value is used as a reference during training, the objective score value output by the trained neural network model when used is closer to the subjective score, which can improve the accuracy of the objective assessment, and has a small amount of calculation, which is convenient for online assessment.
[0109] like Fig.10 As shown, an embodiment of the present application provides an electronic device, including a processor 111, a communication interface 112, a memory 113 and a communication bus 114, wherein the processor 111, the communication interface 112, and the memory 113 communicate with each other through the communication bus 114.
[0110] Memory 113, used for storing computer programs;
[0111] In one embodiment of the present application, the processor 111 is used to implement the speech quality assessment method or speech quality assessment model training method provided by any one of the aforementioned method embodiments when executing the program stored in the memory 113.
[0112] The communication bus mentioned in the above terminal can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.
[0113] The communication interface is used for communication between the above terminal and other devices.
[0114] The memory may include a random access memory (RAM) or a non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.
[0115] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.
[0116] An embodiment of the present application also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the speech quality assessment method or speech quality assessment model training method provided in any of the aforementioned method embodiments is implemented.
[0117] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0118] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a general hardware platform, and of course, by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.
[0119] It should be understood that the terms used herein are only for the purpose of describing specific example embodiments and are not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms "one", "an" and "said" as used herein may also be meant to include plural forms. The terms "include", "comprise", "contain", and "have" are inclusive, and therefore specify the existence of stated features, steps, operations, elements and / or parts, but do not exclude the existence or addition of one or more other features, steps, operations, elements, parts, and / or combinations thereof. The method steps, processes, and operations described herein are not interpreted as necessarily requiring them to be performed in the specific order described or illustrated, unless the execution order is clearly indicated. It should also be understood that additional or alternative steps may be used.
[0120] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application. In the description, the suffixes such as "module", "component" or "unit" used to represent the elements are only used to facilitate the description of the present application and have no specific meaning in themselves. Therefore, "module", "component" or "unit" can be used in a mixed manner.
[0121] The above description is only a specific implementation of the present application, so that those skilled in the art can understand or implement the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest range consistent with the principles and novel features applied for herein.
Claims
1. A method for evaluating speech quality, characterized in that: The method comprises: Conduct objective quality assessment on speech data without reference sound to obtain a preliminary score value; Performing frequency domain feature extraction on the voice data to obtain frequency domain features of the voice data, and performing time domain feature extraction on the voice data to obtain time domain features of the voice data; The preliminary scoring value, the frequency domain features and the time domain features are input into a preset neural network model to obtain an objective scoring value of the speech data.
2. The method according to claim 1, characterized in that The frequency domain features include Mel frequency cepstrum coefficients, and the frequency domain features are extracted from the speech data to obtain the frequency domain features of the speech data, including: Performing pre-emphasis processing on the voice data using a high-pass filter to obtain a target voice signal; Performing windowing processing on the target speech signal based on a window function to obtain M frames of windowed signals; Performing time-frequency domain conversion on each frame of the windowed signal to obtain target spectrum data; The Mel-frequency cepstrum coefficients of the target spectrum data are extracted based on the Mel filter bank.
3. The method according to claim 1, characterized in that The time domain feature includes a linear prediction coefficient, and the time domain feature is extracted from the speech data to obtain the time domain feature of the speech data, including: Performing frame division and windowing processing on the speech data to obtain N frames of frame division and windowing signals; Performing signal autocorrelation processing on the N frame windowed signals to determine a linear relationship between the frame windowed signals; The linear prediction coefficients are determined based on the linear relationship.
4. The method according to claim 1, characterized in that: Inputting the preliminary scoring value, the frequency domain feature and the time domain feature into a preset neural network model to obtain an objective scoring value of the speech data includes: Combining the preliminary score value, the frequency domain feature and the time domain feature into a multidimensional vector; The multidimensional vector is input into the neural network model to obtain the objective score value.
5. The method according to any one of claims 1 to 4, characterized in that: The neural network model is a recurrent neural network model.
6. A method for training a speech quality assessment model, characterized in that: The method comprises: Acquire training data and a target training score corresponding to the training data; wherein the target training score is a subjective score value obtained based on the training data; Performing objective quality assessment on the training data without reference sound to obtain a preliminary score value; Performing frequency domain feature extraction on the training data to obtain frequency domain features of the training data, and performing time domain feature extraction on the training data to obtain time domain features of the training data; Inputting the preliminary score value, the frequency domain feature and the time domain feature into the model to be trained to obtain an objective score value of the training data; Determine whether the difference data obtained by subtracting the objective score value from the target training score is less than a preset difference; if not, feed back the difference data to the model to be trained so that the model to be trained iterates the training parameters, and after the iteration of the training parameters, repeatedly execute the step of inputting the preliminary score value, the frequency domain features and the time domain features into the model to be trained to obtain the target training score of the training data, until the step of determining whether the difference data obtained by subtracting the objective score value from the target training score is less than a preset difference, until the difference data is less than the preset difference; if so, use the model to be trained as the neural network model obtained by training as described in any one of claims 1 to 5.
7. A speech quality assessment device, characterized in that: The device comprises: The first evaluation module is used to perform objective quality evaluation on the speech data without reference sound to obtain a preliminary score value; A first extraction module is used to perform frequency domain feature extraction on the voice data to obtain the frequency domain features of the voice data, and to perform time domain feature extraction on the voice data to obtain the time domain features of the voice data; The second evaluation module is used to input the preliminary scoring value, the frequency domain features and the time domain features into a preset neural network model to obtain an objective scoring value of the speech data.
8. A speech quality assessment model training device, characterized in that: The device comprises: An acquisition module, used to acquire training data and a target training score corresponding to the training data; wherein the target training score is a subjective score value obtained based on the training data; A third evaluation module is used to perform an objective quality evaluation on the training data without reference sound to obtain a preliminary score value; A second extraction module is used to perform frequency domain feature extraction on the training data to obtain the frequency domain features of the training data, and to perform time domain feature extraction on the training data to obtain the time domain features of the training data; A fourth evaluation module, used for inputting the preliminary score value, the frequency domain feature and the time domain feature into the model to be trained to obtain an objective score value of the training data; A training module, used to determine whether the difference data obtained by subtracting the objective score value from the target training score is less than a preset difference; if not, the difference data is fed back to the model to be trained so that the model to be trained iterates the training parameters, and after the iteration of the training parameters, repeatedly execute the step of inputting the preliminary score value, the frequency domain features and the time domain features into the model to be trained to obtain the target training score of the training data, to the step of determining whether the difference data obtained by subtracting the objective score value from the target training score is less than a preset difference, until the difference data is less than the preset difference; if so, use the model to be trained as the neural network model obtained by training as described in any one of claims 1 to 5.
9. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; The processor is used to implement the speech quality assessment method described in any one of claims 1 to 5 or the speech quality assessment model training method described in claim 6 when executing the program stored in the memory.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the speech quality assessment method according to any one of claims 1 to 5 or the speech quality assessment model training method according to claim 6 is implemented.
Citation Information
Cited By
Room sound effect correction effect evaluation method and device, equipment and storage medium
CN121054038A