Pre-training method of speech quality determination model and speech quality determination method

Abstract

The present disclosure relates to a pre-training method for a speech quality determination model, a speech quality determination method, a device, an electronic device and a storage medium, and belongs to the field of signal processing technology. The method includes: simulating damaged speech signals of various types of damage based on reference speech signals and damage information; obtaining the multiple damaged speech features through a speech quality determination model; determining a first loss value for characterizing the distance between damaged speech features of various types of damage based on the multiple damaged speech features; and pre-training the speech quality determination model based on the first loss value. Through the above technical solution, there is no need to carry a large number of speech signals with sound quality annotations, and the generalization of the speech quality determination model for various damage scenarios can be effectively improved, ensuring that the model has a strong resolution capability for damaged speech signals of various types of damage, and can extract high-accuracy damaged speech features, thereby greatly improving the accuracy of predicting speech quality.

Description

Technical Field

[0001] The present disclosure relates to the field of signal processing technology, and in particular to a pre-training method for a voice quality determination model, a voice quality determination method, a device, an electronic device, and a storage medium. Background Art

[0002] Since speech signals are often damaged by various factors such as background noise, spatial reverberation, and channel transmission, which affects the speech quality, accurately evaluating the speech quality in various scenarios can improve the speech quality in a targeted manner.

[0003] Currently, a speech quality determination model can usually be trained based on a speech signal carrying sound quality annotations, so that the speech quality of a damaged speech signal can be directly determined through the speech quality determination model without referring to the speech signal as a reference.

[0004] However, the above technical solution is difficult to effectively learn the speech features of damaged speech signals, resulting in poor generalization of the speech quality determination model. In addition, the above technical solution requires a large number of speech signals carrying sound quality annotations to train the speech quality determination model, which brings high labor costs, and reducing the amount of data will seriously affect the accuracy of speech quality. Therefore, there is an urgent need for a pre-training method for a speech quality determination model, which does not require a large number of speech signals carrying sound quality annotations, and can obtain a speech quality determination model with strong generalization and high prediction accuracy in the pre-training stage. Summary of the invention

[0005] The present disclosure provides a pre-training method, a method, an apparatus, an electronic device and a storage medium for determining a speech quality model, which can obtain a speech quality determination model with strong generalization and high prediction accuracy in the pre-training stage without a large number of speech signals carrying sound quality annotations. The technical solution of the present disclosure is as follows:

[0006] According to a first aspect of an embodiment of the present disclosure, a pre-training method for a speech quality determination model is provided, the method comprising:

[0007] Obtaining, by means of a speech quality determination model, impaired speech features and predicted quality of a plurality of impaired speech signals, wherein the plurality of impaired speech signals are generated based on a reference speech signal and impairment information, wherein the impairment information indicates an impairment type for the reference speech signal, and the impairment type is used to indicate an impairment scenario and an impairment degree;

[0008] Based on the multiple impaired speech features, determining a first loss value of the speech quality determination model, the first loss value being used to characterize the distance between impaired speech features of the same impairment type and the distance between impaired speech features of different impairment types;

[0009] Based on the first loss value, the speech quality determination model is pre-trained.

[0010] In a possible implementation, pre-training the speech quality determination model based on the first loss value includes:

[0011] Based on the first loss value, the model parameters of the speech quality determination model are adjusted so that among the multiple damaged speech features obtained by the adjusted speech quality determination model, the distance between the damaged speech features of the same damage type is smaller, and the distance between the damaged speech features of different damage types is larger.

[0012] In one possible implementation, the method further includes:

[0013] Determine a second loss value of the speech quality determination model based on the reference quality and the predicted quality of the multiple impaired speech signals, the second loss value being used to characterize an error between the predicted quality and the reference quality, the reference quality being determined based on the multiple impaired speech signals and the reference speech signal;

[0014] Based on the first loss value, pre-training the voice quality determination model includes:

[0015] Based on the first loss value and the second loss value, the model parameters of the speech quality determination model are adjusted so that among the multiple damaged speech features and predicted qualities obtained by the adjusted speech quality determination model, the distance between damaged speech features of the same damage type is smaller, the distance between damaged speech features of different damage types is larger, and the error between the predicted quality and the reference quality is smaller.

[0016] In a possible implementation manner, before obtaining the impaired speech features and predicted quality of a plurality of impaired speech signals through the speech quality determination model, the method further includes:

[0017] Based on the first impairment type included in the impairment information, a first reference signal in the reference speech signal is processed according to an impairment degree corresponding to the first impairment type to obtain a first impairment signal of the first reference signal under the first impairment type;

[0018] Based on the first impairment signal and the first reference signal, a reference quality of the first impairment signal is determined, the reference quality indicating a speech quality of the first impairment signal relative to the first reference signal.

[0019] In a possible implementation manner, obtaining the impaired speech features and the predicted quality of the plurality of impaired speech signals by using the speech quality determination model includes:

[0020] Determine speech feature information of the damaged speech signal on the damage type through multiple two-dimensional convolutional layers of the speech quality determination model;

[0021] Determining characteristic information of the impaired speech signal in a time dimension through a plurality of bidirectional recurrent network layers of the speech quality determination model;

[0022] Based on the speech feature information and the feature information in the time dimension, outputting the damaged speech feature through a first fully connected layer of the speech quality determination model;

[0023] Based on the impaired speech features, the predicted quality is output through the second fully connected layer of the speech quality determination model, and the predicted quality includes a perceptual evaluation of speech quality PESQ and / or short-time objective intelligibility STOI.

[0024] In one possible implementation, the method further includes:

[0025] Inputting the target speech signal into the pre-trained speech quality determination model to obtain a predicted mean opinion score MOS of the target speech signal, wherein the target speech signal carries a mean opinion score MOS annotation;

[0026] Determine a third loss value based on the predicted mean opinion score and the mean opinion score annotation carried by the target speech signal, wherein the third loss value is used to characterize an error between the predicted mean opinion score and the mean opinion score annotation;

[0027] Based on the third loss value, the speech quality determination model obtained by pre-training is trained.

[0028] In a possible implementation manner, when the damage scene is ambient sound damage, the damage degree is determined based on the content of the ambient sound and the signal-to-noise ratio;

[0029] In the case where the damage scenario is reverberation damage, the degree of damage is determined based on the reverberation time;

[0030] When the damage scenario is device frequency response damage, the damage degree is determined based on the cutoff frequency of the filter;

[0031] When the impairment scenario is a codec impairment, the impairment degree is determined based on the bit rate.

[0032] In a possible implementation manner, obtaining the impaired speech features and the predicted quality of the plurality of impaired speech signals by using the speech quality determination model includes:

[0033] Preprocessing the damaged speech signal to obtain conventional speech features of the damaged speech signal;

[0034] The conventional speech feature is input into the speech quality determination model to obtain the impaired speech feature and the predicted quality of the impaired speech signal.

[0035] According to a second aspect of an embodiment of the present disclosure, a method for determining voice quality is provided, the method comprising:

[0036] Acquire speech signals;

[0037] The speech signal is input into a speech quality determination model to obtain the speech quality of the speech signal, and the speech quality determination model is pre-trained based on the pre-training method of the speech quality determination model provided in the first aspect above to obtain the speech quality determination model.

[0038] In a possible implementation, the speech quality includes a mean opinion score (MOS), a perceptual evaluation of speech quality (PESQ), and / or a short-term objective intelligibility (STOI).

[0039] According to a third aspect of an embodiment of the present disclosure, a pre-training device for a speech quality determination model is provided, the device comprising:

[0040] an acquisition unit configured to acquire impaired speech features and predicted quality of a plurality of impaired speech signals through a speech quality determination model, wherein the plurality of impaired speech signals are generated based on a reference speech signal and impairment information, wherein the impairment information indicates an impairment type for the reference speech signal, and the impairment type is used to indicate an impairment scenario and an impairment degree;

[0041] A first determining unit is configured to determine a first loss value of the voice quality determination model based on the multiple impaired speech features, the first loss value being used to characterize the distance between impaired speech features of the same impairment type and the distance between impaired speech features of different impairment types;

[0042] The pre-training unit is configured to pre-train the speech quality determination model based on the first loss value.

[0043] In one possible implementation, the pre-training unit is configured to perform:

[0044] Based on the first loss value, the model parameters of the speech quality determination model are adjusted so that among the multiple damaged speech features obtained by the adjusted speech quality determination model, the distance between the damaged speech features of the same damage type is smaller, and the distance between the damaged speech features of different damage types is larger.

[0045] In one possible implementation, the device further includes:

[0046] A second determination unit is configured to determine a second loss value of the speech quality determination model based on the reference quality and the predicted quality of the multiple impaired speech signals, wherein the second loss value is used to characterize an error between the predicted quality and the reference quality, and the reference quality is determined based on the multiple impaired speech signals and the reference speech signal;

[0047] The pre-training unit is configured to perform:

[0048] Based on the first loss value and the second loss value, the model parameters of the speech quality determination model are adjusted so that among the multiple damaged speech features and predicted qualities obtained by the adjusted speech quality determination model, the distance between damaged speech features of the same damage type is smaller, the distance between damaged speech features of different damage types is larger, and the error between the predicted quality and the reference quality is smaller.

[0049] In one possible implementation, the device further includes:

[0050] A signal processing unit is configured to process a first reference signal in the reference speech signal according to a degree of damage corresponding to the first damage type based on the first damage type included in the damage information, so as to obtain a first damage signal of the first reference signal under the first damage type;

[0051] The quality determination unit is configured to determine a reference quality of the first impairment signal based on the first impairment signal and the first reference signal, wherein the reference quality indicates a speech quality of the first impairment signal relative to the first reference signal.

[0052] In a possible implementation, the acquisition unit is configured to execute:

[0053] Determine speech feature information of the damaged speech signal on the damage type through multiple two-dimensional convolutional layers of the speech quality determination model;

[0054] Determining characteristic information of the impaired speech signal in a time dimension through a plurality of bidirectional recurrent network layers of the speech quality determination model;

[0055] Based on the speech feature information and the feature information in the time dimension, outputting the damaged speech feature through a first fully connected layer of the speech quality determination model;

[0056] Based on the impaired speech features, the predicted quality is output through the second fully connected layer of the speech quality determination model, and the predicted quality includes a perceptual evaluation of speech quality PESQ and / or short-time objective intelligibility STOI.

[0057] In one possible implementation, the device further includes:

[0058] A training unit is configured to input a target speech signal into the speech quality determination model obtained by pre-training to obtain a predicted mean opinion score MOS of the target speech signal, wherein the target speech signal carries a mean opinion score MOS annotation;

[0059] Determine a third loss value based on the predicted mean opinion score and the mean opinion score annotation carried by the target speech signal, wherein the third loss value is used to characterize an error between the predicted mean opinion score and the mean opinion score annotation;

[0060] Based on the third loss value, the speech quality determination model obtained by pre-training is trained.

[0061] In a possible implementation manner, when the damage scene is ambient sound damage, the damage degree is determined based on the content of the ambient sound and the signal-to-noise ratio;

[0062] In the case where the damage scenario is reverberation damage, the degree of damage is determined based on the reverberation time;

[0063] When the damage scenario is device frequency response damage, the damage degree is determined based on the cutoff frequency of the filter;

[0064] When the impairment scenario is a codec impairment, the impairment degree is determined based on the bit rate.

[0065] In a possible implementation, the acquisition unit is configured to execute:

[0066] Preprocessing the damaged speech signal to obtain conventional speech features of the damaged speech signal;

[0067] The conventional speech feature is input into the speech quality determination model to obtain the impaired speech feature and the predicted quality of the impaired speech signal.

[0068] According to a fourth aspect of an embodiment of the present disclosure, a speech quality determination device is provided, the device comprising:

[0069] A signal acquisition unit, configured to acquire a voice signal;

[0070] The quality determination unit is configured to input the speech signal into a speech quality determination model to obtain the speech quality of the speech signal, and the speech quality determination model is pre-trained based on the pre-training method of the speech quality determination model.

[0071] In a possible implementation, the speech quality includes a mean opinion score (MOS), a perceptual evaluation of speech quality (PESQ), and / or a short-term objective intelligibility (STOI).

[0072] According to a fifth aspect of an embodiment of the present disclosure, an electronic device is provided, the electronic device comprising:

[0073] one or more processors;

[0074] A memory for storing program codes executable by the processor;

[0075] The processor is configured to execute the program code to implement the above-mentioned pre-training method of the speech quality determination model or the speech quality determination method.

[0076] According to the sixth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, which includes: when the program code in the computer-readable storage medium is executed by a processor of an electronic device, the server is enabled to execute the above-mentioned pre-training method of the speech quality determination model or the speech quality determination method.

[0077] According to a seventh aspect of an embodiment of the present disclosure, there is provided a computer program product, comprising a computer program, which, when executed by a processor, implements the above-mentioned pre-training method of the speech quality determination model or the speech quality determination method.

[0078] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0079] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations on the present disclosure.

[0080] Figure 1 is a schematic diagram of an implementation environment of a pre-training method for a voice quality determination model according to an exemplary embodiment;

[0081] Figure 2 is a flow chart of a method for pre-training a voice quality determination model according to an exemplary embodiment;

[0082] Figure 3 is a flow chart of a method for pre-training a voice quality determination model according to an exemplary embodiment;

[0083] Figure 4 is a schematic diagram showing a process of generating a damaged speech signal according to an exemplary embodiment;

[0084] Figure 5 is a structural diagram of a two-dimensional gated convolution module according to an exemplary embodiment;

[0085] Figure 6 is a schematic diagram of a pre-training process according to an exemplary embodiment;

[0086] Figure 7 is a framework diagram of a training system for a speech quality determination model according to an exemplary embodiment;

[0087] Figure 8 is a flow chart of a method for determining voice quality according to an exemplary embodiment;

[0088] Fig. 9 is a block diagram of a pre-training device for a speech quality determination model according to an exemplary embodiment;

[0089] Fig.10 is a block diagram of a device for determining speech quality according to an exemplary embodiment;

[0090] Fig.11 It is a block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0091] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings.

[0092] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0093] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.) and signals involved in this disclosure are all authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions. For example, the reference voice signals involved in this application are all obtained with full authorization.

[0094] Figure 1 is a schematic diagram of an implementation environment of a pre-training method for a voice quality determination model according to an exemplary embodiment, see Figure 1 , the implementation environment includes: a terminal 101 and a server 102.

[0095] The terminal 101 may be at least one of a smart phone, a smart watch, a desktop computer, a laptop computer, a virtual reality terminal, an augmented reality terminal, a wireless terminal, and a laptop computer. The terminal 101 has a communication function and can access the Internet. The terminal 101 may generally refer to one of a plurality of terminals. This embodiment is only illustrated by the terminal 101. The terminal 101 may run various types of applications, such as a voice call application, a music application, etc. Optionally, the terminal 101 may send various voice signals to the server 102, and determine the quality of the voice signal through a voice quality determination model provided by the server 102.

[0096] Optionally, the server is associated with a voice database, which includes multiple types of voice signals. The server 102 can train a voice quality determination model based on the voice database, and can also determine the voice quality of the voice signal in the terminal 101 through the voice quality determination model. The server 102 can be an independent physical server, or a server cluster or distributed file system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN), and big data and artificial intelligence platforms.

[0097] The server 102 and the terminal 101 can be directly or indirectly connected via wired or wireless communication, which is not limited in the embodiments of the present disclosure. Optionally, the number of the above-mentioned terminals 101 and servers 102 can be more or less, which is not limited in the embodiments of the present disclosure. Of course, the server 102 can also include other functional servers to provide more comprehensive and diversified services.

[0098] Figure 2 is a flowchart of a method for pre-training a voice quality determination model according to an exemplary embodiment. Figure 2 As shown, the method is executed by the server and includes the following steps:

[0099] In step 201, the server obtains the impaired speech features and predicted quality of multiple impaired speech signals through a speech quality determination model. The multiple impaired speech signals are generated based on a reference speech signal and impairment information. The impairment information represents an impairment type for the reference speech signal. The impairment type is used to indicate an impairment scenario and an impairment degree.

[0100] Among them, based on the reference speech signal and the damage information, speech signals of various types of damage can be effectively simulated, that is, the multiple damaged speech signals. Based on this, the server processes the damaged speech signal through the speech quality determination model, and can obtain damaged speech features with various damage degrees in various damage scenarios, thereby effectively enhancing the coverage of model training for various damage scenarios, thereby improving the generalization of the speech quality determination model.

[0101] In some embodiments, the reference speech signal is considered to be an undamaged speech signal and can be used as a speech signal with standard quality to simulate damaged speech signals in different scenarios. In some embodiments, the damage scenario indicates the way in which the speech signal is damaged, for example, environmental noise; the degree of damage can be determined based on parameters corresponding to the damage scenario, for example, the signal-to-noise ratio of the environmental noise. Accordingly, different degrees of damage in the same damage scenario can correspond to different types of damage, for example, environmental noise with different signal-to-noise ratios for different types of damage.

[0102] The predicted quality is predicted by the speech quality determination model based on the multiple impaired speech features. In some embodiments, the predicted quality is an objective evaluation index of speech quality, such as perceptual evaluation of speech quality (PESQ) and short-time objective intelligibility (STOI).

[0103] In step 202, the server determines a first loss value of the speech quality determination model based on the multiple impaired speech features, where the first loss value is used to characterize the distance between impaired speech features of the same impairment type and the distance between impaired speech features of different impairment types.

[0104] In some embodiments, the server determines the distance between the impaired speech features of the same impairment type and the distance between the impaired speech features of different impairment types based on the impairment types corresponding to the multiple impaired speech features. It can be understood that the smaller the distance between the impaired speech features of the same impairment type, the higher the accuracy of the feature extraction of the voice quality determination model for the impairment type; the larger the distance between the impaired speech features of different impairment types, the stronger the ability of the voice quality determination model to distinguish between multiple impairment types.

[0105] In step 203, the server pre-trains the speech quality determination model based on the first loss value.

[0106] In the disclosed embodiment, the server can adjust the model parameters of the voice quality determination model based on the first loss value.

[0107] In some embodiments, adjusting the model parameters based on the first loss value can make the distance between the damaged speech features of the same damage type smaller and the distance between the damaged speech features of different damage types larger among the multiple damaged speech features acquired by the pre-trained speech quality determination model, thereby improving the feature extraction capability of the speech quality determination model for multiple damage types, so that when facing damaged speech signals of multiple damage types, the speech quality determination model not only has a strong resolution capability, but also can extract damaged speech features with high accuracy.

[0108] Through the technical solution provided by the embodiments of the present disclosure, the generalization of the speech quality determination model for various damage scenarios can be effectively improved without the need for a large number of speech signals carrying sound quality annotations, ensuring that the speech quality determination model not only has a strong resolution capability when facing damaged speech signals of various damage types, but can also extract highly accurate damaged speech features, thereby greatly improving the accuracy of predicting speech quality.

[0109] Above Figure 2 What is shown is only the basic process of the present disclosure, and the solution provided by the present disclosure is further explained below. Figure 3 is a flowchart of a method for pre-training a voice quality determination model according to an exemplary embodiment. Figure 3 As shown, the method is executed by the above server and includes the following steps:

[0110] In step 301, the server generates a plurality of impaired speech signals based on a reference speech signal and impairment information, wherein the impairment information represents an impairment type for the reference speech signal, and the impairment type is used to indicate an impairment scenario and an impairment degree.

[0111] This step refers to step 201.

[0112] In some embodiments, the server is associated with a voice database, and the voice database stores a reference voice signal. In other embodiments, the reference voice signal can be sent by the terminal to the server for use, which is not limited in the embodiments of the present disclosure.

[0113] In some embodiments, when the damage scenario is ambient sound damage, the damage degree is determined based on the content of the ambient sound and the signal-to-noise ratio. For example, the content of the ambient sound may be ambient noise such as cars, subways, music, keyboards, mice, etc., and the signal-to-noise ratio may be -5 decibels (dB), 0dB, 5dB, 10dB, 20dB, etc. It can be understood that the signal-to-noise ratio is the ratio of the effective signal power to the noise signal power, which can indicate the intensity of the noise in the ambient sound, thereby reflecting the degree of damage of the ambient sound to the speech signal.

[0114] In some embodiments, when the damage scenario is reverberation damage, the degree of damage is determined based on the reverberation time, for example, the reverberation time under the sound field attenuation 60dB (Reverberation Time 60dB, RT60) standard is 0.3 seconds, 0.6 seconds, 0.9 seconds or 1.2 seconds, wherein the greater the reverberation time, the longer the noise lasts, and the more severe the damage to the speech signal.

[0115] In some embodiments, when the damage scenario is device frequency response damage, the degree of damage is determined based on the cutoff frequency of the filter, for example, 1000 Hz, 2400 Hz, 3600 zHz or 6000 Hz, wherein the frequency response range of the device determines the ability of the device to support different frequency components in the voice signal, and the cutoff frequency generally indicates the upper and lower limits of the device's capabilities.

[0116] In some embodiments, when the damage scenario is codec damage, the degree of damage is determined based on the bit rate, for example, 3 kilobits per second (kbps), 6kbps, 12kbps or 24kbps, wherein the larger the bit rate, the lower the degree of compression of the voice signal, and the less damage the codec causes to the voice quality.

[0117] In some embodiments, the server processes the first reference signal in the reference speech signal according to the degree of damage corresponding to the first damage type based on the first damage type included in the damage information to obtain a first damage signal of the first reference signal under the first damage type.

[0118] In order to facilitate understanding of the above process of generating a damaged speech signal, the present disclosure provides a schematic diagram of a process of generating a damaged speech signal. Figure 4 . Figure 4 is a schematic diagram of a damaged speech signal generation process according to an exemplary embodiment, wherein 401 is the nth group of n groups of reference speech signals, including reference speech signals and the reference speech signal 402 is the nth group of n groups of loss information, damage type and injury type The server combines 401 and 402 in pairs to perform impairment simulation and generate four impaired speech signals 403: and

[0119] In step 302, the server determines reference qualities of the multiple impaired speech signals based on the multiple impaired speech signals and the reference speech signal.

[0120] The reference speech signal is considered to be an unimpaired speech signal and can be used as a relative standard for speech quality. In some embodiments, the server determines a reference quality of the first impaired signal based on a first impaired signal among multiple impaired speech signals and a first reference signal among the reference speech signals, and the reference quality indicates the speech quality of the first impaired signal relative to the first reference signal.

[0121] It is understandable that the reference quality can be used as a training target to improve the prediction accuracy of the voice quality determination model for voice quality during the model training process. Optionally, the reference quality can be an objective evaluation indicator of voice quality, such as perceptual evaluation of speech quality (PESQ) and short-time objective intelligibility (STOI).

[0122] It should be noted that step 302 can be performed before step 308 .

[0123] In step 303, the server determines the speech feature information of the damaged speech signal in terms of the damage type through multiple two-dimensional convolutional layers of the speech quality determination model.

[0124] In the embodiment of the present disclosure, the multiple two-dimensional convolution layers can focus on learning the speech feature information of the damaged speech signal on the damage type during the feature extraction process. Optionally, the two-dimensional convolution layer can be a two-dimensional gated convolution (Gated-Convolution2d, GConv2d) module, which enables the voice quality determination model to focus on learning important features in the damaged speech signal based on a dynamic feature selection mechanism similar to the self-attention mechanism, that is, the speech feature information on the damage type. Exemplarily, a feature selection mechanism based on parameter learning can be applied to multiple (convolution output) channels of the two-dimensional convolution layer, so that during the training process, the focus is gradually placed on channels that can reflect important features. Further, through multiple two-dimensional convolution layers, the number of channels is gradually increased to refine the granularity of feature extraction, see Table 1, Table 1 is a model structure and parameter setting table provided in the embodiment of the present disclosure, wherein O is a custom voice quality prediction dimension and is a positive integer; Dense is a fully connected layer.

[0125] Table 1

[0126]

[0127] In some embodiments, the speech feature information includes weight values ​​learned by the damaged speech signal through multiple two-dimensional convolutional layers.

[0128] For ease of understanding, the present disclosure provides a structural diagram of a two-dimensional gated convolution module, see Figure 5 . Figure 5 It is a structural diagram of a two-dimensional gated convolution module according to an exemplary embodiment, wherein the first branch of GConv2d includes a two-dimensional convolution layer conv2d, a batch normalization layer batchnorm for accelerating convergence, and a random dropout layer dropout for avoiding overfitting, and the second branch of GConv2d includes a two-dimensional convolution layer conv2d and an activation function layer sigmoid, wherein a dynamic feature selection mechanism similar to the self-attention mechanism is introduced in the second branch to improve the accuracy of feature extraction.

[0129] In some embodiments, the server preprocesses the impaired speech signal to obtain the conventional speech features of the impaired speech signal, and then inputs the conventional speech features of the impaired speech signal into the speech quality determination model for processing. The preprocessing can highlight the speech features of the impaired speech signal to improve the efficiency of subsequent model training.

[0130] Optionally, the conventional speech feature may be a Mel spectrum feature of an impaired speech signal, or may be a speech feature obtained by transforming the signal into the frequency domain or the spatial domain by other signal processing means, which is not limited in the embodiments of the present disclosure. Exemplarily, the process of preprocessing to obtain the conventional speech feature includes: performing a short-time Fourier transform on the impaired speech signal to obtain a spectrum, and then performing a dot multiplication with a plurality of Mel filters to obtain the Mel spectrum of the impaired speech signal, wherein the number of filters may be determined based on the sampling frequency of the signal, for example, for an impaired speech signal with a sampling frequency of 16kHz, the number of Mel filters may be set to 80.

[0131] In step 304, the server determines feature information of the impaired speech signal in the time dimension through multiple bidirectional recurrent network layers of the speech quality determination model.

[0132] Among them, since the damaged speech signal is composed of multiple speech frames with a time sequence relationship, after feature extraction, the bidirectional gate recurrent unit (BiGRU) can be used to learn the characteristic information of the damaged speech signal in the time dimension. Through the multiple bidirectional cycles, the dependencies of multiple speech frames of the damaged speech signal in the time series can be learned based on multiple reverse updated hidden layers, thereby constructing the characteristics of the damaged speech signal in the time dimension. Optionally, by setting the number of RNNs in each layer of BiGRU to gradually decrease, the model parameters are gradually converged from the shallow layer to the deep layer, see Table 1.

[0133] In step 305, the server outputs the damaged speech features through the first fully connected layer of the speech quality determination model based on the speech feature information and the feature information in the time dimension, and executes steps 306 and 307.

[0134] The first fully connected layer integrates the speech feature information and the feature information in the time dimension obtained by layer-by-layer processing to output the damaged speech feature.

[0135] In step 306, the server outputs the predicted quality of the impaired speech signal through the second fully connected layer of the speech quality determination model based on the impaired speech features, and executes step 308. The predicted quality includes the perceptual evaluation of speech quality PESQ and / or short-time objective intelligibility STOI.

[0136] The second fully connected layer predicts the damaged speech features output by the first fully connected layer to output the predicted quality. Optionally, the predicted quality may include multiple quality prediction dimensions (i.e., O in Table 1), and the server may set the parameters of the second fully connected layer according to the required quality evaluation index, which is not limited in the embodiments of the present disclosure.

[0137] In step 307, the server determines a first loss value of the speech quality determination model based on the multiple impaired speech features, where the first loss value is used to characterize the distance between impaired speech features of the same impairment type and the distance between impaired speech features of different impairment types.

[0138] This step refers to step 203.

[0139] In some embodiments, the first loss value can be determined by the following formulas (1) to (5).

[0140]

[0141] Among them, x n is the distance between the nth group of impaired speech features belonging to the same impairment type; and are two different reference speech signals, and Indicates two different types of injuries; and are two impaired speech signals belonging to the same impairment type, and are two impaired speech signals belonging to the same impairment type; F θ is the parameter of the speech quality determination model for extracting damaged speech features; ‖‖2 is the L2 norm.

[0142]

[0143] Among them, x′ n is the distance between the nth group of impaired speech features belonging to different impairment types; and are two different reference speech signals, and Indicates two different types of injuries; and are two impaired speech signals belonging to different impairment types. and are two damaged speech signals belonging to different damage types; F θ is the parameter of the speech quality determination model for extracting damaged speech features; ‖‖2 is the L2 norm.

[0144]

[0145] X=[x1,…x n ,x′1,…,x′ n ] (4)

[0146] Y=[1,…1,0,…,0,] (5)

[0147] in, The value of is the first loss value; wherein X is a 2N-dimensional vector, the first N vector values ​​are the distances between damaged speech features of the same damage type determined based on formula (1), and the last N vector values ​​are the distances between damaged speech features of different damage types determined based on formula (2); Y is a 2N-dimensional vector, the first N vector values ​​are 1, and the last N vector values ​​are 0; the parameter m of the maximum function can be set according to the calculation requirements.

[0148] In step 308, the server determines a second loss value of the speech quality determination model based on the reference quality and predicted quality of the multiple impaired speech signals, and the second loss value is used to characterize the error between the predicted quality and the reference quality, and the reference quality is determined based on the multiple impaired speech signals and the reference speech signal.

[0149] In some embodiments, the server can determine the second loss value based on a mean square error (MSE) between the predicted quality and the reference quality, and the larger the MSE is, the larger the second loss value is.

[0150] Optionally, in the case where the predicted quality includes a perceptual evaluation of speech quality PESQ and a short-term objective intelligibility STOI, the second loss value includes a value determined based on a mean square error between the predicted PESQ and a reference PESQ. and is determined based on the mean square error between the predicted STOI and the reference STOI

[0151] In step 309, the server adjusts the model parameters of the speech quality determination model based on the first loss value and the second loss value.

[0152] In some embodiments, the server can determine the total loss value used to pre-train the speech quality determination model based on the sum of the first loss value and the second loss value. Refer to formula (6).

[0153]

[0154] It can be understood that referring to the first loss value and the second loss value at the same time can effectively strengthen the correlation between the damaged speech feature and the speech quality, and thus efficiently improve the performance of the speech quality determination model from the two perspectives of feature extraction accuracy and speech quality prediction accuracy.

[0155] Among them, by adjusting the model parameters, the distance between damaged speech features of the same damage type and the distance between damaged speech features of different damage types can be smaller, and the error between the predicted quality and the reference quality can be smaller among the multiple damaged speech features and the predicted quality obtained by the adjusted speech quality determination model.

[0156] In other embodiments, the server can adjust the model parameters of the speech quality determination model based only on the first loss value, and improve the accuracy of predicting speech quality on the basis that the model can extract damaged speech features with high accuracy, thereby saving certain computing resources.

[0157] In order to facilitate understanding of the principle of the above-mentioned pre-training process to change the distance between the damaged speech features, the present disclosure embodiment provides a schematic diagram of the pre-training process, see Figure 6 . Figure 6 is a schematic diagram of a pre-training process according to an exemplary embodiment, wherein the speech quality determination model is used for the damaged speech signal and Perform feature extraction to obtain the corresponding four damaged speech features, and adjust the parameters of the speech quality determination model to make the distance between damaged speech features of the same damage type smaller and the distance between damaged speech features of different damage types larger. and are two different reference speech signals, and Indicates two different types of damage, and are two impaired speech signals belonging to different impairment types. and are two impaired speech signals belonging to the same impairment type.

[0158] The above steps 301 to 309 illustrate the process of pre-training the speech quality determination model. The server can end the pre-training of the speech quality determination model when the total loss value is less than the target threshold or the number of training iterations reaches the target number.

[0159] In some embodiments, after completing the pre-training of the speech quality determination model, the server performs the training process of steps 1 to 3 below to fine-tune the speech quality determination model.

[0160] In step 1, the server inputs the target speech signal into the pre-trained speech quality determination model to obtain a predicted mean opinion score of the target speech signal, where the target speech signal carries a mean opinion score label.

[0161] Among them, the speech quality determination model obtained based on pre-training can extract features for various types of damage, and has reliable accuracy in predicting various quality evaluation indicators such as PESQ and STOI. Based on this, the server can fine-tune the pre-trained speech quality determination model based on the prediction needs of speech quality, for example, for the subjective quality evaluation indicator of speech quality: Mean Opinion Score (MOS), through a small amount of labeled data.

[0162] It is understandable that since subjective quality assessment indicators require high labor costs to obtain, the technical solution provided by the embodiments of the present disclosure only requires a small amount of data carrying MOS annotations to train a speech quality determination model that can accurately predict the MOS of speech signals.

[0163] In step 2, the server determines a third loss value based on the predicted mean opinion score and the mean opinion score annotation carried by the target speech signal, where the third loss value is used to characterize the error between the predicted mean opinion score and the mean opinion score annotation.

[0164] This step is the same as step 308 and will not be described in detail here.

[0165] In step 3, the server trains the pre-trained speech quality determination model based on the third loss value.

[0166] Among them, the server can adjust the model parameters of the pre-trained speech quality determination model based on the third loss value, thereby improving the accuracy of its predicted mean opinion score on the basis of its ability to accurately extract damaged speech features of various damage types.

[0167] It can be understood that the above process only takes the mean opinion score MOS as an example. In other embodiments, the above steps 1 to 3 can be performed based on any speech quality evaluation index, which is not limited in the embodiments of the present disclosure.

[0168] Through the above technical solution, there is no need to carry a large number of speech signals with sound quality annotations, which can effectively improve the generalization of the speech quality determination model for various damage scenarios, ensure that the model has strong resolution capabilities for damaged speech signals of various damage types, and can extract high-accuracy damaged speech features, thereby greatly improving the accuracy of predicting speech quality.

[0169] Furthermore, referring to the first loss value obtained based on the damaged speech feature and the second loss value obtained for the predicted quality at the same time can effectively strengthen the correlation between the damaged speech feature and the speech quality, and then efficiently improve the generalization and prediction accuracy of the speech quality determination model from the two perspectives of feature extraction accuracy and speech quality prediction accuracy.

[0170] In order to facilitate understanding of the pre-training stage and the subsequent fine-tuning stage of the above-mentioned voice quality determination model, the present disclosure provides a training system framework diagram of the voice quality determination model, see Figure 7 . Figure 7 The present invention is a training system framework diagram of a speech quality determination model according to an exemplary embodiment, wherein the pre-training stage includes the process of generating a damaged speech signal and determining a reference quality as described in steps 301 to 302; the speech quality determination model includes a two-dimensional convolutional layer, a bidirectional recurrent network layer, and a fully connected layer; through steps 303 to 307, damaged speech features and predicted quality (including PESQ and STOI) are obtained; in the fine-tuning stage, a target speech signal carrying a MOS annotation is used to obtain a predicted MOS based on the speech quality determination model obtained by pre-training.

[0171] Based on the pre-training method of the speech quality determination model introduced above, the process of using the speech quality determination model to determine speech quality is described below. Figure 8 is a flow chart of a method for determining voice quality according to an exemplary embodiment. Figure 8 As shown, the method is executed by the above server and includes the following steps:

[0172] In step 801, the server acquires a voice signal.

[0173] In some embodiments, the server can obtain the voice signal uploaded by the terminal, thereby providing a voice quality determination service for the terminal. Optionally, the voice signal can be obtained in real time during the voice call of the terminal, thereby providing a real-time voice quality detection service for the terminal. It should be noted that the process of obtaining the voice signal from the terminal is carried out after full authorization.

[0174] In step 802, the server inputs the speech signal into a speech quality determination model to obtain the speech quality of the speech signal. The speech quality determination model is based on Figure 2 or Figure 3 The speech quality determination model obtained by pre-training the speech quality determination model provided by the corresponding embodiment is trained.

[0175] In some embodiments, the speech quality includes a mean opinion score (MOS). In other embodiments, the speech quality may also include a perceptual evaluation of speech quality (PESQ) or a short-term objective intelligibility (STOI). The speech quality includes an indicator based on a quality prediction dimension of a speech quality determination model (see Figure 3 The corresponding embodiments) are determined, and the embodiments of the present disclosure do not limit this.

[0176] In some embodiments, based on the above Figure 2 or Figure 3 The speech quality determination model obtained by pre-training the corresponding speech quality determination model pre-training method can extract features for various types of damage and has reliable accuracy in predicting various quality evaluation indicators such as PESQ and STOI. Therefore, by continuing to train on the basis of the speech quality determination model obtained by the above pre-training, it is possible to quickly obtain a speech quality determination model with reliable accuracy for the specified quality prediction dimension according to the prediction requirements for speech quality. In this way, in step 802, the server inputs the speech signal into the speech quality determination model obtained by continuing to train after pre-training, and the speech quality of the speech signal can be accurately obtained. Among them, the process of continuing training can refer to the above Figure 3 The fine-tuning phase described in steps 1 to 3 in the corresponding embodiment will not be described in detail here.

[0177] Through the above technical solution, accurate voice quality can be obtained in various complex scenarios that may damage voice quality, thereby providing a stable and effective reference for improving voice quality.

[0178] Fig. 9 is a block diagram of a pre-training device for a speech quality determination model according to an exemplary embodiment. Fig. 9 , the device comprises:

[0179] An acquisition unit 901 is configured to acquire impaired speech features and predicted quality of a plurality of impaired speech signals through a speech quality determination model, wherein the plurality of impaired speech signals are generated based on a reference speech signal and impairment information, wherein the impairment information indicates an impairment type for the reference speech signal, and the impairment type is used to indicate an impairment scenario and an impairment degree;

[0180] A first determining unit 902 is configured to determine a first loss value of the voice quality determination model based on the multiple impaired speech features, the first loss value being used to characterize the distance between impaired speech features of the same impairment type and the distance between impaired speech features of different impairment types;

[0181] The pre-training unit 903 is configured to pre-train the speech quality determination model based on the first loss value.

[0182] In one possible implementation, the pre-training unit 903 is configured to execute:

[0183] Based on the first loss value, the model parameters of the speech quality determination model are adjusted so that among the multiple damaged speech features obtained by the adjusted speech quality determination model, the distance between the damaged speech features of the same damage type is smaller, and the distance between the damaged speech features of different damage types is larger.

[0184] In one possible implementation, the device further includes:

[0185] A second determination unit is configured to determine a second loss value of the speech quality determination model based on the reference quality and the predicted quality of the multiple impaired speech signals, wherein the second loss value is used to characterize an error between the predicted quality and the reference quality, and the reference quality is determined based on the multiple impaired speech signals and the reference speech signal;

[0186] The pre-training unit 903 is configured to perform:

[0187] Based on the first loss value and the second loss value, the model parameters of the speech quality determination model are adjusted so that among the multiple damaged speech features and predicted qualities obtained by the adjusted speech quality determination model, the distance between damaged speech features of the same damage type is smaller, the distance between damaged speech features of different damage types is larger, and the error between the predicted quality and the reference quality is smaller.

[0188] In one possible implementation, the device further includes:

[0189] A signal processing unit is configured to process a first reference signal in the reference speech signal according to a degree of damage corresponding to the first damage type based on the first damage type included in the damage information, so as to obtain a first damage signal of the first reference signal under the first damage type;

[0190] The quality determination unit is configured to determine a reference quality of the first impairment signal based on the first impairment signal and the first reference signal, wherein the reference quality indicates a speech quality of the first impairment signal relative to the first reference signal.

[0191] In a possible implementation manner, the obtaining unit 901 is configured to execute:

[0192] Determine speech feature information of the damaged speech signal on the damage type through multiple two-dimensional convolutional layers of the speech quality determination model;

[0193] Determining characteristic information of the impaired speech signal in a time dimension through a plurality of bidirectional recurrent network layers of the speech quality determination model;

[0194] Based on the speech feature information and the feature information in the time dimension, outputting the damaged speech feature through a first fully connected layer of the speech quality determination model;

[0195] Based on the impaired speech features, the predicted quality is output through the second fully connected layer of the speech quality determination model, and the predicted quality includes a perceptual evaluation of speech quality PESQ and / or short-time objective intelligibility STOI.

[0196] In one possible implementation, the device further includes:

[0197] A training unit is configured to input a target speech signal into the speech quality determination model obtained by pre-training to obtain a predicted mean opinion score MOS of the target speech signal, wherein the target speech signal carries a mean opinion score MOS annotation;

[0198] Determine a third loss value based on the predicted mean opinion score and the mean opinion score annotation carried by the target speech signal, wherein the third loss value is used to characterize an error between the predicted mean opinion score and the mean opinion score annotation;

[0199] Based on the third loss value, the speech quality determination model obtained by pre-training is trained.

[0200] In a possible implementation manner, when the damage scene is ambient sound damage, the damage degree is determined based on the content of the ambient sound and the signal-to-noise ratio;

[0201] In the case where the damage scenario is reverberation damage, the degree of damage is determined based on the reverberation time;

[0202] When the damage scenario is device frequency response damage, the damage degree is determined based on the cutoff frequency of the filter;

[0203] When the impairment scenario is a codec impairment, the impairment degree is determined based on the bit rate.

[0204] In a possible implementation manner, the obtaining unit 901 is configured to execute:

[0205] Preprocessing the damaged speech signal to obtain conventional speech features of the damaged speech signal;

[0206] The conventional speech feature is input into the speech quality determination model to obtain the impaired speech feature and the predicted quality of the impaired speech signal.

[0207] Through the above technical solution, there is no need to carry a large number of speech signals with sound quality annotations, and the generalization of the speech quality determination model for various damage scenarios can be effectively improved. This ensures that the speech quality determination model not only has strong resolution capabilities when facing damaged speech signals of various damage types, but can also extract highly accurate damaged speech features, thereby greatly improving the accuracy of predicting speech quality.

[0208] It should be noted that: the pre-training device for determining the voice quality model provided in the above embodiment only uses the division of the above functional modules as an example when executing the corresponding steps. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the pre-training device for determining the voice quality model provided in the above embodiment and the pre-training method embodiment of the voice quality determination model belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0209] Fig.10 is a block diagram of a device for determining speech quality according to an exemplary embodiment. Fig.10 , the device comprises:

[0210] The signal acquisition unit 1001 is configured to acquire a speech signal;

[0211] The quality determination unit 1002 is configured to input the speech signal into a speech quality determination model to obtain the speech quality of the speech signal. The speech quality determination model is trained based on the speech quality determination model pre-trained by the pre-training method of the above-mentioned speech quality determination model.

[0212] In a possible implementation, the speech quality includes a mean opinion score (MOS), a perceptual evaluation of speech quality (PESQ), and / or a short-term objective intelligibility (STOI).

[0213] Through the above technical solution, accurate voice quality can be obtained in various complex scenarios that may damage voice quality, thereby providing a stable and effective reference for improving voice quality.

[0214] It should be noted that: when the voice quality determination device provided in the above embodiment performs the corresponding steps, only the division of the above functional modules is used as an example. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the voice quality determination device provided in the above embodiment and the voice quality determination method embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0215] Fig.11It is a block diagram of an electronic device shown according to an exemplary embodiment, and the electronic device 1100 can be implemented as the above-mentioned terminal or server. The electronic device 1100 may have relatively large differences due to different configurations or performances, and may include one or more processors (Central Processing Units, CPU) 1101 and one or more memories 1102, wherein the one or more memories 1102 store at least one program code, and the at least one program code is loaded and executed by the one or more processors 1101 to implement the pre-training method of the voice quality determination model provided by the above-mentioned various method embodiments or the process executed by the server in the voice quality determination method. Of course, the electronic device 1100 may also have components such as a wired or wireless network interface, a keyboard, and an input and output interface for input and output, and the electronic device 1100 may also include other components for implementing device functions, which will not be repeated here.

[0216] In an exemplary embodiment, a computer-readable storage medium including a program code is also provided, such as a memory 1102 including the program code, and the program code can be executed by the processor 1101 of the electronic device 1100 to complete the pre-training method of the voice quality determination model or the voice quality determination method. Optionally, the computer-readable storage medium can be a read-only memory (ROM), a random access memory (RAM), a compact disc (CD-ROM), a magnetic tape, a floppy disk, an optical data storage device, etc.

[0217] In an exemplary embodiment, a computer program product is also provided, including a computer program, which implements the above-mentioned pre-training method of the speech quality determination model or the speech quality determination method when executed by a processor.

[0218] In some embodiments, the computer program involved in the embodiments of the present disclosure may be deployed and executed on a computer device, or on multiple computer devices located at one location, or on multiple computer devices distributed at multiple locations and interconnected through a communication network. Multiple computer devices distributed at multiple locations and interconnected through a communication network may constitute a blockchain system.

[0219] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0220] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A pre-training method for a speech quality determination model, characterized in that: The method comprises: Obtaining impaired speech features and predicted quality of a plurality of impaired speech signals through a speech quality determination model, wherein the plurality of impaired speech signals are generated based on a reference speech signal and impairment information, wherein the reference speech signal is an unimpaired speech signal, and the impairment information indicates an impairment type for the reference speech signal, wherein the impairment type is used to indicate an impairment scenario and an impairment degree; Determine a first loss value of the voice quality determination model based on the multiple impaired speech features, the first loss value being used to characterize the distance between impaired speech features of the same impairment type and the distance between impaired speech features of different impairment types; Based on the first loss value, the model parameters of the speech quality determination model are adjusted so that among the multiple damaged speech features obtained by the adjusted speech quality determination model, the distance between the damaged speech features of the same damage type is smaller, and the distance between the damaged speech features of different damage types is larger.

2. The pre-training method for the voice quality determination model according to claim 1, characterized in that: The method further comprises: Determine a second loss value of the speech quality determination model based on the reference quality and the predicted quality of the plurality of impaired speech signals, wherein the second loss value is used to characterize an error between the predicted quality and the reference quality, wherein the reference quality is determined based on the plurality of impaired speech signals and the reference speech signal; The pre-training of the speech quality determination model based on the first loss value comprises: Based on the first loss value and the second loss value, the model parameters of the speech quality determination model are adjusted so that among the multiple damaged speech features and predicted qualities obtained by the adjusted speech quality determination model, the distance between damaged speech features of the same damage type is smaller, the distance between damaged speech features of different damage types is larger, and the error between the predicted quality and the reference quality is smaller.

3. The pre-training method of the speech quality determination model according to claim 1, characterized in that: Before obtaining the impaired speech features and predicted quality of a plurality of impaired speech signals through the speech quality determination model, the method further includes: Based on a first impairment type included in the impairment information, processing a first reference signal in the reference speech signal according to an impairment degree corresponding to the first impairment type to obtain a first impairment signal of the first reference signal under the first impairment type; Based on the first impairment signal and the first reference signal, a reference quality of the first impairment signal is determined, the reference quality indicating a speech quality of the first impairment signal relative to the first reference signal.

4. The pre-training method of the speech quality determination model according to claim 1, characterized in that: The method of obtaining the damaged speech features and the predicted quality of the multiple damaged speech signals by using the speech quality determination model comprises: Determining speech feature information of the damaged speech signal on the damage type through a plurality of two-dimensional convolutional layers of the speech quality determination model; Determining characteristic information of the impaired speech signal in a time dimension through a plurality of bidirectional recurrent network layers of the speech quality determination model; Based on the speech feature information and the feature information in the time dimension, outputting the damaged speech feature through a first fully connected layer of the speech quality determination model; Based on the impaired speech features, the predicted quality is output through the second fully connected layer of the speech quality determination model, where the predicted quality includes a perceptual evaluation of speech quality PESQ and / or short-time objective intelligibility STOI.

5. The pre-training method for the voice quality determination model according to claim 1, characterized in that: The method further comprises: Inputting a target speech signal into the speech quality determination model obtained by pre-training to obtain a predicted mean opinion score MOS of the target speech signal, wherein the target speech signal carries a mean opinion score MOS annotation; Determine a third loss value based on the predicted mean opinion score and the mean opinion score annotation carried by the target speech signal, wherein the third loss value is used to characterize an error between the predicted mean opinion score and the mean opinion score annotation; Based on the third loss value, the speech quality determination model obtained by pre-training is trained.

6. The pre-training method for a speech quality determination model according to any one of claims 1 to 5, characterized in that: In the case where the damage scenario is ambient sound damage, the damage degree is determined based on the content of the ambient sound and the signal-to-noise ratio; In the case where the damage scene is reverberation damage, the damage degree is determined based on the reverberation time; In the case where the damage scenario is device frequency response damage, the damage degree is determined based on the cutoff frequency of the filter; In the case where the impairment scenario is codec impairment, the impairment degree is determined based on the bit rate.

7. The pre-training method for the voice quality determination model according to claim 1, characterized in that: The method of obtaining the damaged speech features and the predicted quality of the multiple damaged speech signals by using the speech quality determination model comprises: Preprocessing the damaged speech signal to obtain conventional speech features of the damaged speech signal; The conventional speech features are input into the speech quality determination model to obtain the impaired speech features and the predicted quality of the impaired speech signal.

8. A method for determining speech quality, characterized in that: The method comprises: Acquire speech signals; The speech signal is input into a speech quality determination model to obtain the speech quality of the speech signal, wherein the speech quality determination model is trained based on a speech quality determination model pre-trained using a pre-training method for a speech quality determination model according to any one of claims 1 to 7.

9. The method for determining speech quality according to claim 8, characterized in that: The speech quality includes the mean opinion score MOS, and the perceptual evaluation of speech quality PESQ and / or the short-term objective intelligibility STOI.

10. A pre-training device for a speech quality determination model, characterized in that: The device comprises: an acquisition unit, configured to acquire impaired speech features and predicted quality of a plurality of impaired speech signals through a speech quality determination model, wherein the plurality of impaired speech signals are generated based on a reference speech signal and impairment information, wherein the reference speech signal is an unimpaired speech signal, and the impairment information indicates an impairment type for the reference speech signal, wherein the impairment type is used to indicate an impairment scenario and an impairment degree; A first determining unit is configured to determine a first loss value of the voice quality determination model based on the multiple impaired speech features, wherein the first loss value is used to characterize the distance between impaired speech features of the same impairment type and the distance between impaired speech features of different impairment types; The pre-training unit is configured to adjust the model parameters of the speech quality determination model based on the first loss value so that among the multiple damaged speech features obtained by the adjusted speech quality determination model, the distance between the damaged speech features of the same damage type is smaller, and the distance between the damaged speech features of different damage types is larger.

11. A speech quality determination device, characterized in that: The device comprises: A signal acquisition unit, configured to acquire a voice signal; The quality determination unit is configured to input the speech signal into a speech quality determination model to obtain the speech quality of the speech signal, wherein the speech quality determination model is trained based on a speech quality determination model pre-trained by a pre-training method for a speech quality determination model according to any one of claims 1 to 7.

12. An electronic device, characterized in that: The electronic device comprises: one or more processors; a memory for storing program code executable by the processor; The processor is configured to execute the program code to implement the pre-training method for determining the speech quality model as described in any one of claims 1 to 7 or the speech quality determination method as described in any one of claims 8 to 9.

13. A computer-readable storage medium, characterized in that: When the program code in the computer-readable storage medium is executed by a processor of a server, the server is enabled to execute the pre-training method of the speech quality determination model as described in any one of claims 1 to 7 or the speech quality determination method as described in any one of claims 8 to 9.

14. A computer program product, characterized in that The method comprises one or more instructions, wherein the one or more instructions are executed by one or more processors of a server, so that the server can execute the pre-training method of the speech quality determination model as described in any one of claims 1 to 7 or the speech quality determination method as described in any one of claims 8 to 9.

Images