Speech quality assessment method, and method and apparatus for training speech quality assessment model

Abstract

The embodiments of the present application relate to the technical field of artificial intelligence. Disclosed are a speech quality assessment method, and a method and apparatus for training a speech quality assessment model. The speech quality assessment method comprises: acquiring a speech signal to be assessed and a reference speech signal corresponding thereto; and using a speech quality assessment model to extract a latent feature representation of the reference speech signal and extract a latent feature representation of said speech signal, fusing the latent feature representation of the reference speech signal and the latent feature representation of said speech signal to obtain a fused feature representation, and using the fused feature representation to perform mapping to obtain a speech quality assessment result of said speech signal, wherein the speech quality assessment model is pre-trained using a neural network. The present application provides a neural-network-based speech quality assessment method, improving the efficiency and accuracy of speech quality assessment.

Description

Voice quality assessment method, method and device for training voice quality assessment model

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on March 18, 2024, with application number 202410311773.5, and entitled “Speech Quality Assessment Method, Method and Device for Training Speech Quality Assessment Model”, the entire contents of which are incorporated herein by reference. Technical Field

[0002] The present application relates to the field of artificial intelligence technology, and in particular to a method for speech quality assessment, and a method and device for training a speech quality assessment model. Background Art

[0003] In today's communications, multimedia processing, and intelligent voice interaction, voice quality assessment is a critical step in ensuring user experience and voice system performance. Excellent voice quality not only improves the clarity and efficiency of voice communications but also plays a vital role in technologies such as speech recognition and speech synthesis.

[0004] Speech quality assessment is an important measure of sound quality. However, existing speech quality evaluation models often rely on traditional signal processing methods. These methods are often limited to specific noise types and environments, making them difficult to adapt to diverse real-world scenarios. In complex speech processing scenarios, they face problems such as insufficient accuracy and poor adaptability, failing to fully reflect the human ear's true perception of speech quality. Furthermore, these models often require manual parameter adjustment, making the evaluation process cumbersome and inefficient. Summary of the Invention

[0005] The present application provides a speech quality assessment method, a method and an apparatus for training a speech quality assessment model, so as to improve the efficiency and accuracy of speech quality assessment.

[0006] This application provides the following solutions:

[0007] In a first aspect, a method for evaluating speech quality is provided, the method comprising:

[0008] Obtaining a speech signal to be evaluated and its corresponding reference speech signal;

[0009] Extracting the implicit feature representation of the reference speech signal and the implicit feature representation of the speech signal to be evaluated using a speech quality assessment model, fusing the implicit feature representation of the reference speech signal and the implicit feature representation of the speech signal to be evaluated to obtain a fused feature representation, and mapping the fused feature representation to obtain a speech quality assessment result of the speech signal to be evaluated;

[0010] The speech quality assessment model is pre-trained using a neural network.

[0011] According to an achievable manner in an embodiment of the present application, the method further includes: performing feature extraction on the reference speech signal and the speech signal to be evaluated, respectively, to obtain spectral features of the reference speech signal and spectral features of the speech signal to be evaluated;

[0012] The extracting of the implicit feature representation of the reference speech signal using the speech quality assessment model and the extracting of the implicit feature representation of the speech signal to be evaluated include: downsampling the spectral features of the reference speech signal using the speech quality assessment model to obtain the implicit feature representation of the reference speech signal, and downsampling the spectral features of the speech signal to be evaluated to obtain the implicit feature representation of the speech signal to be evaluated.

[0013] According to an implementable manner in an embodiment of the present application, the spectrum feature includes a Mel spectrum feature;

[0014] Performing feature extraction on the reference speech signal and the speech signal to be evaluated respectively to obtain the spectral features of the reference speech signal and the spectral features of the speech signal to be evaluated includes:

[0015] Converting the reference speech signal and the speech signal to be evaluated into Mel spectra respectively;

[0016] Dividing the Mel-spectrogram corresponding to the reference speech signal into one or more spectrum segments of a preset length, with adjacent spectrum segments partially overlapping, and using the one or more spectrum segments of the preset length as spectrum features of the reference speech signal; and

[0017] The Mel-spectrum spectrum corresponding to the speech signal to be evaluated is divided into one or more spectrum segments of a preset length, with adjacent spectrum segments partially overlapping, and the one or more spectrum segments of the preset length are used as the spectrum features of the speech signal to be evaluated.

[0018] According to an achievable method in an embodiment of the present application, fusing the implicit feature representation of the reference speech signal and the implicit feature representation of the speech signal to be evaluated to obtain a fused feature representation includes:

[0019] Performing a dot product on the latent feature representation of the reference speech signal and the latent feature representation of the speech signal to be evaluated;

[0020] The feature representation obtained by the dot product is concatenated with the latent feature representation of the speech signal to be evaluated to obtain the fused feature representation.

[0021] According to an achievable method in an embodiment of the present application, obtaining a speech quality assessment result of the speech signal to be assessed by using the fusion feature representation mapping includes:

[0022] Performing pooling processing on the fused feature representation, wherein the pooling processing includes pooling based on an attention mechanism;

[0023] The feature representation obtained by the pooling process is mapped to a speech quality score space to obtain a speech quality score of the speech signal to be evaluated, or the feature representation obtained by the pooling process is mapped to a speech quality rating space to obtain a speech quality rating of the speech signal to be evaluated.

[0024] In a second aspect, a testing method is provided, the method comprising:

[0025] Obtaining a reference speech signal;

[0026] Inputting the reference speech signal into the system to be tested, and obtaining the speech signal output by the system to be tested as the speech signal to be evaluated;

[0027] Obtaining a speech quality evaluation result of the speech signal to be evaluated using any of the methods described above;

[0028] The test result of the system to be tested is determined using the speech quality evaluation result.

[0029] In a third aspect, a method for training a speech quality assessment model is provided, the method comprising:

[0030] Acquiring training data comprising a plurality of training samples, wherein the training samples include impaired speech signal samples and corresponding reference speech signal samples and speech quality assessment labels;

[0031] The training data is used to train a neural network-based speech quality assessment model, wherein the speech quality assessment model extracts latent feature representations of the reference speech signal samples and the impaired speech signal samples, fuses the latent feature representations of the reference speech signal samples and the impaired speech signal samples to obtain a fused feature representation, and uses the fused feature representation to map and obtain a speech quality assessment result of the impaired speech signal sample; the training objectives include: minimizing the difference between the speech quality assessment result of the impaired speech signal sample and the corresponding speech quality assessment label.

[0032] According to an achievable manner in an embodiment of the present application, the method further includes: performing feature extraction on the reference speech signal sample and the impaired speech signal sample, respectively, to obtain spectral features of the reference speech signal sample and spectral features of the impaired speech signal sample, the spectral features including a Mel spectrum;

[0033] The speech quality assessment model extracts the implicit feature representation of the reference speech signal sample, and extracts the implicit feature representation of the damaged speech signal sample, including: the speech quality assessment model downsampling the spectral features of the reference speech signal sample to obtain the implicit feature representation of the reference speech signal sample, and downsampling the spectral features of the damaged speech signal sample to obtain the implicit feature representation of the damaged speech signal sample.

[0034] According to an achievable method in an embodiment of the present application, fusing the latent feature representation of the reference speech signal sample and the latent feature representation of the damaged speech signal sample to obtain a fused feature representation includes:

[0035] Performing a dot product on the latent feature representation of the reference speech signal sample and the latent feature representation of the impaired speech signal sample;

[0036] The feature representation obtained by the dot product is concatenated with the latent feature representation of the damaged speech signal sample to obtain the fused feature representation.

[0037] According to an achievable method in an embodiment of the present application, obtaining a speech quality assessment result of the impaired speech signal sample by using the fused feature representation mapping includes:

[0038] Performing pooling processing on the fused feature representation, wherein the pooling processing includes pooling based on an attention mechanism;

[0039] The feature representation obtained by the pooling process is mapped to a speech quality score space to obtain a speech quality score of the impaired speech signal sample, or the feature representation obtained by the pooling process is mapped to a speech quality rating space to obtain a speech quality rating of the impaired speech signal sample.

[0040] In a fourth aspect, a speech quality assessment device is provided, the device comprising:

[0041] a signal acquisition unit, configured to acquire a speech signal to be evaluated and a corresponding reference speech signal;

[0042] a quality assessment unit configured to extract an implicit feature representation of the reference speech signal and an implicit feature representation of the speech signal to be assessed using a speech quality assessment model, fuse the implicit feature representation of the reference speech signal and the implicit feature representation of the speech signal to be assessed to obtain a fused feature representation, and map the fused feature representation to obtain a speech quality assessment result of the speech signal to be assessed;

[0043] The speech quality assessment model is pre-trained using a neural network.

[0044] In a fifth aspect, a device for training a speech quality assessment model is provided, the device comprising:

[0045] a sample acquisition unit configured to acquire training data comprising a plurality of training samples, wherein the training samples include a damaged speech signal sample and its corresponding reference speech signal sample and a speech quality assessment label;

[0046] A model training unit is configured to train a neural network-based speech quality assessment model using the training data, wherein the speech quality assessment model extracts latent feature representations of the reference speech signal sample and the impaired speech signal sample, fuses the latent feature representations of the reference speech signal sample and the impaired speech signal sample to obtain a fused feature representation, and uses the fused feature representation to map to obtain a speech quality assessment result of the impaired speech signal sample; the training goal includes: minimizing the difference between the speech quality assessment result of the impaired speech signal sample and the corresponding speech quality assessment label.

[0047] According to a sixth aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the program is executed by a processor, the steps of the method described in any one of the first, second and third aspects are implemented.

[0048] According to a seventh aspect, there is provided an electronic device, comprising:

[0049] one or more processors; and

[0050] A memory associated with the one or more processors, the memory being used to store program instructions, wherein the program instructions, when read and executed by the one or more processors, execute the steps of the method described in any one of the first, second and third aspects above.

[0051] According to an eighth aspect, a computer program product is provided, which includes a computer program, and when the computer program is executed by a processor, the steps of the method described in any one of the first, second and third aspects are implemented.

[0052] According to the specific embodiments provided in this application, this application discloses the following technical effects:

[0053] 1) The embodiments of the present application utilize a speech quality assessment model trained using a neural network to perform speech quality assessment. This model extracts latent feature representations of the speech signal to be assessed and the reference speech signal, fuses the latent feature representations, and obtains a quality assessment result based on the fused features. This method extracts the key features of the two speech signals at a lower dimension, reducing the complexity of speech quality assessment. Furthermore, by fusing the key features of the two speech signals, it increases the correlation between the features, thereby improving the efficiency and accuracy of speech quality assessment.

[0054] 2) This application extracts spectral features from speech signals. Spectral features improve the robustness of speech signal features compared to time-domain features. Downsampling these spectral features reduces the dimensionality of the data while preserving important features, lowering computational complexity and improving the efficiency of speech evaluation.

[0055] 3) This application uses Mel spectrum features as spectral features. Mel spectrum features are a frequency domain representation that is more consistent with the auditory characteristics of the human ear. Therefore, it can make the evaluation of speech quality more consistent with the real perception of the human ear.

[0056] 4) This application divides the Mel spectrum into spectrum segments of a preset length, which are more convenient for spectrum feature extraction. At the same time, there is partial overlap between adjacent spectrum segments, which avoids the loss of spectrum information caused by the spectrum segmentation operation.

[0057] 5) This application combines the feature representation obtained by performing a dot product of the latent feature representation of the reference speech signal and the latent feature representation of the speech signal to be evaluated with the latent feature representation of the speech signal to be evaluated to obtain the fused feature representation. This fusion method is simple and efficient, and can fully reflect the differences between the reference speech signal and the speech signal to be evaluated, thereby making the subsequent speech quality assessment results based on the fused feature representation more accurate.

[0058] 6) This application performs pooling processing on the fused feature representation, further reducing the dimension of the fused feature representation and extracting more representative information, thereby reducing the complexity and accuracy of generating the speech quality score.

[0059] 7) This application can perform pooling processing based on the attention mechanism, which can reduce the dimension of the fusion feature representation on the one hand, and fully consider the impact of the correlation between the features of the reference speech signal and the speech signal to be evaluated on the speech quality evaluation on the other hand, thereby improving the efficiency and accuracy of speech evaluation.

[0060] Of course, any product implementing the present application does not necessarily need to achieve all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0062] FIG1 is a diagram of a system architecture applicable to an embodiment of the present application;

[0063] FIG2 is a flow chart of a method for evaluating speech quality according to an embodiment of the present invention;

[0064] FIG3a is a schematic diagram showing the principle of a speech quality assessment model provided in an embodiment of the present application;

[0065] FIG3 b is a schematic diagram showing the principle of a training speech quality assessment model provided in an embodiment of the present application;

[0066] FIG4 is a schematic diagram of a framework for testing a system to be tested according to an embodiment of the present application;

[0067] FIG5 is a schematic diagram of a speech quality assessment device provided in an embodiment of the present application;

[0068] FIG6 is a schematic diagram of an apparatus for training a speech quality assessment model according to an embodiment of the present application;

[0069] FIG7 is a schematic block diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0070] The following will be combined with the accompanying drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field are within the scope of protection of this application.

[0071] The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments and are not intended to limit the embodiments of the present application. The singular forms "a", "an", "the" and "the" used in the embodiments of the present application and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise.

[0072] It should be understood that the term "and / or" as used herein is merely a description of the relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A exists alone, A and B exist simultaneously, or B exists alone. Furthermore, the character " / " in this document generally indicates that the associated objects are in an "or" relationship.

[0073] The word "if," as used herein, may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to the determination" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)," depending on the context.

[0074] Current speech quality assessments mostly rely on subjective listening, which is labor-intensive. Existing speech quality evaluation models rely on traditional signal processing methods, which are limited to specific noise types and environments and are difficult to adapt to diverse real-world scenarios.

[0075] In view of this, the present application provides a new approach. To facilitate understanding of the present application, the system architecture on which the present application is based is first described. Figure 1 shows an exemplary system architecture to which embodiments of the present application can be applied. As shown in Figure 1, the system architecture may include user equipment and a voice quality assessment device located on the server side, a voice quality assessment model, and a model training device.

[0076] The evaluator can send the speech signal to be evaluated and the reference speech signal to the speech quality evaluation device on the server side through the user equipment.

[0077] User devices may include, but are not limited to, smart mobile terminals, smart home devices, wearable devices, and personal computers (PCs). Smart mobile devices may include mobile phones, tablets, laptops, PDAs (Personal Digital Assistants), and internet-connected cars. Smart home devices may include smart TVs and smart refrigerators. Wearable devices may include smart watches, smart glasses, virtual reality devices, augmented reality devices, and mixed reality devices (i.e., devices that support both virtual reality and augmented reality).

[0078] The speech quality assessment device can use the method provided in the embodiments of the present application to generate a speech quality assessment result. The assessment process of the speech quality assessment device involves the use of a speech quality assessment model.

[0079] The model training device can use the method provided in the embodiments of the present application to pre-train a speech quality assessment model.

[0080] The speech quality assessment device and model training device can be deployed on a standalone server, within a server cluster, or even on a cloud server. A cloud server, also known as a cloud computing server or cloud host, is a host product within the cloud computing service ecosystem. It addresses the management difficulties and scalability limitations of traditional physical hosts and virtual private servers (VPS). In addition to the architecture shown in Figure 1, the speech quality assessment device and model training device can also be deployed on a computer terminal with significant computing power.

[0081] In one possible implementation, an evaluator can input a speech signal to be evaluated and a reference speech signal through a user device. The user device then transmits the speech signal to be evaluated and the reference speech signal via a network to a server-side speech quality assessment device. The speech quality assessment device generates a speech quality assessment result for the speech signal and returns the result to the user device via the network.

[0082] It should be understood that the user equipment, speech quality assessment device, speech quality assessment model and model training device in Figure 1 are merely illustrative. Any number of user equipment, speech quality assessment devices, speech quality assessment models and model training devices may be provided as required.

[0083] In addition to the system architecture shown in FIG1 , the voice quality assessment device may also be an application located in the user equipment, or may also be a functional unit such as a plug-in or a software development kit (SDK) provided in the application.

[0084] FIG2 is a flow chart of a method for evaluating speech quality according to an embodiment of the present application, which may be performed by the speech quality evaluation device in the system shown in FIG1. ​​As shown in FIG2, the method may include the following steps:

[0085] Step 201: Acquire a speech signal to be evaluated and its corresponding reference speech signal.

[0086] Step 202: Using a speech quality assessment model, extracting latent feature representations of a reference speech signal and a speech signal to be assessed, fusing the latent feature representations of the reference speech signal and the latent feature representations of the speech signal to be assessed to obtain a fused feature representation, and mapping the fused feature representation to obtain a speech quality assessment result of the speech signal to be assessed; wherein the speech quality assessment model is pre-trained using a neural network.

[0087] As can be seen from the above process, the embodiment of the present application uses a speech quality assessment model obtained by neural network training to perform speech quality assessment. This model extracts latent feature representations of the speech signal to be evaluated and the reference speech signal, fuses the latent feature representations, and obtains a quality assessment result based on the fused features. By extracting the latent feature representations of the two speech signals, this method can obtain the key features of the speech signal at a lower dimension and fuse the key features of the two speech signals, thereby increasing the correlation between the features and improving the efficiency and accuracy of speech assessment.

[0088] The following describes in detail the steps in the above process and the effects that can be further produced in conjunction with the embodiments.

[0089] First, the above step 201, namely “obtaining the speech signal to be evaluated and its corresponding reference speech signal”, is described in detail with reference to the embodiment.

[0090] This application obtains the result of speech quality evaluation by comparing the difference between the speech signal to be evaluated and the reference speech signal. The speech signal to be evaluated and the reference speech signal correspond to each other, and the reference speech signal is the original unprocessed speech signal. The speech signal to be evaluated is the signal for which speech quality evaluation is required. The signal can be a speech signal obtained after the reference speech signal is processed by a communication system, a speech recognition system or other speech processing system. In this scenario, the evaluation of the speech signal to be evaluated can actually be regarded as a test or evaluation of the communication system, the speech recognition system or other speech processing system. Among them, the speech signal to be evaluated and the reference speech signal are usually time domain speech signals.

[0091] In the embodiments of the present application, a variety of methods can be used to obtain the reference speech signal and the speech signal to be evaluated, including but not limited to the following methods:

[0092] Method 1: The reference speech signal and the speech signal to be evaluated can be obtained through real-time recording. The evaluator can use a tape recorder or other recording equipment to record the reference speech signal and the speech signal to be evaluated for subsequent quality evaluation comparison.

[0093] For example, a person speaks in real time, and the evaluator records this speech using a recorder as a reference speech signal. This reference speech signal then passes through a hearing aid, and the evaluator then collects and records the speech produced after passing through the hearing aid as the speech signal to be evaluated. This method is often used when testing hearing aids and other devices.

[0094] Method 2: The reference speech signal and the speech signal to be evaluated can also be pre-stored audio files. For example, the evaluator can use speech samples from a public speech dataset as the reference speech signal, while simultaneously obtaining the speech signal to be evaluated from the same or a different dataset for quality comparison. This method is often used to test speech quality assessment models.

[0095] Method 3: The evaluator can also use a recorded audio file as a reference speech signal and simultaneously obtain the audio file of the speech signal to be evaluated in real time for quality evaluation comparison.

[0096] For example, a pre-recorded audio file is played, and the speech signal in the audio file serves as the reference speech signal. After the reference speech signal passes through a hearing aid, the evaluator collects and records the speech generated after passing through the hearing aid as the speech signal to be evaluated. This method is often used in testing hearing aids and other devices.

[0097] The following describes in detail the step 202, "using a speech quality assessment model to extract latent feature representations of a reference speech signal and a speech signal to be assessed, fusing the latent feature representations of the reference speech signal and the latent feature representations of the speech signal to be assessed to obtain a fused feature representation, and mapping the fused feature representation to obtain a speech quality assessment result of the speech signal to be assessed; wherein the speech quality assessment model is pre-trained using a neural network," in conjunction with an embodiment.

[0098] This application uses a speech quality assessment model to perform speech quality assessment. Figure 3a is a schematic diagram of the principle of the speech quality assessment model provided in an embodiment of this application. As shown in Figure 3a, the speech quality assessment model includes a latent feature extraction module set for the reference speech signal and the speech signal to be evaluated, respectively. The latent feature extraction module outputs the latent feature representation of the reference speech signal and the speech signal to be evaluated.

[0099] As one feasible approach, the latent feature extraction module can directly extract time-domain features from the reference speech signal and the speech signal to be evaluated, respectively, as latent feature representations. However, this approach needs to be improved in terms of the robustness of speech signal evaluation.

[0100] Therefore, an embodiment of the present application provides another more preferred method, as shown in FIG3a. First, feature extraction is performed on the reference speech signal and the speech signal to be evaluated to obtain spectral features of the reference speech signal and the speech signal to be evaluated. The spectral features can be obtained by Fourier transform, fast Fourier transform, etc. Then, the spectral features of the reference speech signal are downsampled using the latent feature extraction module in the speech quality assessment model to obtain a latent feature representation of the reference speech signal. The spectral features of the speech signal to be evaluated are also downsampled to obtain a latent feature representation of the speech signal to be evaluated.

[0101] As an implementable method, the present application obtains spectral features by converting a reference speech signal and a speech signal to be evaluated into a Mel-spectrogram. Mel-spectrogram is a frequency domain representation that is more in line with the auditory characteristics of the human ear. The sound is mapped to the Mel scale through a set of Mel filter banks. The filters are densely distributed in the low frequency range and sparsely distributed in the high frequency range. Therefore, the Mel-spectrogram is nonlinear. In this way, the perceptual differences of the human ear for two pairs of frequencies with equal distances on the Mel scale are also the same, that is, the human ear perception and the Mel scale are linearly related. Therefore, based on the Mel-spectrogram features, it is possible to simulate the way the human auditory system perceives the sound spectrum. Specifically, the input reference speech signal and the speech signal to be evaluated are first converted into Mel-spectrograms respectively; the Mel-spectrogram corresponding to the reference speech signal is divided into one or more spectral segments of a preset length, with adjacent spectral segments partially overlapping, and the spectral segments of the preset length or more are used as the spectral features of the reference speech signal; and the Mel-spectrogram corresponding to the speech signal to be evaluated is divided into one or more spectral segments of a preset length, with adjacent spectral segments partially overlapping, and the spectral segments of the preset length or more are used as the spectral features of the speech signal to be evaluated.

[0102] The spectral features of both the reference speech signal and the speech signal to be evaluated can be expressed as [B, L, T, F], where B represents the batch size, which refers to the number of reference speech signals and speech signals to be evaluated input at a time. During training, B is typically a positive integer greater than 1, and its specific value can be pre-set based on empirical data. During actual inference (i.e., speech quality assessment), B is 1. L is the number of spectral segments, which can be determined based on the time width of the speech signal and the value of T. T is the time width of the spectral segment, which can be pre-set based on empirical data and is typically set to 100 to 300 milliseconds. F is the height of the spectral segment, which is determined by the energy-weighted sum of the signal's power spectrum (or energy spectrum) in the Mel filter bank.

[0103] The above-mentioned spectral features are downsampled, that is, further information is extracted from the spectral features to obtain high-dimensional hidden layer features. Downsampling is used to reduce the dimension of the spectral features while retaining important features. Downsampling can be performed using methods such as average pooling, maximum pooling, and downsampling filtering. As an implementable method, the latent feature extraction module of the present application can use a convolutional neural network for downsampling.

[0104] The latent feature extraction module can further include three modules: convolution, pooling, and normalization. The convolution module is used to identify important features in the spectral characteristics of the reference speech signal and the speech signal to be evaluated; the pooling module is used to reduce the dimensionality of the spectral features, reduce the amount of computation, and extract key information; and the normalization module can help the neural network better learn the distribution characteristics of the data, thereby improving the performance of the model. After the spectral features [B, L, T, F] of the reference speech signal and the speech signal to be evaluated are input into the latent feature extraction module, the spectral features are extracted through the convolution layer, the pooling layer reduces the dimensionality of the spectral features, and the normalization layer adjusts the distribution of activation values. The output latent features of the two speech signals are represented in the size of [B, L, H].

[0105] After obtaining the latent feature representations of the reference and evaluated speech signals, the feature fusion module in the speech quality assessment model can be used to fuse these two latent feature representations to obtain a fused feature representation. When fusing the latent features of the reference and evaluated speech signals, the two latent features can be simply concatenated, or a relationship can be established by performing operations on the two features.

[0106] As a feasible method, a dot product can be performed on the latent feature representation of the reference speech signal and the latent feature representation of the speech signal to be evaluated; the feature representation obtained by the dot product is concatenated with the latent feature representation of the speech signal to be evaluated to obtain a fused feature representation.

[0107] Specifically, based on the implicit feature representation [B, L, H] of the speech signal to be evaluated, a new Spliced ​​to H dimensions. Where D represents the latent feature representation of the speech signal to be evaluated, with a size of [L, H]; R represents the latent feature representation of the reference speech signal, also with a size of [L, H]. Represents the dot product of each number in the two matrices and normalized to the range of [-1, 1]. Finally, the size of the fused feature representation is [B, L, Hx2].

[0108] As an alternative solution, the dot product operation of the latent feature representation of the reference speech signal and the latent feature representation of the speech signal to be evaluated mentioned above can also be replaced by operations such as addition, subtraction, and normalization of the latent feature representation of the reference speech signal and the latent feature representation of the speech signal to be evaluated, which can also achieve the technical effect of fusing the latent features.

[0109] Finally, the speech quality assessment result of the speech signal to be evaluated is obtained by using the fusion feature representation mapping.

[0110] The speech quality assessment model is pre-trained using a neural network. This pre-training establishes a mapping relationship between the fused feature representation and the speech quality assessment results. This mapping can be linear or nonlinear. During the model inference phase, the speech quality assessment model uses the fused feature representation to obtain the speech quality assessment results for the speech signal being evaluated.

[0111] As a more preferred embodiment, the present application can further perform pooling processing on the fused features, and map the feature representations obtained by the pooling processing to obtain the speech quality assessment results of the speech signal to be evaluated. As shown in Figure 3a, the speech quality assessment model also includes a pooling processing module. The pooling processing module can be implemented using conventional maximum pooling, average pooling, adaptive pooling, etc. As a more preferred method, an attention-based pooling processing method can be used to pool the fused features. The attention-based pooling processing method combines the attention mechanism and pooling operation, aiming to allow the model to dynamically focus on different parts of the input features during the feature pooling process, thereby better capturing the key information of the input data. The attention-based pooling processing method generally includes the following steps: First, through a learned attention mechanism, the attention weight of each input fused feature is calculated. These weights represent the degree to which the model should pay attention to each fused feature when performing feature pooling. Based on the calculated attention weights, the fused features are weighted pooled. Specifically, each fused feature vector is weighted by its corresponding attention weight, and then the weighted feature vectors are added or weighted averaged to obtain the final pooled representation.

[0112] Among them, the feature representation obtained by pooling processing can be mapped to the speech quality score space to obtain the speech quality score of the speech signal to be evaluated, or the feature representation obtained by pooling processing can be mapped to the speech quality rating space to obtain the speech quality rating of the speech signal to be evaluated.

[0113] The speech quality rating space typically refers to a continuous, numerical space used to represent the quality levels of different speech signals. Within this space, speech signal quality is typically represented by a continuous score, such as a scale from 1 to 5, where scores from 1 to 5 correspond to successively better quality. This rating space is often used for subjective quality assessment, where human listeners assess speech quality to determine its level. Within this rating space, speech quality assessment results can be expressed using a mean opinion score (MOS). MOS represents a subjective listening experience, with 5 being the highest score and 1 being the lowest.

[0114] The speech quality rating space is a discrete, categorized space used to classify speech signals into different quality levels or categories. Within this space, speech signals are typically categorized into several predefined levels, each representing a specific quality level, such as "excellent," "good," "fair," or "poor." This rating space is typically used for objective quality assessment, where automated algorithms or models are used to rate speech signals.

[0115] The present application also provides a method for training a speech quality assessment model. The speech quality assessment model trained by this method can be used in the speech quality assessment method of the present application.

[0116] Figure 3b is a schematic diagram of the principle of training a speech quality assessment model provided in an embodiment of the present application. As shown in Figure 3b, the method for training a speech quality assessment model includes: obtaining training data including multiple training samples, where the training samples include damaged speech signal samples and their corresponding reference speech signal samples and speech quality assessment labels.

[0117] Training samples can be obtained from existing speech datasets or through subjective evaluation experiments, where humans listen to speech and assign quality scores to evaluate the quality of various speech samples, thereby obtaining labeled training data. Speech quality assessment labels are parameters used to represent speech quality, and can be expressed using the MOS score mentioned above or speech quality ratings such as "Excellent," "Good," "Fair," and "Poor."

[0118] The training data is used to train a neural network-based speech quality assessment model, wherein the speech quality assessment model extracts latent feature representations of reference speech signal samples and latent feature representations of damaged speech signal samples, fuses the latent feature representations of the reference speech signal samples and the latent feature representations of the damaged speech signal samples to obtain a fused feature representation, and uses the fused feature representation mapping to obtain a speech quality assessment result of the damaged speech signal sample.

[0119] There are many methods for extracting latent features from training samples. As shown in Figure 3b, the speech quality assessment model includes latent feature extraction modules set up for reference speech signal samples and damaged speech signal samples respectively. The latent feature extraction modules output latent feature representations of the reference speech signal samples and the damaged speech signal samples.

[0120] As one feasible approach, the latent feature extraction module can directly extract time-domain features from the reference and impaired speech signal samples as latent feature representations. However, this approach needs to be improved in terms of the robustness of speech signal evaluation.

[0121] Therefore, an embodiment of the present application provides another more preferred method, as shown in FIG3b . First, feature extraction is performed on the reference speech signal sample and the impaired speech signal sample to obtain spectral features of the reference speech signal sample and the impaired speech signal sample. Spectral features can be obtained using methods such as Fourier transform and fast Fourier transform. Then, the latent feature extraction module in the speech quality assessment model is used to downsample the spectral features of the reference speech signal sample to obtain a latent feature representation of the reference speech signal sample, and the spectral features of the impaired speech signal sample are also downsampled to obtain a latent feature representation of the impaired speech signal sample.

[0122] As an implementable method, the present application obtains spectral features by converting reference speech signal samples and impaired speech signal samples into mel-spectrograms. Specifically, the input reference speech signal samples and impaired speech signal samples are first converted into mel-spectrograms respectively; the mel-spectrogram corresponding to the reference speech signal sample is divided into one or more spectral segments of a preset length, with adjacent spectral segments partially overlapping, and the one or more spectral segments of the preset length are used as the spectral features of the reference speech signal sample; and the mel-spectrogram corresponding to the impaired speech signal sample is divided into one or more spectral segments of a preset length, with adjacent spectral segments partially overlapping, and the one or more spectral segments of the preset length are used as the spectral features of the impaired speech signal sample.

[0123] Among them, the spectral characteristics of the reference speech signal samples and the damaged speech signal samples can be expressed as the size of [B, L, T, F]. B represents the batch size, which refers to the number of reference speech signal samples and damaged speech signal samples input in a single time. During the training process, B is usually a positive integer greater than 1, and the specific value of B can be pre-set based on empirical values; L is the number of spectrum segments, which can be determined based on the time width and T value of the sample speech signal; T is the time width of the spectrum segment, which can be pre-set based on empirical values ​​or modified in real time during the model training process, and can usually be set to 100 to 300 milliseconds; F is the height of the spectrum segment, and its value is determined based on the energy-weighted sum of the power spectrum (or energy spectrum) of the sample signal in the Mel filter bank.

[0124] The above-mentioned spectral features are downsampled and further information is extracted from the spectral features to obtain high-dimensional hidden layer features. Downsampling can be performed using methods such as average pooling, maximum pooling, downsampling filtering, etc. As an implementable method, the latent feature extraction module of the present application can use a convolutional neural network for downsampling.

[0125] The latent feature extraction module can further include three modules: convolution, pooling, and normalization. After the spectral features [B, L, T, F] of the reference and impaired speech signal samples are input into the latent feature extraction module, the convolution layer extracts the spectral features, the pooling layer reduces the dimensionality of the spectral features, and the normalization layer adjusts the distribution of activation values. The output latent features are represented as [B, L, H] in size.

[0126] After obtaining the latent feature representations of the reference and impaired speech signal samples, the feature fusion module in the speech quality assessment model can be used to fuse the latent feature representations of the two speech signal samples to obtain a fused feature representation. Fusion of the latent features of the reference and impaired speech signal samples can be achieved by simply concatenating the latent features of the two speech signal samples, or by performing certain operations on the two features to establish a relationship between them.

[0127] As a feasible method, we can perform a dot product on the latent feature representation of the reference speech signal sample and the latent feature representation of the impaired speech signal sample; then concatenate the feature representation obtained by the dot product with the latent feature representation of the impaired speech signal sample to obtain a fused feature representation. Specifically, based on the latent feature representation [B, L, H] of the impaired speech signal sample, we add Spliced ​​to H dimensions. Where D represents the latent feature representation in the damaged speech signal sample, with a size of [L, H]; R represents the latent feature representation in the reference speech signal sample, with a size of [L, H]. Represents the dot product of each number in the two matrices and normalized to the range of [-1, 1]. Finally, the size of the fused feature representation is [B, L, Hx2].

[0128] As an alternative solution, the dot multiplication operation of the latent feature representation of the reference speech signal sample and the latent feature representation of the imperfect speech signal sample mentioned above can also be replaced by operations such as addition, subtraction, and normalization of the latent feature representation of the reference speech signal sample and the latent feature representation of the imperfect speech signal, which can also achieve the technical effect of fusing the latent features.

[0129] Finally, the fused feature representation is used to map the speech quality of the speech signal to be evaluated. The model is trained to establish a mapping relationship between the fused feature representation and the speech quality evaluation result. This mapping can be linear or nonlinear.

[0130] As a more preferred embodiment, the present application can further perform pooling processing on the fused feature representation and map the feature representation obtained by the pooling processing. As shown in Figure 3b, the speech quality assessment model also includes a pooling processing module. The pooling processing module can be implemented using conventional maximum pooling, average pooling, adaptive pooling, etc. As a more preferred embodiment, an attention-based pooling method can be used to pool the fused features.

[0131] Among them, the feature representation obtained by pooling processing can be mapped to the speech quality score space to obtain the speech quality score of the damaged speech signal sample, and the feature representation obtained by pooling processing can also be mapped to the speech quality rating space to obtain the speech quality rating of the damaged speech signal sample.

[0132] In the process of training the speech quality assessment model, the training objective may include minimizing the difference between the speech quality assessment result of the damaged speech signal sample and the corresponding speech quality assessment label.

[0133] A loss function can be constructed based on the above training objectives. In each iteration, the value of the loss function is used to update the model parameters (i.e., the parameters of the latent feature extraction module, the feature fusion module, and the pooling processing module) using methods such as gradient descent until the preset training end conditions are met. The training end conditions may include, for example, the value of the loss function is less than or equal to a preset loss function threshold, the number of iterations reaches a preset number threshold, etc.

[0134] More specifically, during the model training phase, an attention mechanism enables the neural network to focus on learning the differences and sound quality relationships between the reference and imperfect speech signal samples, ultimately outputting a MOS score. A loss is calculated from the difference between the model-predicted MOS and the label, and backpropagated to update the model parameters to minimize the loss function. Model parameters can be updated using the Adam optimizer, which utilizes the Adam (Adaptive Moment Estimation) optimization algorithm. The Adam optimization algorithm combines momentum and an adaptive learning rate and is commonly used for parameter optimization in deep learning models. The Adam algorithm utilizes momentum to accelerate the optimization process, helping the optimization algorithm gain a certain degree of "inertia" in the current gradient direction during parameter updates, thereby reducing parameter update oscillations. The Adam algorithm also introduces the concept of adaptive learning rates, which uses a different learning rate for each parameter. It adaptively adjusts the learning rate for each parameter by calculating the first-order moment estimate (mean) and second-order moment estimate (variance) of the gradient. This allows the learning rate to be automatically adjusted for different parameters based on their historical performance of the gradient, thereby updating the parameters more effectively.

[0135] The speech quality assessment method provided in this application can be used to test a system under test, which is a device or application that has input and output of speech signals. For example, the system under test can be used to test a communication link that needs to transmit speech signals, audio processing software, or a smart speaker that can interact with the user through speech.

[0136] FIG4 is a schematic diagram of a framework for testing a system to be tested provided by an embodiment of the present application. As shown in FIG4 , during the test, a reference voice signal is first obtained. The nature and source of the reference voice signal are the same as those described above. The reference voice signal is input into the system to be tested, and the voice signal output by the system to be tested is obtained as the voice signal to be evaluated. The voice signal to be evaluated is a voice signal obtained by processing the reference voice signal by the system to be tested. The related processing may include operations such as transmission or encoding and decoding. The difference between the voice signal to be evaluated and the reference voice signal can reflect the interference of the system to be tested on the voice signal.

[0137] A reference speech signal and a speech signal to be evaluated are input into a speech quality evaluation device. The speech quality evaluation method described in the above embodiment is used to obtain a speech quality evaluation result for the speech signal to be evaluated. The speech quality evaluation result is then used to determine the test result of the system to be tested. The speech quality evaluation result, such as a MOS value, can be used directly as the evaluation result for the system to be tested. Alternatively, the speech quality evaluation result can be mapped to obtain the evaluation result for the system to be tested. For example, if the speech quality evaluation result has a MOS value greater than or equal to 3, the test result of the system to be tested is determined to be acceptable.

[0138] The above method provided in the embodiment of the present application can be applied to a variety of application scenarios. In the field of communications, it can be applied to fields such as telephone communications, video calls, and Internet calls (VoIP). Voice quality assessment helps evaluate call quality and detect possible problems (such as noise, distortion, etc.) in voice signals, thereby improving communication quality. In the field of audio processing, the voice quality assessment method can be used to evaluate the effect of audio signal processing, which helps to optimize and improve audio processing algorithms. The voice quality assessment method can also be applied to speech recognition systems to evaluate the quality and clarity of voice signals, helping to improve the accuracy and stability of speech recognition.

[0139] The foregoing description of this specification describes specific embodiments. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that described in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or the sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0140] According to another embodiment, a speech quality assessment device is provided. FIG5 shows a schematic block diagram of the speech quality assessment device according to one embodiment, which is provided on the server side of the architecture shown in FIG1 . As shown in FIG5 , the device 500 includes: a signal acquisition unit 501 and a quality assessment unit 502. The main functions of each component unit are as follows:

[0141] The signal acquisition unit 501 is configured to acquire a speech signal to be evaluated and its corresponding reference speech signal.

[0142] The quality assessment unit 502 is configured to extract the latent feature representation of the reference speech signal and the latent feature representation of the speech signal to be assessed using the speech quality assessment model, fuse the latent feature representation of the reference speech signal and the latent feature representation of the speech signal to be assessed to obtain a fused feature representation, and use the fused feature representation mapping to obtain a speech quality assessment result of the speech signal to be assessed; wherein the speech quality assessment model is pre-trained using a neural network.

[0143] As one of the possible implementation methods, the quality assessment unit 502 can be configured to: perform feature extraction on the reference speech signal and the speech signal to be evaluated respectively to obtain the spectral features of the reference speech signal and the spectral features of the speech signal to be evaluated; use the speech quality assessment model to extract the implicit feature representation of the reference speech signal, and extracting the implicit feature representation of the speech signal to be evaluated includes: using the speech quality assessment model to downsample the spectral features of the reference speech signal to obtain the implicit feature representation of the reference speech signal, and downsample the spectral features of the speech signal to be evaluated to obtain the implicit feature representation of the speech signal to be evaluated.

[0144] As one of the possible implementation methods, the spectrum feature includes a Mel spectrum feature.

[0145] As one of the implementable manners, the quality assessment unit 502 may be specifically configured as follows when performing feature extraction on the reference speech signal and the speech signal to be assessed, respectively, to obtain the spectral features of the reference speech signal and the spectral features of the speech signal to be assessed:

[0146] The reference speech signal and the speech signal to be evaluated are respectively converted into Mel-spectra; the Mel-spectra corresponding to the reference speech signal is divided into one or more spectrum segments of a preset length, with adjacent spectrum segments partially overlapping, and the one or more spectrum segments of the preset length are used as the spectrum features of the reference speech signal; and the Mel-spectra corresponding to the speech signal to be evaluated is divided into one or more spectrum segments of a preset length, with adjacent spectrum segments partially overlapping, and the one or more spectrum segments of the preset length are used as the spectrum features of the speech signal to be evaluated.

[0147] As one possible implementation, when the quality assessment unit 502 fuses the implicit feature representation of the reference speech signal and the implicit feature representation of the speech signal to be assessed to obtain the fused feature representation, it can be specifically configured as follows:

[0148] Perform dot product on the latent feature representation of the reference speech signal and the latent feature representation of the speech signal to be evaluated; concatenate the feature representation obtained by the dot product with the latent feature representation of the speech signal to be evaluated to obtain a fused feature representation.

[0149] As one of the implementable manners, when the quality assessment unit 502 obtains the speech quality assessment result of the speech signal to be assessed by using the fusion feature representation mapping, it can be specifically configured as follows:

[0150] Pooling is performed on the fused feature representation; the feature representation obtained by the pooling process is mapped to a speech quality score space to obtain a speech quality score of the speech signal to be evaluated, or the feature representation obtained by the pooling process is mapped to a speech quality rating space to obtain a speech quality rating of the speech signal to be evaluated.

[0151] As one possible implementation method, the quality assessment unit 502 may be configured to adopt pooling based on an attention mechanism.

[0152] According to another embodiment, a device for training a speech quality assessment model is provided. FIG6 shows a schematic block diagram of the device for training a speech quality assessment model according to one embodiment, which is provided on the server side of the architecture shown in FIG1 . As shown in FIG6 , the device 600 includes: a sample acquisition unit 601 and a model training unit 602. The main functions of each component unit are as follows:

[0153] The sample acquisition unit 601 is configured to acquire training data including a plurality of training samples, where the training samples include damaged speech signal samples and their corresponding reference speech signal samples and speech quality assessment labels.

[0154] The model training unit 602 is configured to train a neural network-based speech quality assessment model using training data, wherein the speech quality assessment model extracts latent feature representations of reference speech signal samples and latent feature representations of damaged speech signal samples, fuses the latent feature representations of the reference speech signal samples and the latent feature representations of the damaged speech signal samples to obtain a fused feature representation, and uses the fused feature representation to map to obtain a speech quality assessment result of the damaged speech signal sample; the training objectives include: minimizing the difference between the speech quality assessment result of the damaged speech signal sample and the corresponding speech quality assessment label.

[0155] As one of the possible implementation methods, the model training unit 602 can be configured to: perform feature extraction on the reference speech signal sample and the impaired speech signal sample respectively to obtain the spectral features of the reference speech signal sample and the spectral features of the impaired speech signal sample; the speech quality assessment model extracts the implicit feature representation of the reference speech signal sample, and the extraction of the implicit feature representation of the impaired speech signal sample includes: the speech quality assessment model downsamples the spectral features of the reference speech signal sample to obtain the implicit feature representation of the reference speech signal sample, and downsamples the spectral features of the impaired speech signal sample to obtain the implicit feature representation of the impaired speech signal sample.

[0156] As one of the possible implementation methods, the spectrum feature includes a Mel spectrum feature.

[0157] As one possible implementation, when the model training unit 602 performs feature extraction on the reference speech signal sample and the impaired speech signal sample to obtain the spectral features of the reference speech signal sample and the spectral features of the impaired speech signal sample, the model training unit 602 may be specifically configured as follows:

[0158] The reference speech signal sample and the impaired speech signal sample are respectively converted into Mel-spectra; the Mel-spectra corresponding to the reference speech signal sample is divided into one or more spectrum segments of a preset length, with adjacent spectrum segments partially overlapping, and the one or more spectrum segments of the preset length are used as the spectrum features of the reference speech signal sample; and the Mel-spectra corresponding to the impaired speech signal sample is divided into one or more spectrum segments of a preset length, with adjacent spectrum segments partially overlapping, and the one or more spectrum segments of the preset length are used as the spectrum features of the impaired speech signal sample.

[0159] As one possible implementation, when the model training unit 602 fuses the latent feature representation of the reference speech signal sample and the latent feature representation of the impaired speech signal sample to obtain the fused feature representation, it can be specifically configured as follows:

[0160] Perform dot product on the latent feature representation of the reference speech signal sample and the latent feature representation of the impaired speech signal sample; concatenate the feature representation obtained by the dot product with the latent feature representation of the impaired speech signal sample to obtain a fused feature representation.

[0161] As one possible implementation, when the model training unit 602 obtains the speech quality assessment result of the impaired speech signal sample by using the fused feature representation mapping, it can be specifically configured as follows:

[0162] Pooling is performed on the fused feature representation; the feature representation obtained by the pooling is mapped to a speech quality score space to obtain a speech quality score of the damaged speech signal sample, or the feature representation obtained by the pooling is mapped to a speech quality rating space to obtain a speech quality rating of the damaged speech signal sample.

[0163] As one of the possible implementation methods, the model training unit 602 can be configured to adopt pooling based on the attention mechanism.

[0164] As one of the possible implementations, the model training unit 602 may be configured to: during the training process, based on the loss function value corresponding to the training objective, use the Adam optimizer optimization algorithm to update the model parameters of the speech quality assessment model.

[0165] Each embodiment in this specification is described in a progressive manner. The same or similar parts between the embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments. In particular, for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple. For the relevant parts, refer to the partial description of the method embodiment. The device embodiment described above is merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. A person of ordinary skill in the art can understand and implement it without expending creative work.

[0166] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0167] In addition, an embodiment of the present application further provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps of any one of the methods in the aforementioned method embodiments are implemented.

[0168] And an electronic device comprising:

[0169] one or more processors; and

[0170] A memory associated with the one or more processors, the memory being used to store program instructions, wherein the program instructions, when read and executed by the one or more processors, execute the steps of the method described in any one of the aforementioned method embodiments.

[0171] The present application also provides a computer program product, comprising a computer program, which implements the steps of any one of the methods described in the aforementioned method embodiments when executed by a processor.

[0172] 7 exemplarily shows the architecture of an electronic device, which may include a processor 710, a video display adapter 711, a disk drive 712, an input / output interface 713, a network interface 714, and a memory 720. The processor 710, the video display adapter 711, the disk drive 712, the input / output interface 713, the network interface 714, and the memory 720 may be communicatively connected via a communication bus 730.

[0173] Among them, the processor 710 can be implemented by a general-purpose CPU, a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, etc., to execute relevant programs to implement the technical solutions provided in this application.

[0174] The memory 720 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 720 can store an operating system 721 for controlling the operation of the electronic device 700 and a basic input and output system (BIOS) 722 for controlling the low-level operations of the electronic device 700. In addition, a web browser 723, a data storage management system 724, and a voice quality assessment device 725 can also be stored. The above-mentioned voice quality assessment device 725 can be an application program that specifically implements the operations of the aforementioned steps in the embodiment of the present application. In short, when the technical solution provided by the present application is implemented by software or firmware, the relevant program code is stored in the memory 720 and is called and executed by the processor 710.

[0175] The input / output interface 713 is used to connect input / output modules to implement information input and output. The input / output modules can be configured as components in the device (not shown in the figure) or can be externally connected to the device to provide corresponding functions. Input devices may include a keyboard, mouse, touch screen, microphone, various sensors, etc., and output devices may include a display, speaker, vibrator, indicator light, etc.

[0176] The network interface 714 is used to connect to a communication module (not shown) to enable communication between the device and other devices. The communication module can communicate via a wired method (such as USB, network cable, etc.) or a wireless method (such as mobile network, WiFi, Bluetooth, etc.).

[0177] The bus 730 comprises a pathway for transmitting information between the various components of the device (eg, the processor 710 , the video display adapter 711 , the disk drive 712 , the input / output interface 713 , the network interface 714 , and the memory 720 ).

[0178] It should be noted that although the above device only shows the processor 710, video display adapter 711, disk drive 712, input / output interface 713, network interface 714, memory 720, bus 730, etc., in the specific implementation process, the device may also include other components necessary for normal operation. In addition, it will be understood by those skilled in the art that the above device may also include only the components necessary to implement the solution of the present application, and does not necessarily include all the components shown in the figure.

[0179] Through the description of the above embodiments, it can be seen that those skilled in the art can clearly understand that the present application can be implemented by means of software plus a necessary general hardware platform. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer program product, which can be stored in a storage medium such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in various embodiments of the present application or certain parts of the embodiments.

[0180] The above is a detailed introduction to the technical solutions provided by this application. Specific examples are used herein to illustrate the principles and implementation methods of this application. The description of the above embodiments is only intended to help understand the method and core concept of this application. At the same time, for those skilled in the art, based on the concept of this application, there may be changes in the specific implementation methods and application scope. In summary, the contents of this specification should not be understood as limiting this application.

Claims

1. A method for evaluating speech quality, the method comprising: Obtaining a speech signal to be evaluated and its corresponding reference speech signal; Extracting the implicit feature representation of the reference speech signal and the implicit feature representation of the speech signal to be evaluated using a speech quality assessment model, fusing the implicit feature representation of the reference speech signal and the implicit feature representation of the speech signal to be evaluated to obtain a fused feature representation, and mapping the fused feature representation to obtain a speech quality assessment result of the speech signal to be evaluated; The speech quality assessment model is pre-trained using a neural network.

2. The method according to claim 1, wherein The method further includes: performing feature extraction on the reference speech signal and the speech signal to be evaluated, respectively, to obtain spectral features of the reference speech signal and spectral features of the speech signal to be evaluated; The extracting of the implicit feature representation of the reference speech signal using the speech quality assessment model and the extracting of the implicit feature representation of the speech signal to be evaluated include: downsampling the spectral features of the reference speech signal using the speech quality assessment model to obtain the implicit feature representation of the reference speech signal, and downsampling the spectral features of the speech signal to be evaluated to obtain the implicit feature representation of the speech signal to be evaluated.

3. The method according to claim 2, wherein: The spectrum feature includes a Mel spectrum feature; Performing feature extraction on the reference speech signal and the speech signal to be evaluated respectively to obtain the spectral features of the reference speech signal and the spectral features of the speech signal to be evaluated includes: Converting the reference speech signal and the speech signal to be evaluated into Mel spectra respectively; Dividing the Mel-spectrogram corresponding to the reference speech signal into one or more spectrum segments of a preset length, with adjacent spectrum segments partially overlapping, and using the one or more spectrum segments of the preset length as spectrum features of the reference speech signal; and The Mel-spectrum spectrum corresponding to the speech signal to be evaluated is divided into one or more spectrum segments of a preset length, with adjacent spectrum segments partially overlapping, and the one or more spectrum segments of the preset length are used as the spectrum features of the speech signal to be evaluated.

4. The method according to any one of claims 1 to 3, wherein Fusing the latent feature representation of the reference speech signal and the latent feature representation of the speech signal to be evaluated to obtain a fused feature representation includes: Performing a dot product on the latent feature representation of the reference speech signal and the latent feature representation of the speech signal to be evaluated; The feature representation obtained by the dot product is concatenated with the latent feature representation of the speech signal to be evaluated to obtain the fused feature representation.

5. The method according to any one of claims 1 to 4, wherein Obtaining a speech quality assessment result of the speech signal to be assessed by using the fusion feature representation mapping includes: Performing pooling processing on the fused feature representation, wherein the pooling processing includes pooling based on an attention mechanism; The feature representation obtained by the pooling process is mapped to a speech quality score space to obtain a speech quality score of the speech signal to be evaluated, or the feature representation obtained by the pooling process is mapped to a speech quality rating space to obtain a speech quality rating of the speech signal to be evaluated.

6. A testing method, comprising: Obtaining a reference speech signal; Inputting the reference speech signal into the system to be tested, and obtaining the speech signal output by the system to be tested as the speech signal to be evaluated; Obtaining a speech quality assessment result of the speech signal to be assessed using the method described in any one of claims 1 to 5; The test result of the system to be tested is determined using the speech quality evaluation result.

7. A method for training a speech quality assessment model, the method comprising: Acquiring training data comprising a plurality of training samples, wherein the training samples include impaired speech signal samples and corresponding reference speech signal samples and speech quality assessment labels; The training data is used to train a neural network-based speech quality assessment model, wherein the speech quality assessment model extracts latent feature representations of the reference speech signal samples and the impaired speech signal samples, fuses the latent feature representations of the reference speech signal samples and the impaired speech signal samples to obtain a fused feature representation, and uses the fused feature representation to map and obtain a speech quality assessment result of the impaired speech signal sample; the training objectives include: minimizing the difference between the speech quality assessment result of the impaired speech signal sample and the corresponding speech quality assessment label.

8. The method according to claim 7, wherein: The method further includes: performing feature extraction on the reference speech signal sample and the impaired speech signal sample respectively to obtain spectral features of the reference speech signal sample and spectral features of the impaired speech signal sample, wherein the spectral features include Mel spectrum; The speech quality assessment model extracts the implicit feature representation of the reference speech signal sample, and extracts the implicit feature representation of the damaged speech signal sample, including: the speech quality assessment model downsampling the spectral features of the reference speech signal sample to obtain the implicit feature representation of the reference speech signal sample, and downsampling the spectral features of the damaged speech signal sample to obtain the implicit feature representation of the damaged speech signal sample.

9. The method according to claim 7 or 8, wherein Fusing the latent feature representation of the reference speech signal sample and the latent feature representation of the damaged speech signal sample to obtain a fused feature representation includes: Performing a dot product on the latent feature representation of the reference speech signal sample and the latent feature representation of the impaired speech signal sample; The feature representation obtained by the dot product is concatenated with the latent feature representation of the damaged speech signal sample to obtain the fused feature representation.

10. The method according to any one of claims 7 to 9, wherein Obtaining a speech quality assessment result of the impaired speech signal sample by using the fused feature representation mapping includes: Performing pooling processing on the fused feature representation, wherein the pooling processing includes pooling based on an attention mechanism; The feature representation obtained by the pooling process is mapped to a speech quality score space to obtain a speech quality score of the impaired speech signal sample, or the feature representation obtained by the pooling process is mapped to a speech quality rating space to obtain a speech quality rating of the impaired speech signal sample.

11. A speech quality assessment device, comprising: a signal acquisition unit, configured to acquire a speech signal to be evaluated and a corresponding reference speech signal; a quality assessment unit configured to extract an implicit feature representation of the reference speech signal and an implicit feature representation of the speech signal to be assessed using a speech quality assessment model, fuse the implicit feature representation of the reference speech signal and the implicit feature representation of the speech signal to be assessed to obtain a fused feature representation, and map the fused feature representation to obtain a speech quality assessment result of the speech signal to be assessed; The speech quality assessment model is pre-trained using a neural network.

12. A device for training a speech quality assessment model, the device comprising: a sample acquisition unit configured to acquire training data comprising a plurality of training samples, wherein the training samples include a damaged speech signal sample and its corresponding reference speech signal sample and a speech quality assessment label; A model training unit is configured to train a neural network-based speech quality assessment model using the training data, wherein the speech quality assessment model extracts latent feature representations of the reference speech signal sample and the impaired speech signal sample, fuses the latent feature representations of the reference speech signal sample and the impaired speech signal sample to obtain a fused feature representation, and uses the fused feature representation to map to obtain a speech quality assessment result of the impaired speech signal sample; the training goal includes: minimizing the difference between the speech quality assessment result of the impaired speech signal sample and the corresponding speech quality assessment label.

13. A computer-readable storage medium having a computer program stored thereon, wherein when the program is executed by a processor, the steps of the method according to any one of claims 1 to 10 are implemented.

14. An electronic device comprising: one or more processors; as well as A memory associated with the one or more processors, the memory being used to store program instructions, wherein the program instructions, when read and executed by the one or more processors, perform the steps of the method according to any one of claims 1 to 10.

15. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the steps corresponding to the method according to any one of claims 1 to 10.

Images