Speech quality evaluation method and device, storage medium and electronic equipment

Abstract

Embodiments of the present disclosure relate to a speech quality evaluation method and device, a computer readable storage medium and an electronic device, and relate to the technical field of computers. The method comprises: obtaining a corpus file and a corresponding recording file; extracting corpus features corresponding to the corpus file and recording features corresponding to the recording file based on a trained evaluation model; fusing the corpus features and the recording features to obtain fused features; and fitting the fused features to obtain an evaluation result. The present disclosure can improve the accuracy of speech quality evaluation.

Description

TECHNICAL FIELD

[0001] Embodiments of the present disclosure relate to the technical field of computer, and in particular, to a voice quality evaluation method, a voice quality evaluation device, a computer readable storage medium, and an electronic device. BACKGROUND

[0002] In order to improve the call quality in the mobile communication process, the call quality in the mobile communication process can be evaluated.

[0003] In the related art, the prediction can be performed based on the MOS (Mean Opinion Score). Specifically, the degraded signal after wireless network transmission can be processed by a professional instrument, compared with the original signal after transformation, and mapped to the prediction of the subjective average opinion score through a cognitive model.

[0004] In the above manner, the playback and recording test is performed by two communication terminals, the difference between the lossy waveform after network transmission and the original waveform is compared, and the subjective model is used, which cannot be directly associated with user perception. Moreover, it can lead to a lower MOS score and inaccurate evaluation. SUMMARY

[0005] The purpose of the present disclosure is to provide a voice quality evaluation method, a voice quality evaluation device, an electronic device, and a computer readable storage medium, thereby at least partially overcoming the problem of low accuracy of voice quality evaluation due to the limitations and defects of the related art.

[0006] According to a first aspect of the present disclosure, a voice quality evaluation method is provided, comprising: obtaining a corpus file and a corresponding recording file; extracting corpus features corresponding to the corpus file and recording features corresponding to the recording file based on a trained evaluation model respectively; fusing the corpus features and the recording features to obtain fused features, and fitting the fused features to obtain an evaluation result.

[0007] In an exemplary embodiment of the present disclosure, the extracting of the corpus features corresponding to the corpus file and the recording features corresponding to the recording file based on the trained evaluation model respectively comprises: performing pre-emphasis, framing, and slicing operations on the corpus file and the recording file respectively to obtain corresponding mel spectrograms; performing convolution processing on the mel spectrograms to obtain the corpus features corresponding to the corpus file and the recording features corresponding to the recording file.

[0008] In an example embodiment of the present disclosure, the fusing the corpus feature and the recording feature to obtain a fused feature and fitting the fused feature to obtain the evaluation result comprises: splicing the corpus feature and the recording feature to obtain a spliced feature; and performing full connection processing on the spliced feature to obtain the evaluation result.

[0009] In an example embodiment of the present disclosure, the method further comprises: evaluating a sample corpus file and a sample recording file based on a reference evaluation manner to obtain a reference evaluation result; evaluating the sample corpus file and the sample recording file based on the evaluation model to obtain a predicted evaluation result; analyzing the reference evaluation result and the predicted evaluation result, and training the evaluation model according to an analysis result to obtain a trained evaluation model.

[0010] In an example embodiment of the present disclosure, the analyzing the reference evaluation result and the predicted evaluation result, and training the evaluation model according to an analysis result to obtain a trained evaluation model comprises: measuring the reference evaluation result and the predicted evaluation result based on a loss function to obtain difference data between the reference evaluation result and the predicted evaluation result; training the evaluation model based on the difference data to obtain the trained evaluation model.

[0011] In an example embodiment of the present disclosure, the training the evaluation model based on the difference data to obtain the trained evaluation model comprises: performing comparative analysis on the difference data and determining a derivative change rate corresponding to the difference data to smooth the difference data; and iteratively adjusting model parameters of the evaluation model based on the smoothed difference data until a training end condition is met to obtain the trained evaluation model.

[0012] In an example embodiment of the present disclosure, the iteratively adjusting model parameters of the evaluation model based on the smoothed difference data until a training end condition is met to obtain the trained evaluation model comprises: iteratively adjusting model parameters of the evaluation model based on the smoothed difference data at a learning rate until a difference between an output predicted evaluation result and the reference evaluation result is less than a difference threshold to obtain the trained evaluation model.

[0013] According to a second aspect of the present disclosure, a speech quality evaluation device is provided, comprising: a file acquisition module configured to acquire a corpus file and a corresponding recording file; corpus features corresponding to the corpus file are extracted based on a trained evaluation model, and recording features corresponding to the recording file are extracted; the corpus features and the recording features are fused to obtain fused features, and the fused features are fitted to obtain an evaluation result.

[0014] According to a third aspect of the present disclosure, a computer readable storage medium is provided, which stores a computer program, and the computer program is executed by a processor to implement the speech quality evaluation method according to any one of the preceding aspects.

[0015] According to a fourth aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory configured to store executable instructions of the processor; wherein the processor is configured to execute the speech quality evaluation method according to any one of the preceding aspects via execution of the executable instructions.

[0016] In the speech quality evaluation method, the speech quality evaluation device, the computer readable storage medium and the electronic device provided in the embodiments of the present disclosure, on the one hand, the trained evaluation model has the same processing manner for corpus files of any type, so that the evaluation process is not affected in the case of switching corpus files, which is more suitable for various actual application scenarios and improves the matching degree and authenticity with real scenarios. On the other hand, the trained evaluation model can cope with various new problems, reduce the inaccuracy of the MOS score caused by the difference between the terminal sound card and other factors or the corpus file and the recording file, and improve the comprehensiveness and accuracy of the evaluation result. On the other hand, the trained evaluation model is used for speech quality evaluation. Since the trained evaluation model is obtained by large-scale training of specific environment and specific factors, the speech evaluation can cope with various new problems as the training is enriched and improved, which increases the application range and improves the universality.

[0017] It should be understood that the foregoing general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0018] The accompanying drawings, which are incorporated into and form part of the specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the disclosure. Obviously, the drawings in the following description are only some embodiments of the present disclosure, and other drawings can be obtained by those skilled in the art without creative labor based on these drawings.

[0019] Figure 1 A system architecture diagram for speech quality evaluation is schematically shown in an embodiment of the present disclosure.

[0020] Figure 2 A flow chart of a voice quality evaluation method according to an embodiment of the present disclosure is shown schematically.

[0021] Figure 3 A flow chart of preprocessing of an original recording file according to an embodiment of the present disclosure is shown schematically.

[0022] Figure 4 A flow chart of transmission between a corpus file and a recording file according to an embodiment of the present disclosure is shown schematically.

[0023] Figure 5 A flow chart of evaluation based on an evaluation model according to an embodiment of the present disclosure is shown schematically.

[0024] Figure 6 A flow chart of training of an evaluation model according to an embodiment of the present disclosure is shown schematically.

[0025] Figure 7 A flow chart of training of an evaluation model according to an embodiment of the present disclosure is shown schematically.

[0026] Figure 8 A flow chart of quality evaluation according to a trained evaluation model according to an embodiment of the present disclosure is shown schematically.

[0027] Figure 9 A block diagram of a voice quality evaluation apparatus according to an embodiment of the present disclosure is shown schematically.

[0028] Figure 10 A block diagram of an electronic device according to an embodiment of the present disclosure is shown schematically. DETAILED DESCRIPTION

[0029] Example implementations will now be described more fully with reference to the accompanying drawings. Example implementations can be implemented in any

[0030] Furthermore, the accompanying drawings are merely illustrative of this disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0031] In related technologies, a MOS-based approach can be used for voice quality assessment. Specifically, the degraded signal transmitted via wireless network is processed through level adjustment, filtering, and other steps using specialized instruments. The transformed signal is then compared with the original signal, and a cognitive model is used to map it to a prediction of the subjective average opinion score.

[0032] The above method may have the following problems: Playback and recording tests using two communication terminals, comparing the lossy waveform after network transmission with the original waveform, rely on a subjective model that cannot directly correlate with user perception. If the recorded file deviates from the corpus file by more than 2 seconds, it will result in a lower MOS score. Furthermore, with the increasing noise reduction capabilities of mobile terminals, applying the original voice evaluation method is prone to inaccurate assessments.

[0033] In this embodiment of the disclosure, to address the aforementioned technical problems, a voice quality assessment method is provided, which can be applied to application scenarios based on mobile communication networks. A classification neural network is introduced into the algorithm model, and through large-scale training of the scoring network under specific environments and factors in the early stages, voice quality assessment is performed through corpus file processing, voice recording, evaluation model training, and voice evaluation processes. This makes the voice call quality assessment results initiated based on mobile network environments such as 4G, 5G, and WiFi more realistic.

[0034] Figure 1 The diagram illustrates an application scenario for communication based on a mobile network. (Reference) Figure 1 As shown, communication terminal 101 and communication terminal 102 conduct voice communication based on a mobile network. During the communication process, either communication terminal can be used as the sender and the other as the receiver. Here, we will use communication terminal 101 as the sender and communication terminal 102 as the receiver as an example for explanation.

[0035] In some embodiments, the corpus file can be played at a sending end and transmitted to a receiving end through a mobile network to output an audio file at the receiving end. The audio file can be a lossy audio file. Further, the corpus file and the audio file can be input to the trained evaluation model, and the corpus file and the audio file are processed by the trained evaluation model to output an evaluation result, so as to determine the voice quality according to the evaluation result.

[0036] In the embodiments of the present disclosure, the corpus features of the corpus file and the audio features of the audio file are extracted by the trained evaluation model, the corpus features and the audio features are fused to obtain fused features, and the evaluation result can be obtained based on the fused features. Since the evaluation process can be completed based on the cloud platform, there is no need for external terminal equipment of the mobile phone, and the test cost is reduced. The method can be adapted to multiple types of speech coding rates.

[0037] Next, each step of the voice quality evaluation method in the embodiments of the present disclosure will be described in detail with reference to Figure 2

[0038] In step S210, the corpus file and the corresponding audio file are obtained.

[0039] In the embodiments of the present disclosure, the corpus file can be a corpus file processed based on an original corpus file. The original corpus file can be composed of corpus files of multiple types of different languages and different types. In order to improve the accuracy, the original corpus file can be processed. For example, the original corpus file can be band-pass filtered and uniformly volume-processed based on the ITU-P863 standard. Further, the processed original corpus file can be used as the corpus file for intelligent voice quality evaluation. The band-pass filtering is used to filter the original corpus file, and the uniform volume processing refers to processing the volume of the original corpus file to the same volume, which can be, for example, -20db.

[0040] The corpus file can be a corpus file played at a sending end, and the audio file can be a lossy audio file output at a receiving end, the content of which can correspond to the corpus file. The sending end and the receiving end can perform voice communication based on a mobile communication network. The transmission mode of the sending end and the receiving end can be any one of 4G, 5G network or WiFi mode. The sending end transmits the corpus file to the receiving end through the mobile communication network, so that the receiving end outputs the lossy audio file.

[0041] ​In the embodiments of the present disclosure, the instrument can play a pre-recorded voice file at the sending end, and the voice file can be used as a corpus file. The sending end can be a same operator network terminal, or a different operator network terminal, etc. The sending end can be a mobile phone terminal, a CPE (Customer Premise Equipment), a tablet computer, or other devices, as long as the voice service of the mobile network can be completed. Specifically, the played corpus file can be any type of language such as Chinese, English, French, etc.; the corpus file can be a voice file of a male or female, etc.; and the corpus file contains voices of any age group. The played corpus file can come from any media, including movies, TV programs, radio programs, Internet audio and video, etc. During the playing of the corpus file, the playing information of the corpus file can be set. The playing information can include one or more of the volume of the played voice, the playing time length, and the playing mode. Among them, the volume of the played voice can be set to 80%, for example; the playing time length of the corpus file can be set to 8 seconds; and the playing mode can be set to one or more of the playback mode, continuous playback, interval playback, and repeated playback.

[0042] Exemplarily, referring to FIG. 1, the original corpus file can be preprocessed, and corresponding processing can be performed on the preprocessed original corpus file according to whether the volume of the preprocessed original corpus file meets the volume requirement. Figure 3 Exemplarily, referring to FIG. 1, the original corpus file can be preprocessed, and corresponding processing can be performed on the preprocessed original corpus file according to whether the volume of the preprocessed original corpus file meets the volume requirement.

[0043] During the communication between the sending end and the receiving end, the receiving end can record the received corpus file as a recording file while the sending end plays the corpus file. Further, the voice quality evaluation process can be implemented based on the received recording file. Exemplarily, referring to FIG. 1, the corpus file, i.e., the original standard corpus file, can be transmitted through a mobile network, and the transmitted corpus file can be used as a recording file, which can be a lossy recording file. Among them, the transmission mode can be any one of a 4G network, a 5G network, or a WiFi mode. Figure 4

[0044] ​Next, in step S220, corpus features corresponding to the corpus files and audio features corresponding to the audio files are extracted based on the trained evaluation model respectively.

[0045] In the embodiments of the present disclosure, the corpus files and the audio files can be input into the trained evaluation model to perform evaluation processing on the corpus files and the audio files by the trained evaluation model, so as to obtain an evaluation score as an evaluation result. It is shown in Figure 5

[0046] It should be noted that, in order to improve the accuracy of the evaluation model, the evaluation model can be trained to obtain a trained evaluation model, and then the speech quality of the corpus files and the audio files is predicted according to the trained evaluation model. In order to improve the accuracy of the evaluation model, the number of preset categories in the training process of the evaluation model is greater than or equal to 4000; the training process needs to invest 1 million-4 million POLQA corpus data for training; the MOS score of the corpus data needs to cover the range of [1 point, 5 points], and the MOS score needs to be uniformly distributed. In addition, in order to make the evaluation result further cover the full-field test data, corpus sample data with typical characteristics can be added to enrich the classification neural network, so that the evaluation process is closer to the actual scene and accurate scoring.

[0047] Figure 6 A flowchart for training the evaluation model is shown in Figure 6 The main steps include the following steps:

[0048] In step S610, the sample corpus files and the sample audio files are evaluated based on a reference evaluation method to obtain a reference evaluation result.

[0049] In step S620, the sample corpus files and the sample audio files are evaluated based on the evaluation model to obtain a predicted evaluation result.

[0050] In step S630, the reference evaluation result and the predicted evaluation result are analyzed, and the evaluation model is trained according to the analysis result to obtain a trained evaluation model.

[0051] ​In the embodiments of the present disclosure, the reference evaluation mode can be a POLQA evaluation mode. The POLQA algorithm is a new generation of speech quality evaluation standard, which is suitable for speech quality evaluation in fixed network, mobile communication network and IP network. POLQA is determined as the recommended specification P.863, which can be used for high-definition voice, 3G, 4G / VoLTE, 5G network speech quality evaluation. Exemplarily, the sample corpus file and the sample recording file can be input into the POLQA evaluation mode as input to realize POLQA evaluation, so as to obtain the POLQA evaluation output as the reference evaluation result. Specifically, the signals of the sample corpus file and the sample recording file are filtered, time-aligned, sample rate estimated, input into a perception model to obtain the internal representation of the signals of the sample corpus file and the sample recording file, and the difference between the internal representations is obtained. The difference between the internal representations is input into a cognitive model to obtain a POLQA score, and finally mapped to a MOS-LQO score as a reference score result. The reference evaluation result can be a standard score obtained by evaluating the sample corpus file and the sample recording file, which can be 3.0 or other numerical values.

[0052] For a sample recording file, if the reference evaluation result given by the standard library is 3.0, some parameters corresponding to the reference evaluation result can be extracted from the standard library according to the reference evaluation result, the frequency and sound characteristics are extracted, an evaluation model is constructed based on the frequency characteristics and sound characteristics, and the score of the sample recording file is predicted based on the evaluation model.

[0053] Next, the sample corpus file and the sample recording file can be evaluated by the evaluation model to obtain a predicted evaluation result. Exemplarily, the sample corpus file and the sample recording file can be input into the evaluation model, and the sample corpus file and the sample recording file can be pre-emphasized, framed and sliced by the evaluation model to obtain corresponding mel spectrograms; the mel spectrograms are convoluted to obtain corpus features corresponding to the sample corpus file and recording features corresponding to the sample recording file. The corpus features and the recording features are spliced to obtain spliced features; the spliced features are fully connected to obtain the predicted evaluation result. The predicted evaluation result can be a predicted score based on the trained evaluation model, which can be the same as or different from the reference evaluation result, and is calculated according to the model training.

[0054] Further, the reference evaluation result and the predicted evaluation result can be analyzed to obtain an analysis result, and the evaluation model can be trained according to the analysis result. Exemplarily, the reference evaluation result and the predicted evaluation result can be compared to obtain a difference between the reference evaluation result and the predicted evaluation result, and the difference between the reference evaluation result and the predicted evaluation result can be analyzed to obtain the analysis result. Figure 7As shown in FIG. 1, the reference evaluation result and the predicted evaluation result can be measured based on a loss function to obtain difference data between the reference evaluation result and the predicted evaluation result. After obtaining the difference data, the difference data can be processed by DIFF to obtain target difference data that has differences.

[0055] On this basis, the derivative change rate can be calculated based on the target difference data to smooth the target difference data to obtain smoothed difference data, and the model parameters of the evaluation model can be adjusted based on the smoothed difference data, that is, the parameter adjustment of the evaluation model is performed based on the smoothed difference data, that is, the Net Change processing. Further, the adjusted evaluation model can be determined based on the adjusted model parameters, and the sample corpus file and the sample audio file can be processed based on the adjusted evaluation model to obtain the predicted evaluation result, so as to iteratively train the evaluation model based on the predicted evaluation result until the training end condition is met, so as to obtain the trained evaluation model. The training end condition can be that the difference between the output predicted evaluation result and the reference evaluation result is less than the difference threshold, or the number of training times reaches the number threshold. The difference threshold can be a small number, for example, 0.0001 or other numbers, which can be determined according to actual needs. The number threshold can also be determined according to actual needs.

[0056] It should be noted that when updating the model parameters of the evaluation model, the model parameters of the evaluation model can be updated according to the learning rate. For different iterative training processes, since the model parameters of the evaluation model can be updated in each iteration process, the predicted evaluation result obtained based on the evaluation model with different parameters can also be different, which can cause the difference data in each training process to be different, and therefore the learning rate used in each training process can be the same or different. The learning rate is used to determine whether the objective function can converge to a local minimum value and when to converge to a minimum value. The learning rate can be increased or decreased according to the change of the number of training times, which is determined according to the actual difference data.

[0057] In the evaluation process, the reference Figure 7 As shown in FIG. 1, the trained evaluation model can include an MFCC processing process, a CNN processing process, a CAT processing process, and a DNN processing process. Based on this, the corpus file and the audio file can be processed by the MFCC processing process and the CNN processing process, respectively.

[0058] Firstly, the MFCC processing of the corpus file is described. The MFCC processing process can be a mel-frequency cepstrum operation process. The MFCC represents the short-time power spectrum of the speech, which is formed by performing linear cosine conversion on the log power spectrum of the speech in the corpus file in a nonlinear mel scale of frequency. In the MFCC processing process, the main operations can include pre-emphasis, framing, and slicing of the signal, based on which the mel spectrogram corresponding to the corpus file and the mel spectrogram corresponding to the recording file can be obtained. The result obtained by the MFCC processing process, i.e., the mel spectrogram, is also a two-dimensional image, i.e., two-dimensional data.

[0059] Next, the two-dimensional data corresponding to the mel spectrogram can be subjected to a CNN processing process to perform convolution processing to obtain corpus features corresponding to the corpus file. The corpus features can be one-dimensional features. Exemplarily, the two-dimensional data can be processed into one-dimensional features by convolution processing, and the product-sum process can be realized by cross-correlation operation, so as to obtain the corpus features corresponding to the corpus file.

[0060] The process of processing the recording file is the same as the process of processing the corpus file. Exemplarily, the recording file can be subjected to pre-emphasis, framing, and slicing operations to obtain a mel spectrogram corresponding to the recording file; and the mel spectrogram corresponding to the recording file is further subjected to convolution processing to process the two-dimensional data represented by the mel spectrogram corresponding to the recording file into one-dimensional features, and the one-dimensional features are determined as recording features corresponding to the recording file.

[0061] Next, with reference to the step S230 shown in Figure 2 , the corpus features and the recording features are fused to obtain fusion features, and the fusion features are fitted to obtain an evaluation result.

[0062] In the embodiment of the present disclosure, after obtaining the trained evaluation model, the corpus features and the recording features extracted by the features can be fused to obtain fusion features based on the trained evaluation model. Here, the fusion can be a concatenation CAT process, and the obtained fusion features are concatenation features. Further, the obtained concatenation features can be subjected to full connection processing to realize fitting of the concatenation features, thereby obtaining an evaluation result. Exemplarily, the obtained fusion features can be input into a DNN layer for full connection processing, so that the trained evaluation model outputs an evaluation result. The full connection processing can be realized by convolution operation.

[0063] The evaluation result can be a MOS score. The MOS (Mean Opinion Score) score uses a waveform attenuation score technology of sound. The sound playback and recording test is performed through two mobile phones, the difference between the lossy waveform after network transmission and the original waveform is compared, and a subjective model is used, which cannot be directly associated with user perception. The MOS score can be obtained by scoring the difference between the corpus file and the recording file, indicating the quality of the recording file corresponding to the corpus file, thereby representing the voice quality in the process of voice transmission based on the mobile network. Moreover, the MOS score is positively correlated with the quality, that is, the higher the MOS score, the better the quality.

[0064] The technical solution in the embodiment of the disclosure can not affect the evaluation process in the case of arbitrary switching of corpus files, is more suitable for various actual application occasions, can adapt to multiple types of voice coding rates, has universality and higher authenticity. Since the voice quality is automatically predicted by the trained evaluation model, the MOS score is reduced due to factors such as terminal sound cards (such as noise reduction), and the accuracy of the score and the accuracy of the voice quality evaluation are improved. Since the trained evaluation model has a classification neural network, the evaluation model can be enriched and improved during the training process, so that the trained evaluation model is more accurate, and can cope with various newly added problems, improving the comprehensiveness and accuracy of the prediction. In addition, the voice quality evaluation process of the recording file can be completed based on the trained evaluation model in the cloud platform, without the need for external terminal equipment of the mobile phone, reducing the test cost, and improving the processing efficiency. It can improve the evaluation ability of the operator in the mobile network VoNR, VoLTE and other voice services, directly master the voice service perception of the user. The MOS evaluation is not realistic due to factors such as terminal equipment and test personnel. The quality of modern wireless communication networks is generally good, and the patent can promote the refinement or redefinition of the MOS score standard, and realize more targeted or practical voice quality evaluation.

[0065] The disclosure also provides a voice quality evaluation device. Referring to Figure 9 The voice quality evaluation device 900 mainly includes the following modules:

[0066] The file acquisition module 901 is configured to acquire a corpus file and a corresponding recording file.

[0067] The feature extraction module 902 is configured to extract corpus features corresponding to the corpus file and extract recording features corresponding to the recording file based on the trained evaluation model.

[0068] The evaluation prediction module 903 is configured to fuse the corpus features and the recording features to obtain fused features, and fit the fused features to obtain evaluation results.

[0069] In an example embodiment of the present disclosure, the extracting the corpus features corresponding to the corpus files and the recording features corresponding to the recording files based on the trained evaluation model respectively includes: performing pre-emphasis, framing and slicing operations on the corpus files and the recording files respectively to obtain corresponding mel spectrograms; and performing convolution processing on the mel spectrograms to obtain the corpus features corresponding to the corpus files and the recording features corresponding to the recording files.

[0070] In an example embodiment of the present disclosure, the fusing the corpus features and the recording features to obtain fused features, and fitting the fused features to obtain evaluation results includes: splicing the corpus features and the recording features to obtain spliced features; and performing full connection processing on the spliced features to obtain the evaluation results.

[0071] In an example embodiment of the present disclosure, the method further includes: evaluating sample corpus files and sample recording files based on a reference evaluation method to obtain reference evaluation results; evaluating the sample corpus files and the sample recording files based on the evaluation model to obtain predicted evaluation results; analyzing the reference evaluation results and the predicted evaluation results, and training the evaluation model according to an analysis result to obtain a trained evaluation model.

[0072] In an example embodiment of the present disclosure, the analyzing the reference evaluation results and the predicted evaluation results, and training the evaluation model according to an analysis result to obtain a trained evaluation model includes: measuring the reference evaluation results and the predicted evaluation results based on a loss function to obtain difference data between the reference evaluation results and the predicted evaluation results; training the evaluation model based on the difference data to obtain the trained evaluation model.

[0073] In an example embodiment of the present disclosure, the training the evaluation model based on the difference data to obtain a trained evaluation model includes: performing comparative analysis on the difference data, and determining a derivative change rate corresponding to the difference data to smooth the difference data; iteratively adjusting model parameters of the evaluation model based on the smoothed difference data until a training end condition is met, to obtain the trained evaluation model.

[0074] In an example embodiment of the present disclosure, the model parameters of the evaluation model are iteratively adjusted based on the smoothed difference data until a training end condition is met to obtain the trained evaluation model, comprising: iteratively adjusting the model parameters of the evaluation model based on the smoothed difference data according to a learning rate until a difference between an output predicted evaluation result and the reference evaluation result is less than a difference threshold to obtain the trained evaluation model.

[0075] It should be noted that the specific details of each module in the voice quality evaluation device described above have been described in detail in the corresponding voice quality evaluation method, and therefore will not be described here.

[0076] It should be noted that although several modules or units of the device for action execution are mentioned in the above detailed description, such division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided into several modules or units.

[0077] In addition, although the steps of the method in the present disclosure are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired result. In addition or alternatively, some steps can be omitted, several steps can be combined into one step, and / or one step can be divided into several steps, etc.

[0078] In an example embodiment of the present disclosure, an electronic device capable of implementing the above method is also provided.

[0079] Those skilled in the art can understand that various aspects of the present disclosure can be implemented as a system, a method or a program product. Therefore, various aspects of the present disclosure can be embodied as a complete hardware embodiment, a complete software embodiment (including firmware, microcode, etc.), or an embodiment combining hardware and software aspects, which can be collectively referred to as "circuitry", "module" or "system" here.

[0080] The electronic device 1000 according to this embodiment of the present disclosure will be described below with reference to Figure 10 Figure 10 The electronic device 1000 shown is merely an example and should not impose any limitation on the functions and use range of the embodiments of the present disclosure.

[0081] As Figure 10 ​As shown, the electronic device 1000 is in the form of a general computing device. Components of the electronic device 1000 can include, but are not limited to, the at least one processing unit 1010 described above, the at least one storage unit 1020 described above, a bus 1030 that connects the different system components including the storage unit 1020 and the processing unit 1010, and a display unit 1040.

[0082] The storage unit stores program code that can be executed by the processing unit 1010 such that the processing unit 1010 performs the steps described in the above "Exemplary Methods" section of this specification in accordance with various exemplary embodiments of the present disclosure. For example, the processing unit 1010 can perform the steps shown in FIG. 1 1 1 as described in the above "Exemplary Methods" section of this specification. Figure 2

[0083] The storage unit 1020 can include a readable medium in the form of volatile storage such as a random access memory (RAM) 10201 and / or cache memory 10202, and can further include a read-only memory (ROM) 10203.

[0084] The storage unit 1020 can also include program / utility 10204 having a set of at least one program modules 10205, including but not limited to, an operating system, one or more application programs, other program modules, and program data, and can include an implementation of a network environment, each or a combination thereof.

[0085] The bus 1030 can be representative of one or more of several types of bus structures, including a storage bus or bus controller, a peripheral bus, a graphics acceleration port, a processing bus, or a local bus using any of a variety of bus architectures.

[0086] ​The electronic device 1000 can also communicate with one or more external devices 1100 such as a keyboard, a pointing device, a Bluetooth device, etc.; and one or more devices that enable a user to interact with the electronic device 1000 and / or one or more devices that enable the electronic device 1000 to communicate with one or more other computing devices. Such communication can be via the input / output (I / O) interface 1050. Further, the electronic device 1000 can communicate with one or more networks, such as a local area network (LAN), a wide area network (WAN), and / or the public network, such as the Internet, via the network adapter 1060. As depicted, the network adapter 1060 is in communication with the other modules of the electronic device 1000 through the bus 1030. As will be appreciated by those skilled in the art, the electronic device 1000 can be implemented using one or more of a variety of hardware and / or software components, including but not limited to microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data archival storage systems, etc.

[0087] From the above description of the embodiments, those skilled in the art will readily appreciate that the example embodiments described herein can be implemented by software and / or by hardware. Accordingly, the technical solutions of the embodiments of the present disclosure can be embodied in the form of a software product. The software product can be stored in a non-volatile storage medium (such as a CD-ROM, a USB flash disk, or a mobile hard disk, etc.) or a network, and includes a number of instructions for causing a computing device (such as a personal computer, a server, a terminal device, or an electronic device, etc.) to perform the methods according to the embodiments of the present disclosure.

[0088] In the example embodiments of the present disclosure, a computer-readable storage medium is also provided, which stores a program product capable of implementing the above-described methods. In some possible embodiments, various aspects of the present disclosure can also be implemented in the form of a program product, which includes program codes for causing a terminal device to perform the steps described in the above “Example Method” section according to various example embodiments of the present disclosure when the program product is run on the terminal device.

[0089] The program product for implementing the above-described methods according to the embodiments of the present disclosure can take the form of a portable compact disc read-only memory (CD-ROM) and include program codes, and can be run on a terminal device, such as a personal computer. However, the program product of the present disclosure is not limited to this, and in this document, a readable storage medium can be any tangible medium that contains or stores a program, which can be used by or in conjunction with an instruction execution system, device, or apparatus.

[0090] The program product can employ any combination of one or more computer-readable media. The computer-readable media can be a computer-readable storage medium or a computer-readable signal medium. The computer-readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer-readable storage medium include the following: an electrical connection having one or more wires, a portable disc, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0091] The computer-readable signal medium can include a computer-readable storage medium that is propagated as a carrier wave. The computer-readable signal medium can further be any computer-readable medium that is not a storage medium. The computer-readable signal medium can be a computer-readable storage medium that is a propagated signal on a computer-readable storage medium.

[0092] The program code embodied on the computer-readable media can be transmitted using any appropriate medium, including but not limited to wireless, wired, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0093] The program code can be executed by one or more programmable processors, which can be implemented in one or more computer devices including any combination of a microprocessor, a microcontroller, a digital signal processor, or other processing circuitry. The program code can be written in any combination of one or more programming languages, including an object-oriented programming language such as Java, C++, or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer device, partly on the user's computer device, as a stand-alone software package, partly on the user's computer device and partly on a remote computer device or entirely on the remote computer device or server. In the latter scenario, the remote computer device can be connected to the user's computer device through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer device such as through the Internet using an Internet Service Provider. The program code can also be downloaded to the user's computer device from the remote computer device or server through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer device such as through the Internet using an Internet Service Provider.

[0094] In addition, the above-described flowcharts are merely illustrative of the processes included in the method according to the exemplary embodiments of the present disclosure, and are not intended to limit the purpose. It is easily understood that the processes shown in the above-described flowcharts do not indicate or limit the time sequence of the processes. In addition, it is easily understood that the processes can be executed synchronously or asynchronously, for example, in a plurality of modules.

[0095] Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the features of the disclosure as set forth above. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.

Claims

1. A method of speech quality assessment, characterized by, The method comprises the following steps: obtain a corpus file and a corresponding audio file, the corpus file is a corpus file played by a sending end, the audio file is a lossy audio file output by a receiving end, the corpus file corresponds to the content of the audio file; extract corpus features corresponding to the corpus file and audio features corresponding to the audio file based on the trained evaluation model respectively; fuse the corpus features and the audio features to obtain fused features, and fit the fused features to obtain an evaluation result; the evaluation result is an average opinion score, which is used to represent the quality of the audio file corresponding to the corpus file; wherein, the step of extracting the corpus features corresponding to the corpus file and the audio features corresponding to the audio file based on the trained evaluation model respectively comprises: performing pre-emphasis, framing and slicing operations on the corpus file and the audio file respectively to obtain corresponding mel spectrograms; performing convolution processing on two-dimensional data corresponding to the mel spectrograms, and realizing product summation process through cross-correlation operation to process the two-dimensional data into one-dimensional features, thereby obtaining the corpus features corresponding to the corpus file and the audio features corresponding to the audio file.

2. The voice quality assessment method of claim 1, wherein, The step of fusing the corpus features and the audio features to obtain fused features, and fitting the fused features to obtain an evaluation result comprises: splicing the corpus features and the audio features to obtain spliced features; performing full connection processing on the spliced features to obtain the evaluation result.

3. The voice quality assessment method of claim 1, wherein, The method further comprises: evaluating sample corpus files and sample audio files based on a reference evaluation method to obtain reference evaluation results; evaluating sample corpus files and sample audio files based on an evaluation model to obtain predicted evaluation results; combining the reference evaluation results and the predicted evaluation results for analysis, and training the evaluation model according to the analysis results to obtain a trained evaluation model.

4. The voice quality assessment method of claim 3, wherein, The step of combining the reference evaluation results and the predicted evaluation results for analysis, and training the evaluation model according to the analysis results to obtain a trained evaluation model comprises: measuring the reference evaluation results and the predicted evaluation results based on a loss function to obtain difference data between the reference evaluation results and the predicted evaluation results; training the evaluation model based on the difference data to obtain a trained evaluation model.

5. The voice quality assessment method of claim 4, wherein, The step of training the evaluation model based on the difference data to obtain a trained evaluation model comprises: performing comparative analysis on the difference data, and determining a derivative change rate corresponding to the difference data to smooth the difference data; iteratively adjusting model parameters of the evaluation model based on the smoothed difference data until a training end condition is met, thereby obtaining the trained evaluation model.

6. The voice quality assessment method of claim 5, wherein, The step of iteratively adjusting model parameters of the evaluation model based on the smoothed difference data until a training end condition is met, thereby obtaining the trained evaluation model comprises: According to a learning rate, the model parameters of the evaluation model are iteratively adjusted based on the smoothed difference data until a difference between an output predicted evaluation result and the reference evaluation result is less than a difference threshold, to obtain the trained evaluation model.

7. A speech quality assessment apparatus characterized by comprising: Comprise: A file acquisition module configured to acquire a corpus file and a corresponding audio file, the corpus file being a corpus file played by a sending end, the audio file being a lossy audio file output by a receiving end, the corpus file corresponding to the content of the audio file; A feature extraction module configured to extract corpus features corresponding to the corpus file based on the trained evaluation model, and extract audio features corresponding to the audio file; An evaluation prediction module configured to fuse the corpus features and the audio features to obtain fused features, and fit the fused features to obtain an evaluation result; the evaluation result being a mean opinion score, used to represent the quality of the audio file corresponding to the corpus file; The feature extraction module is configured to perform: Pre-emphasize, frame and slice the corpus file and the audio file respectively to obtain corresponding mel spectrograms; Convolve the two-dimensional data corresponding to the mel spectrograms, and realize the product summation process through cross-correlation operation, to process the two-dimensional data into one-dimensional features, to obtain the corpus features corresponding to the corpus file and the audio features corresponding to the audio file.

8. A computer-readable storage medium having stored thereon a computer program, characterized in that The computer program is executed by the processor to realize the voice quality evaluation method of any one of claims 1-6.

9. An electronic device, comprising: Comprise: A processor; And A memory configured to store executable instructions of the processor; Wherein, the processor is configured to execute the executable instructions to perform the voice quality evaluation method of any one of claims 1-6.

Images