Method, apparatus, device and storage medium for determining the quality of voice data

Abstract

Embodiments of the present disclosure relate to methods, devices, equipment and storage media for determining the quality of voice data. The method includes determining the characteristics of the acquired voice data. The method also includes obtaining a first quality level and a second quality level for the voice data based on the characteristics, the first quality level being related to a first language, and the second quality level being related to a second language. The method also includes determining a target quality level for the voice data based on the first quality level and the second quality level, the target quality level indicating the voice quality of the voice data. Through this method, the accuracy and efficiency of cross-domain evaluation of the quality of voice data can be improved, the utilization rate of data is improved, and the user experience is improved.

Description

Technical Field

[0001] Embodiments of the present disclosure generally relate to the field of voice data processing, and more particularly to a method, apparatus, device, and storage medium for determining the quality of voice data. Background Art

[0002] With the development of computer technology, the level of speech processing is also improving rapidly. Using computing devices to synthesize speech data or convert speech data is increasingly used in various devices and applications. For these speech data, speech quality can be evaluated, because speech quality is the main indicator reflecting the performance of systems such as speech synthesis and speech conversion. The Mean Opinion Score (MOS) is a subjective evaluation score of the speech quality of the audio after the annotator performs a listening test on the synthesized audio. Since traditional MOS scoring requires the participation of a large number of annotators, this subjective evaluation process will result in high costs and excessive time consumption. Therefore, there are still many problems that need to be solved in the processing of speech data. Summary of the invention

[0003] Embodiments of the present disclosure provide a method, an apparatus, a device, and a storage medium for determining the quality of voice data.

[0004] According to a first aspect of the present disclosure, a method for determining the quality of speech data is provided. The method includes determining features of the acquired speech data. The method also includes acquiring a first quality level and a second quality level for the speech data based on the features, the first quality level being related to a first language, and the second quality level being related to a second language. The method also includes determining a target quality level for the speech data based on the first quality level and the second quality level, the target quality level indicating the speech quality of the speech data.

[0005] In a second aspect of the present disclosure, a device for determining the quality of speech data is provided. The device includes a feature determination module configured to determine the features of the acquired speech data; a quality level acquisition module configured to acquire a first quality level and a second quality level for the speech data based on the features, the first quality level being related to a first language, and the second quality level being related to a second language; and a target quality level determination module configured to determine a target quality level for the speech data based on the first quality level and the second quality level, the target quality level indicating the speech quality of the speech data.

[0006] In a third aspect of the present disclosure, an electronic device is provided, comprising at least one processor; and a storage device for storing at least one program, wherein when the at least one program is executed by the at least one processor, the at least one processor implements the method according to the first aspect of the present disclosure.

[0007] In a fourth aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the method according to the first aspect of the present disclosure is implemented.

[0008] It should be understood that the contents described in this content section are not intended to limit the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The above and other objects, features and advantages of the present disclosure will become more apparent through a more detailed description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, wherein like reference numerals generally represent like components throughout the exemplary embodiments of the present disclosure.

[0010] Figure 1 A schematic diagram illustrating an example environment 100 in which devices and / or methods of embodiments of the present disclosure may be implemented;

[0011] Figure 2 illustrates a flow chart of a process 200 for determining the quality of speech data according to an embodiment of the present disclosure;

[0012] Figure 3 A schematic diagram illustrating an example 300 for generating a score for speech data according to an embodiment of the present disclosure;

[0013] Figure 4 A schematic diagram of a process 400 for training a decoder and a speech representation model according to an embodiment of the present disclosure is illustrated;

[0014] Figure 5 A schematic diagram illustrating an example 500 of training a decoder and a speech representation model according to an embodiment of the present disclosure is illustrated;

[0015] Figure 6 A schematic block diagram of an apparatus 600 for determining the quality of speech data according to an embodiment of the present disclosure is illustrated;

[0016] Figure 7 A schematic block diagram of an example device 700 suitable for implementing embodiments of the present disclosure is illustrated.

[0017] In the various drawings, the same or corresponding reference numerals represent the same or corresponding parts. DETAILED DESCRIPTION

[0018] It is understandable that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, scope of use, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0019] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly prompt the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, application, server, or storage medium that performs the operation of the technical solution of the present disclosure according to the prompt message.

[0020] As an optional but non-limiting implementation, in response to receiving an active request from the user, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0021] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that meet the relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0022] It is understandable that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and relevant provisions.

[0023] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0024] In the description of the embodiments of the present disclosure, the term "including" and similar terms should be understood as open inclusion, that is, "including but not limited to". The term "based on" should be understood as "based at least in part on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The terms "first", "second", etc. may refer to different or the same objects. Other explicit and implicit definitions may also be included below.

[0025] As mentioned above, using annotators to score the speech quality of speech data will result in high costs and excessive time. Therefore, the MOS automatic scoring system was proposed, which mainly uses machines to score synthetic audio, replacing the subjective evaluation of annotators, thereby saving time and resources. However, there are at least two challenges in using machines for MOS scoring. The first challenge is the problem of data sparsity. There is not much data used to train the MOS scoring system, which will limit the performance of the scoring system. The second challenge is the scoring of cross-domain synthetic audio (synthesized audio in different languages). Synthesized audio in various languages ​​may not be able to give an accurate score in the MOS scoring system due to the lack of training data in the corresponding language.

[0026] In some traditional solutions, an automatic scoring system is designed. This system uses a pre-trained encoder to automatically score MOS. This solution can achieve good scoring results on data in the same field (the language of the training set and the test set is the same).

[0027] However, this traditional solution has many problems in cross-domain tasks. For example, when training sets from different fields are used separately, cross-domain training sets are very scarce, which will limit the performance of the encoder; when training sets from different fields are used together, the system cannot adapt the synthetic audio of the test set well because the language backgrounds of some training sets and test sets are different. This leads to inaccurate scoring of the MOS system.

[0028] In addition, these traditional solutions do not consider the relationship between different tasks, such as the relationship between automatic speech recognition tasks and MOS automatic scoring. MOS scoring data is sparse, but automatic speech recognition data is massive.

[0029] At least to solve the above and other potential problems, an embodiment of the present disclosure proposes a method for determining the quality of speech data. In this method, a computing device determines the characteristics of the acquired speech data. Then, using the obtained characteristics, a first quality level and a second quality level for the speech data are acquired. These quality levels are related to the type of language. Then, based on the first quality level and the second quality level, the computing device determines a target quality level for the speech data. Through this method, the accuracy and efficiency of cross-domain evaluation of the quality of speech data can be improved, and the utilization rate of data is improved, and the user experience is improved.

[0030] The embodiments of the present disclosure will be described in detail below in conjunction with the accompanying drawings. Figure 1 An example environment 100 is shown in which devices and / or methods of embodiments of the present disclosure may be implemented.

[0031] Included in the environment 100 is a computing device 104 for processing speech data 102 to determine a target quality level 112 in the speech data 102 .

[0032] Examples of computing device 104 include, but are not limited to, personal computers, server computers, handheld or laptop devices, mobile devices (such as mobile phones, personal digital assistants (PDAs), media players, etc.), multiprocessor systems, consumer electronics, minicomputers, mainframe computers, distributed computing environments that include any of the above systems or devices, and the like.

[0033] The voice data 102 received by the computing device 104 includes the voice data of the speaker. The voice data includes, but is not limited to, synthesized voice data or converted language data. For the synthesized voice data, it includes the voice of the simulated person synthesized by the computing device. For the converted voice data, it includes converting the voice of the speaker into the voice of another speaker or converting the content of the speaker into another language. Figure 1 The computing device 104 is shown to receive the voice data 102. This is only an example and not a limitation of the present disclosure. The computing device 104 may also generate the voice data or the voice data may be stored in a local memory of the computing device 104.

[0034] After obtaining the speech data 102, the computing device 104 may process the speech data 102 to obtain the features 106 of the speech data. In one example, the computing device may run a trained wav2vec2 model to extract the contextual representation of the speech data as the features 106. In another example, the computing device runs other speech representation models to obtain the features of the speech data, such as any suitable speech representation model that converts speech information into a vector representation, which can be used to obtain the features of the speech data. The above examples are only used to describe the present disclosure, and are not specific limitations of the present disclosure. Those skilled in the art may use any suitable model to obtain the features of the speech data.

[0035] The computing device 104 can also obtain a first quality level and a second quality level for the speech data based on the feature 106. The first quality level is related to the first language, and the second quality level is related to the second language. In one example, the first language is English and the second language is Chinese. In another example, the first language is Chinese and the second language is English. The above examples are only used to describe the present disclosure, but are not specific limitations of the present disclosure. Those skilled in the art can set the first language and the second language as needed. In some embodiments, the first quality level is generated using an encoder related to the first language, and the second quality level is generated using an encoder related to the second language.

[0036] Figure 1 The computing device 104 is shown to obtain the quality level 108 and the quality level 110, which is only an example and not a specific limitation of the present disclosure. The computing device 104 can also obtain any number of quality levels for any number of languages ​​as needed.

[0037] The computing device 104 determines a target quality level for the speech data 102 from the first quality level 108 and the second quality level 110 to indicate the speech quality of the speech data 102. In an example where more than two quality levels are generated by the computing device 104, the computing device 104 selects the target quality level from the plurality of quality levels. Alternatively or additionally, the computing device 104 uses the language of the speech data to select the quality level corresponding to the language as the target quality level.

[0038] In one example, the target quality level is a score. In another example, the target quality level is different levels, such as good, medium, poor or A, B, C, D. The above examples are only used to describe the present disclosure, but not to specifically limit the present disclosure. Those skilled in the art can set the content of the target quality level as needed.

[0039] Through this method, the accuracy and efficiency of cross-domain evaluation of voice data quality can be improved, the utilization rate of data is increased, and the user experience is improved.

[0040] Combined with the above Figure 1 A block diagram of an example environment 100 is depicted in which embodiments of the present disclosure can be implemented. Figure 2 A flowchart depicting a process 200 for determining the quality of speech data according to an embodiment of the present disclosure. The process 200 may be performed at Figure 1 Executed at computing device 104 .

[0041] At block 202 , characteristics of the acquired speech data are determined. For example, the computing device 104 acquires speech data and then processes the speech data to determine characteristics of the acquired speech data.

[0042] In some embodiments, the voice data acquired by the computing device 104 may be voice data generated by speech synthesis. For example, voice data generated by converting text into speech. In some embodiments, the voice data acquired by the computing device 104 is generated by speech conversion. For example, speech data of a speaker speaking Chinese is converted into speech data of a speaker speaking English. The above examples are only used to describe the present disclosure, but are not specific limitations of the present disclosure. Any suitable voice data can be used to determine the quality of the speech. In this way, the quality of the synthesized speech or converted speech can be quickly determined.

[0043] In some embodiments, the feature of the speech data is a vector obtained for the speech data. In some embodiments, the feature of the speech data is a contextual representation of the speech data. In some embodiments, the computing device 104 obtains a fine-tuned speech representation model. The feature is then obtained by applying the fine-tuned speech representation model to the speech data. In some embodiments, the computing device 104 applies the adjusted speech representation model to the speech data to obtain the feature. The adjusted speech representation model is obtained by adjusting the fine-tuned speech representation model when training the speech data evaluation model as a whole using sample speech data and corresponding quality levels. The above examples are only used to describe the present disclosure and are not specific limitations of the present disclosure. Those skilled in the art may use any suitable model method to obtain the features of the speech data. Alternatively or additionally, the contextual representation is related to the text corresponding to the speech data. The training of the model will be described below in conjunction with Figure 4 and Figure 5 Further description.

[0044] At block 204, based on the feature, a first quality level and a second quality level are obtained for the speech data, the first quality level being associated with the first language and the second quality level being associated with the second language. For example, computing device 104 processes the feature to obtain first quality level 108 and second quality level 110 for speech data 102.

[0045] In some embodiments, the computing device 104 obtains a first quality level by applying a trained first decoder to the feature, and obtains a second quality level by applying a trained second decoder to the feature. The trained first decoder and the trained second decoder can be used to process the features of the speech data to obtain corresponding quality levels. The first decoder is trained using speech data related to the first language, and the second decoder is trained using speech data related to the second language. In this way, it is possible to determine the quality level of speech data using different decoders, thereby enabling quality detection of speech data in various languages.

[0046] Alternatively or additionally, when obtaining the quality grade using the first decoder or the second decoder, it is also necessary to input the identification ID of the provider for providing the quality evaluation to obtain the quality grade given by the provider. In the inference use process of the first decoder or the second decoder, the provider is a virtual scorer. Figure 4 and 5 Further description.

[0047] Alternatively or additionally, the computing device 104 also has decoders for other languages. For example, based on the features, the computing device 104 may also obtain a third quality level for the speech data, where the third quality level is related to a third language.

[0048] At block 206, a target quality level for the speech data is determined based on the first quality level and the second quality level, the target quality level indicating a speech quality of the speech data. The computing device 104 determines the target quality level using the first quality level and the second quality level.

[0049] In some embodiments, computing device 104 selects a quality level corresponding to the language of the speech data from the first quality level and the second quality level as the target quality level.

[0050] In some embodiments, if the computing device 104 further calculates a third quality level, then determining the target quality level for the speech data further includes determining the target quality level based on the first quality level, the second quality level, and the third quality level.

[0051] Through this method, the accuracy and efficiency of cross-domain evaluation of voice data quality can be improved, the utilization rate of data is increased, and the user experience is improved.

[0052] Combined with the above Figure 2 A flowchart of a process 200 for determining the quality of speech data according to an embodiment of the present disclosure is described below. Figure 3 A specific example 300 of determining the voice quality of voice data is described. Figure 3 In the described example, the speech quality of the data is represented by a score, and the speech representation model is the wav2vec2 model.

[0053] like Figure 3 As shown, the synthesized speech data 302 is input into the trained wav2vec2 model 304. In one example, the trained wav2vec2 model 304 is obtained by fine-tuning the pre-trained wav2vec2 model using a set of speech data and corresponding text. In another example, after fine-tuning the pre-trained wav2vec2 model, the trained wav2vec2 model 304 needs to be further adjusted using sample speech data and sample quality levels when training the decoder later.

[0054] The pre-trained wav2vec2 model is obtained by self-learning using a large amount of speech data. The pre-trained wav2vec2 consists of three parts: an encoder composed of a convolutional neural network, a context processor composed of a Transformer, and a quantizer. Its input is the original speech signal. The encoder can encode a 25ms segment into a latent vector every 20ms for a speech signal with a sampling rate of 16kHz. The context processor can consider the information from other segments in the entire speech on the current segment and further process the latent vector into a context-related segment representation. The quantizer is only used during wav2vec2 pre-training. The pre-trained wav2vec2 model can be provided by another provider or trained by the user himself using speech data.

[0055] When fine-tuning the pre-trained wav2vec2 model, the pre-trained wav2vec2 retains the parameters of the encoder of the convolutional neural network and the context processor of the Transformer, and adds a fully connected layer with initialized parameters at the end. During fine-tuning, the pre-trained wav2vec2 is trained using sample speech data and the corresponding text, and the connectionist temporal classification CTC loss function is used to adjust the parameters of wav2vec2. After fine-tuning wav2vec2 in a speech recognition manner, the fine-tuned wav2vec2 has the ability to recognize pronunciation, and the determined context representation is related to the text content corresponding to the speech data. When the fine-tuned wav2vec2 is used for inference in this solution, the last fully connected layer added during training is removed.

[0056] The trained wav2vec2 model 304 processes the synthesized speech data 303 to obtain a context vector 306. The obtained context vector 306 is then input into a decoder A 310 and a decoder B 314, respectively. Decoder A 310 is used to score English speech data, and decoder B is used to score Chinese speech data. When scoring, decoder A 310 also needs to input an English virtual scorer ID 308, and decoder B 314 also needs to input a Chinese virtual scorer ID 312. Then decoder A and decoder B obtain the scores of the corresponding virtual scorers, respectively. Then, in a score selection module 316, a score corresponding to the language of the synthesized speech data 303 is selected as a score 318 for evaluating the synthesized speech data 303 according to the language of the synthesized speech data 303. For example, if the synthesized speech data is in English, the score of decoder A 310 is selected, and if the synthesized speech data is in Chinese, the score of decoder B 314 is selected. The above examples are only used to describe the present disclosure, and are not specifically limited to the present disclosure. The number of decoders and the corresponding languages ​​can be set by the user according to actual needs. For example, when training a decoder, multiple decoders can be trained using speech data in multiple languages.

[0057] Through this method, the accuracy and efficiency of cross-domain evaluation of voice data quality can be improved, the utilization rate of data is increased, and the user experience is improved.

[0058] The above describes the reasoning process using the speech representation model and decoder. Figure 4 A schematic diagram of a process 400 for training a decoder and a speech representation model according to an embodiment of the present disclosure is described. The process 400 may be performed by Figure 1 The process may be executed by a computing device in or other suitable computing device.

[0059] The training process is to obtain an adjusted speech representation model, a trained first decoder and a trained second decoder by using sample speech data and sample scores.

[0060] At block 402, the computing device 104 first obtains a fine-tuned speech representation model, a first decoder, and a second decoder. In one example, the fine-tuned speech representation model, the first decoder, and the second decoder are received from other devices. In another example, the fine-tuned speech representation model, the first decoder, and the second decoder are obtained from the computing device 104.

[0061] At box 404, the computing device 104 obtains a first set of sample speech data and corresponding sample quality levels. The computing device obtains the sample quality levels corresponding to the set of sample data. For example, each sample speech data has four sample quality levels, three of which are quality levels provided by real providers, the fourth is a quality level provided by a virtual provider, and the fourth quality level is the average of the other three quality levels. The above examples are only used to describe the present disclosure, and are not specific limitations of the present disclosure. Those skilled in the art can set the number of quality levels and the composition of the quality levels as needed.

[0062] At block 406, computing device 104 then determines a first set of identifiers for providers that provide sample quality levels. These providers include an identifier of a virtual provider. For example, there are four provider identifiers, three of which are real provider identifiers and one of which is a virtual provider identifier.

[0063] At box 408, the computing device 104 obtains the adjusted speech representation model and the trained first decoder or the trained second decoder by jointly training the fine-tuned speech representation model and the first decoder or the second decoder using the first set of sample speech data, the corresponding sample quality levels, and the first set of identifiers.

[0064] In the training process, the computing device 104 obtains sample speech data from a set of sample speech data, and then obtains corresponding features through the fine-tuned speech representation model. The features are then input into a decoder corresponding to the language in the sample speech data. In addition, the decoder also needs to input the identifier of the provider of the sample quality level for the speech data. For example, if there are 4 providers, the decoder will output four scores, and then adjust the fine-tuned speech representation model and the corresponding decoder parameters based on the output four scores and the scores given by the four providers.

[0065] The speech representation model and decoder of joint training fine-tuning are described above. In certain embodiments, the first decoder and the second decoder can be trained separately without training together with the speech representation model of fine-tuning. At this time, the speech representation model of fine-tuning is considered to be a trained speech representation model. The computing device 104 utilizes the speech representation model of fine-tuning to determine the sample features corresponding to the second group of sample voice data. Then the computing device 104 determines the second group of identifications for the provider, and the provider is used to provide the sample quality level. Then the computing device 104 utilizes the sample features, the second group of identifications and the sample quality level to train the first decoder or the second decoder to obtain the trained first decoder or the described trained second decoder.

[0066] In some embodiments, the fine-tuned speech representation model may be trained by the computing device 104 or by other computing devices. When acquiring the fine-tuned speech representation model, first acquire the pre-trained speech representation model. Then acquire the third set of sample speech data and the corresponding sample text. Then, the pre-trained speech representation model is trained with the third set of sample speech data and the corresponding sample text to generate the fine-tuned speech representation model. In this way, a fine-tuned speech representation model related to the text of the speech data can be obtained, thereby improving the accuracy of the evaluation.

[0067] The pre-trained speech representation model may be trained by the computing device 104 or by other computing devices. When generating the pre-trained speech representation model, first obtain the fourth set of sample speech data. Then, the pre-trained speech representation model is generated by training the speech representation model using the fourth set of sample speech data. The speech representation model is a neural network model that converts speech signals into corresponding context representations.

[0068] Alternatively, the above shows the process of training the first encoder and the second encoder on the computing device 104 , and the first encoder and the second encoder may also be trained on other computing devices.

[0069] In some embodiments, the first decoder is a decoder for Chinese, and the second decoder is a decoder for English.

[0070] Combined with the above Figure 4 Describes the example training process, the following combines Figure 5 An example 500 for training a model is described.

[0071] like Figure 5 As shown, when training the speech quality assessment model, the model includes three parts, a fine-tuned wav2vec2, a decoder A 512 and a decoder B 514. At this time, the fine-tuned wav2vec2 removes the last fully connected layer. When training, English synthesized audio data 502 or Chinese synthesized audio data 504 is input into the fine-tuned wav2vec2 model 506, and then a corresponding context vector 508 is generated, and the context vector 508 is input into decoder A 512 or decoder B 514. If the context vector 508 is English synthesized audio data, it enters decoder A 512. At the same time, decoder A 512 also inputs a group of scorer IDs 510 of English synthesized data, one of which is a virtual scorer ID, and the score corresponding to the virtual scorer ID is the average of the scores of other scorers in the group. Then obtain the scores 518 corresponding to different scorers. Then the obtained scores are compared with the sample scores for the speech data to adjust the parameters of the fine-tuned wav2vec2 and decoder A.

[0072] If the context vector 508 is Chinese synthesized audio data, it enters decoder B 514, and decoder B 514 also inputs a group of scorer IDs 516 of Chinese synthesized data, one of which is the ID of a virtual scorer, and the score corresponding to the virtual scorer ID is the average of the scores of other scorers in the group. Then, scores corresponding to different scorers are obtained 520. The obtained scores are then compared with the sample scores for the speech data to adjust the parameters of the fine-tuned wav2vec2 model 506 and decoder B 514.

[0073] Figure 5 The method of using the sample speech data and the sample score to adjust the fine-tuned wav2vec2, decoder A 512 and decoder B 514 at the same time is described. In some embodiments, the parameters of the fine-tuned wav2vec2 may not be adjusted, and the fine-tuned wav2vec2 is used to generate a context vector for the sample speech data during training, and then the parameters of decoder A and decoder B are adjusted in combination with the sample score to achieve training of decoders A and B.

[0074] Figure 6 FIG. 2 shows a schematic block diagram of an apparatus for determining a speaker in a text according to an embodiment of the present disclosure. Figure 6 As shown, the device 600 includes a feature determination module 602, which is configured to determine the features of the acquired speech data; a quality level acquisition module 604, which is configured to acquire a first quality level and a second quality level for the speech data based on the features, the first quality level being related to the first language, and the second quality level being related to the second language; and a target quality level determination module 606, which is configured to determine a target quality level for the speech data based on the first quality level and the second quality level, the target quality level indicating the speech quality of the speech data.

[0075] In some embodiments, the quality level acquisition module 604 includes: a first quality level acquisition module configured to acquire a first quality level by applying a trained first decoder to the features; and a second quality level acquisition module configured to acquire a second quality level by applying a trained second decoder to the features.

[0076] In some embodiments, the feature determination module includes: a first feature acquisition module, configured to apply the adjusted speech representation model to the speech data to obtain features.

[0077] In some embodiments, it also includes a joint training module, which is configured to obtain an adjusted speech representation model, a trained first decoder and a trained second decoder through the following modules: a model and decoder acquisition module, which is configured to obtain a fine-tuned speech representation model, a first decoder and a second decoder; a sample quality level acquisition module, which is configured to obtain a first group of sample speech data and corresponding sample quality levels; a first group identifier determination module, which is configured to determine a first group of identifiers for a provider, and the provider is used to provide the sample quality level; a training acquisition module, which is configured to obtain the adjusted speech representation model and the trained first decoder or the trained second decoder by jointly training the fine-tuned speech representation model and the first decoder or the second decoder using the first group of sample speech data, the corresponding sample quality level and the first group of identifiers.

[0078] In some embodiments, the feature determination module includes: a fine-tuned speech representation model acquisition module, configured to acquire a fine-tuned speech representation model; and a second feature acquisition module, configured to obtain the feature by applying the fine-tuned speech representation model to the speech data.

[0079] In some embodiments, the decoder acquisition module is configured to also include acquiring the trained first decoder and the trained second decoder through the following modules: a sample feature determination module is configured to determine sample features corresponding to a second group of sample speech data; a second group identifier determination module is configured to determine a second group identifier for a provider, the provider being used to provide the sample quality level; a decoder training module is configured to use the sample features, the second group identifier and the sample quality level to train the first decoder or the second decoder to obtain the trained first decoder or the trained second decoder.

[0080] In some embodiments, obtaining the fine-tuned speech representation model includes: a pre-trained model acquisition module, configured to obtain a pre-trained speech representation model; a corresponding sample text acquisition module, configured to obtain a third set of sample speech data and corresponding sample text; and a first generation module, configured to generate the fine-tuned speech representation model by training the pre-trained speech representation model using the third set of sample speech data and the corresponding sample text.

[0081] In some embodiments, the pre-trained model acquisition module includes: a fourth group of sample speech data acquisition module, configured to acquire a fourth group of sample speech data; and a pre-trained speech representation model generation module, configured to generate the pre-trained speech representation model by training the speech representation model using the fourth group of sample speech data.

[0082] In some embodiments, the speech representation model is a neural network model that converts speech signals into corresponding features.

[0083] In some embodiments, the feature is related to text corresponding to the speech data.

[0084] In some embodiments, the device 600 also includes: a third quality level acquisition module, configured to acquire a third quality level for the speech data based on the feature, wherein the third quality level is related to a third language; and the target quality level determination module also includes: a determination module based on three quality levels, configured to determine the target quality level based on the first quality level, the second quality level and the third quality level.

[0085] In some embodiments, the target quality level determination module includes: a selection module configured to select a quality level corresponding to the language of the speech data from the first quality level and the second quality level as the target quality level.

[0086] In some embodiments, the device 600 also includes: a voice data generation module, configured to generate the voice data through voice synthesis or voice conversion.

[0087] Figure 7 A schematic block diagram of an example device 700 that may be used to implement embodiments of the present disclosure is shown. Figure 1 The computing device 104 in the embodiment of the present invention can be implemented by using the device 700. As shown in the figure, the device 700 includes a central processing unit (CPU) 701, which can perform various appropriate actions and processes according to computer program instructions stored in a read-only memory (ROM) 702 or computer program instructions loaded from a storage unit 708 into a random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the device 700 can also be stored. The CPU 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0088] A number of components in the device 700 are connected to the I / O interface 705, including: an input unit 706, such as a keyboard, a mouse, etc.; an output unit 707, such as various types of displays, speakers, etc.; a storage page 708, such as a disk, an optical disk, etc.; and a communication unit 709, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 709 allows the device 700 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0089] The various processes and processing described above, such as methods 200 and 400, may be performed by the processing unit 701. For example, in some embodiments, the methods 200 and 400 may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as the storage unit 708. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 700 via the ROM 702 and / or the communication unit 709. When the computer program is loaded into the RAM 703 and executed by the CPU 701, one or more actions of the methods 200 and 400 described above may be performed.

[0090] The present disclosure may be a method, an apparatus, a system and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for executing various aspects of the present disclosure.

[0091] Computer readable storage medium can be a tangible device that can hold and store instructions used by an instruction execution device. Computer readable storage medium can be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the above. More specific examples (non-exhaustive list) of computer readable storage medium include: portable computer disk, hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disk read-only memory (CD-ROM), digital versatile disk (DVD), memory stick, floppy disk, mechanical encoding device, such as a punch card or a convex structure in a groove on which instructions are stored, and any suitable combination of the above. The computer readable storage medium used here is not interpreted as a transient signal itself, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagated by a waveguide or other transmission medium (e.g., a light pulse by an optical fiber cable), or an electrical signal transmitted by a wire.

[0092] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0093] The computer program instructions for performing the operation of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be customized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.

[0094] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0095] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processing unit of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0096] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0097] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of the module, program segment or instruction includes one or more executable instructions for realizing the specified logical function. In some alternative implementations, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or action, or can be implemented with a combination of special hardware and computer instructions.

[0098] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A method for determining the quality of speech data, comprising: determining characteristics of the acquired speech data; Based on the feature, obtaining a first quality level and a second quality level for the speech data, wherein the first quality level is related to a first language and the second quality level is related to a second language; and determining a target quality level for the voice data based on the first quality level and the second quality level, the target quality level indicating a voice quality of the voice data; Wherein determining the target quality level comprises: A quality level corresponding to the language of the speech data is selected from the first quality level and the second quality level as the target quality level.

2. The method according to claim 1, acquiring the first quality level and the second quality level comprises: obtaining the first quality level by applying a trained first decoder to the features; as well as The second quality level is obtained by applying a trained second decoder to the features.

3. The method of claim 2, wherein the determining the characteristics of the acquired speech data comprises: The features are obtained by applying the adjusted speech representation model to the speech data.

4. The method of claim 3, further comprising: Obtaining a fine-tuned speech representation model, a first decoder, and a second decoder; Obtaining a first set of sample speech data and corresponding sample quality levels; determining a first set of identifiers for providers for providing the sample quality levels; The adjusted speech representation model and the trained first decoder or the trained second decoder are obtained by training the fine-tuned speech representation model and the first decoder or the second decoder using the first set of sample speech data, the corresponding sample quality levels and the first set of identifiers.

5. The method of claim 2, wherein determining the characteristics of the acquired speech data comprises: Obtain a fine-tuned speech representation model; The features are obtained by applying the fine-tuned speech representation model to the speech data.

6. The method according to claim 5, further comprising: determining sample features corresponding to the second set of sample speech data; determining a second set of identifiers for providers for providing the sample quality levels; The first decoder or the second decoder is trained by using the sample features, the second group identifier and the sample quality level to obtain the trained first decoder or the trained second decoder.

7. The method according to claim 4 or 5, wherein obtaining the fine-tuned speech representation model comprises: Get a pre-trained speech representation model; Obtaining a third set of sample voice data and corresponding sample text; as well as The fine-tuned speech representation model is generated by training the pre-trained speech representation model using the third set of sample speech data and the corresponding sample text.

8. The method according to claim 7, wherein obtaining the pre-trained speech representation model comprises: Obtaining a fourth set of sample voice data; as well as The pre-trained speech representation model is generated by training the speech representation model using the fourth set of sample speech data.

9. The method according to claim 8, wherein the speech representation model is a neural network model that converts speech signals into corresponding features.

10. The method of claim 1, wherein the feature is related to text corresponding to the speech data.

11. The method according to claim 1, further comprising: Based on the feature, obtaining a third quality level for the speech data, the third quality level being related to a third language; and Determining a target quality level for the voice data further includes: The target quality level is determined based on the first quality level, the second quality level, and the third quality level.

12. The method according to claim 1, further comprising: The voice data is generated by voice synthesis or voice conversion.

13. An apparatus for determining the quality of speech data, comprising: A feature determination module, configured to determine features of the acquired speech data; a quality level acquisition module, configured to acquire a first quality level and a second quality level for the speech data based on the feature, wherein the first quality level is related to a first language and the second quality level is related to a second language; as well as a target quality level determination module, configured to determine a target quality level for the voice data based on the first quality level and the second quality level, the target quality level indicating the voice quality of the voice data; The target quality level determination module includes: a selection module configured to select a quality level corresponding to the language of the speech data from the first quality level and the second quality level as the target quality level.

14. An electronic device comprising: at least one processor; as well as A storage device, used for storing at least one program, when the at least one program is executed by the at least one processor, so that the at least one processor implements the method according to any one of claims 1-12.

15. A computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the method according to any one of claims 1 to 12 when executed by a processor.

Images