A method, device, storage medium and equipment for processing voice data

Abstract

The present application discloses a method, apparatus, storage medium and equipment for processing speech data, the method comprising: firstly obtaining target speech data to be processed, and performing K-layer unsupervised clustering processing on the target speech data to obtain clustered target speech data; then performing uncertainty sampling on the clustered target speech data, selecting a first preset number of target speech data from each type of target speech data, then using the first preset number of target speech data as sample data, training a speech processing model, and calculating the uncertainty of each sample data during the training process, so as to update the model in a direction in which the uncertainty of the model is reduced, and obtaining a model that has learned the information of the target speech data, and then using the model to repeatedly perform clustering processing and uncertainty sampling on the target speech data until a second preset number of target speech data is selected from each type of target speech data, thereby improving the processing effect of the speech data.

Description

Technical Field

[0001] The present application relates to the field of natural language processing technology, and in particular to a method, apparatus, storage medium and device for processing speech data. Background Art

[0002] With the continuous breakthroughs in deep learning technology, supervised learning has developed rapidly in the field of speech data processing. However, supervised learning requires a large amount of manually labeled data to ensure the generalization effect of the speech processing model, which is costly. Therefore, active learning solutions have emerged, which use unsupervised methods to select more valuable data from the speech data to be labeled for manual labeling, thereby achieving the goal of completing speech processing model training with only a small amount of labeled data.

[0003] At present, there are usually two commonly used active learning schemes: one is a speech data processing scheme based on an additional model, but this method requires additional training of a data selection model, which increases training cost and difficulty. In addition, the additional data selection model will have different training performance for different speech processing tasks, making it difficult to achieve stable universality; the other commonly used active learning scheme is a speech data selection scheme based on posterior statistics. The disadvantage of this method is that it does not take into account the balance of speech data. For example, if the current model performs poorly for a certain type of sample data, then simply selecting by entropy value may result in the selected data being all of this category, resulting in an imbalance in the selected training set, which in turn affects the effect of continued training of the speech processing model. Summary of the Invention

[0004] The main purpose of the embodiments of the present application is to provide a speech data processing method, apparatus, storage medium and equipment, which can achieve the purpose of selecting more valuable data for training and reducing manual labeling costs without increasing the training cost of the speech processing model, thereby improving the processing effect of speech data.

[0005] The present invention provides a method for processing speech data, including:

[0006] Obtain target voice data to be processed;

[0007] Performing K-layer unsupervised clustering processing on the target speech data to obtain clustered target speech data, wherein K is a positive integer greater than 0;

[0008] performing uncertainty sampling on the clustered target speech data, and selecting a first preset number of target speech data from each type of target speech data;

[0009] Training a speech processing model using the first preset amount of target speech data as sample data, and calculating uncertainty of each sample data during the training process so as to update the model in a direction of reducing model uncertainty, thereby obtaining a speech processing model that has learned information about the target speech data;

[0010] Using the speech processing model that has learned the information of the target speech data, repeatedly perform K-layer unsupervised clustering processing and subsequent steps on the target speech data until a second preset number of target speech data are selected from each type of target speech data to obtain the processing result of the target speech data.

[0011] In one possible implementation, K is 2; and performing K-layer unsupervised clustering on the target speech data to obtain clustered target speech data includes:

[0012] Using a pre-trained model corresponding to the first speech processing task, extracting features from the target speech data to obtain a feature vector of the target speech data;

[0013] Using a preset unsupervised clustering algorithm, clustering the feature vectors of the target speech data to obtain N types of target speech data, where N is a positive integer greater than 0;

[0014] Using a pre-trained model or a preset unsupervised clustering algorithm corresponding to the second speech processing task, the target speech data of each category in the N categories of target speech data are clustered again into M categories of target speech data to obtain NM categories of target speech data; M is a positive integer greater than 0; the second speech processing task and the first speech processing task are similar tasks or related tasks.

[0015] In one possible implementation, when the target speech data includes speech data sequences of different lengths, the step of extracting features from the target speech data using a pre-trained model corresponding to the first speech processing task to obtain a feature vector of the target speech data includes:

[0016] Using a pre-trained model corresponding to the first speech processing task, feature extraction is performed on speech data sequences of different lengths in the target speech data to obtain feature vectors corresponding to the speech data sequences of different lengths;

[0017] The average value of the feature vectors corresponding to the speech data sequences of different lengths is calculated as the feature vector of the target speech data.

[0018] In a possible implementation, the first speech processing task is a speech recognition task; and the second speech processing task is a speaker classification task.

[0019] In a possible implementation, performing uncertainty sampling on the clustered target speech data and selecting a first preset number of target speech data from each type of target speech data includes:

[0020] Calculating the entropy value of the feature vector of the clustered target speech data;

[0021] The target speech data in each category of target speech data are arranged from large to small according to the entropy value, and a first preset number of target speech data ranked at the top are selected from the arranged target speech data in each category.

[0022] In a possible implementation, performing uncertainty sampling on the clustered target speech data and selecting a first preset number of target speech data from each type of target speech data includes:

[0023] The clustered target speech data are subjected to minimum confidence sampling or edge sampling, and a first preset number of target speech data are selected from each type of target speech data.

[0024] In one possible implementation, the speech processing model is trained using the first preset amount of target speech data as sample data, and uncertainty of each sample data is calculated during the training process to update the model in a direction of reducing model uncertainty, thereby obtaining a speech processing model that has learned information about the target speech data, including:

[0025] Training a speech processing model using the first preset amount of target speech data as sample data, and during the training process, calculating the uncertainty of each sample data using a given objective function so as to update the model in a direction that reduces the model uncertainty, thereby obtaining a speech processing model that has learned information about the target speech data;

[0026] The objective function is used to perform entropy sampling on the sample data.

[0027] The present invention also provides a voice data processing device, including:

[0028] An acquisition unit, configured to acquire target speech data to be processed;

[0029] A clustering unit, configured to perform K-layer unsupervised clustering processing on the target speech data to obtain clustered target speech data; wherein K is a positive integer greater than 0;

[0030] a sampling unit, configured to perform uncertainty sampling on the clustered target speech data, and select a first preset number of target speech data from each type of target speech data;

[0031] a training unit, configured to train a speech processing model using the first preset amount of target speech data as sample data, and calculate uncertainty of each sample data during the training process so as to update the model in a direction of reducing the model uncertainty, thereby obtaining a speech processing model that has learned information about the target speech data;

[0032] A processing unit is used to use the speech processing model that has learned the information of the target speech data to repeatedly perform K-layer unsupervised clustering processing and subsequent steps on the target speech data until a second preset number of target speech data is selected from each type of target speech data to obtain a processing result of the target speech data.

[0033] In one possible implementation, K is 2; and the clustering unit includes:

[0034] an extraction subunit, configured to perform feature extraction on the target speech data using a pre-trained model corresponding to the first speech processing task, to obtain a feature vector of the target speech data;

[0035] A first clustering subunit is configured to perform clustering processing on the feature vectors of the target speech data using a preset unsupervised clustering algorithm to obtain N types of target speech data, where N is a positive integer greater than 0;

[0036] The second clustering subunit is used to use the pre-trained model or preset unsupervised clustering algorithm corresponding to the second speech processing task to cluster the target speech data of each category in the N categories of target speech data into M categories of target speech data again to obtain NM categories of target speech data; M is a positive integer greater than 0; the second speech processing task and the first speech processing task are similar tasks or related tasks.

[0037] In a possible implementation, when the target speech data includes speech data sequences of different lengths, the extraction subunit includes:

[0038] an obtaining subunit, configured to perform feature extraction on speech data sequences of different lengths in the target speech data using a pre-trained model corresponding to the first speech processing task, and obtain feature vectors corresponding to the speech data sequences of different lengths;

[0039] The first calculation subunit is configured to calculate an average value of the feature vectors corresponding to the speech data sequences of different lengths as the feature vector of the target speech data.

[0040] In a possible implementation, the first speech processing task is a speech recognition task; and the second speech processing task is a speaker classification task.

[0041] In a possible implementation, the sampling unit includes:

[0042] A second calculation subunit, configured to calculate an entropy value of a feature vector of the clustered target speech data;

[0043] The selection subunit is used to arrange the target speech data in each category of target speech data from large to small according to the entropy value, and select a first preset number of target speech data with a top ranking from each category of arranged target speech data.

[0044] In a possible implementation, the sampling unit is specifically configured to:

[0045] The clustered target speech data are subjected to minimum confidence sampling or edge sampling, and a first preset number of target speech data are selected from each type of target speech data.

[0046] In one possible implementation, the training unit is specifically configured to:

[0047] Training a speech processing model using the first preset amount of target speech data as sample data, and during the training process, calculating the uncertainty of each sample data using a given objective function so as to update the model in a direction that reduces the model uncertainty, thereby obtaining a speech processing model that has learned information about the target speech data;

[0048] The objective function is used to perform entropy sampling on the sample data.

[0049] The embodiment of the present application further provides a voice data processing device, comprising: a processor, a memory, and a system bus;

[0050] The processor and the memory are connected via the system bus;

[0051] The memory is used to store one or more programs, and the one or more programs include instructions. When the instructions are executed by the processor, the processor executes any one of the implementations of the above-mentioned voice data processing method.

[0052] An embodiment of the present application also provides a computer-readable storage medium, which stores instructions. When the instructions are executed on a terminal device, the terminal device executes any one of the implementations of the above-mentioned voice data processing method.

[0053] An embodiment of the present application further provides a computer program product, which, when executed on a terminal device, enables the terminal device to execute any one of the implementations of the above-mentioned voice data processing method.

[0054] The embodiments of the present application provide a speech data processing method, apparatus, storage medium, and device, which first obtain target speech data to be processed and perform K-layer unsupervised clustering processing on the target speech data to obtain clustered target speech data; wherein K is a positive integer greater than 0; then uncertainty sampling is performed on the clustered target speech data, and a first preset number of target speech data is selected from each type of target speech data. Subsequently, the first preset number of target speech data is used as sample data to train a speech processing model, and the uncertainty of each sample data is calculated during the training process so as to update the model in a direction of reducing the uncertainty of the model, thereby obtaining a speech processing model that has learned the information of the target speech data. The speech processing model can then be used to repeatedly perform K-layer unsupervised clustering processing and uncertainty sampling on the target speech data until a second preset number of target speech data is selected from each type of target speech data to obtain a processing result of the target speech data.

[0055] It can be seen that since the embodiment of the present application first ensures the balance of the target speech data through unsupervised clustering, and then selects the target speech data with greater uncertainty through uncertainty sampling for training the speech processing model, it can achieve the purpose of selecting more valuable data for training and reducing the cost of manual labeling without increasing the training cost of the speech processing model. In addition, by updating the model in the direction of reducing uncertainty during the model training process, the model can also quickly learn the information of the type of speech data to which the sample data belongs, which facilitates the speech data processing again, thereby improving the processing effect of the speech data. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0057] Figure 1 A flowchart of a method for processing voice data provided in an embodiment of the present application;

[0058] Figure 2 A schematic diagram of the composition of a voice data processing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0059] With the rapid development of deep learning technology, voice data processing is increasingly required in more and more scenarios. In recent years, active learning methods have become increasingly popular for voice data processing. This involves manually labeling the most valuable data from the unlabeled data using an unsupervised approach, thereby enabling voice processing model training with only a small amount of labeled data.

[0060] Currently, existing active learning speech data processing methods generally include the following two types:

[0061] The first approach involves processing speech data based on an additive model. This approach primarily involves attaching a trainable model to trained and untrained speech data to identify differences between the two. This additional model is then used to identify data from the speech data to be labeled that is closest to the untrained data and manually annotate it. However, this approach requires the training of an additional data selection model, which increases training cost and difficulty. Furthermore, the additional data selection model will have varying training performance for different speech processing tasks, making it difficult to achieve stable universality.

[0062] The second is a speech data selection scheme based on posterior statistics. This scheme starts from a specific speech processing task, first loads a pre-trained model, and then uses this pre-trained model to calculate the posterior probability of all data to be labeled, and manually labels the data with greater uncertainty in the posterior probability. However, this method mainly counts the uncertainty of the current speech processing task model in treating the labeled data. The most common scheme is the posterior probability information entropy of the statistical data. The larger the entropy value, the higher the uncertainty of the model about the data, and the more likely the data should be selected. However, the disadvantage of this scheme is that it does not take into account the balance of speech data. For example, if the current model performs poorly on a certain type of sample data, then simply selecting by entropy value may result in the selected data being all of this category, which makes the selected training set unbalanced, and in turn affects the effect of continued training of the speech processing model.

[0063] Therefore, how to improve the processing effect of speech data without increasing the training cost of the speech processing model is a technical problem that needs to be solved urgently.

[0064] To address the above-mentioned defects, the present application provides a speech data processing method, which first obtains the target speech data to be processed and performs K-layer unsupervised clustering processing on it to obtain the clustered target speech data; wherein K is a positive integer greater than 0; then uncertainty sampling is performed on the clustered target speech data, and a preset number of target speech data are selected from each type of target speech data. Subsequently, the preset number of target speech data is used as sample data to train a speech processing model, and the uncertainty of each sample data is calculated during the training process so as to update the model in the direction of reducing the model uncertainty, thereby obtaining a speech processing model that has learned the information of the target speech data. The speech processing model can then be used to repeatedly perform K-layer unsupervised clustering processing and uncertainty sampling on the target speech data until a second preset number of target speech data is selected from each type of target speech data to obtain the processing result of the target speech data.

[0065] It can be seen that since the embodiment of the present application first ensures the balance of the target speech data through unsupervised clustering, and then selects the target speech data with greater uncertainty through uncertainty sampling for training the speech processing model, it can achieve the purpose of selecting more valuable data for training and reducing the cost of manual labeling without increasing the training cost of the speech processing model. In addition, by updating the model in the direction of reducing uncertainty during the model training process, the model can also quickly learn the information of the type of speech data to which the sample data belongs, which facilitates the speech data processing again, thereby improving the processing effect of the speech data.

[0066] To make the purpose, technical solutions, and advantages of the embodiments of this application more clear, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0067] First embodiment

[0068] See also Figure 1 , is a flow chart of a voice data processing method provided in this embodiment, the method comprising the following steps:

[0069] S101: Acquire target speech data to be processed.

[0070] In this embodiment, any speech data processed by this embodiment is defined as target speech data. Furthermore, this embodiment does not limit the language type of the target speech data; for example, the target speech data can be Chinese speech data or an English audiobook. Furthermore, this embodiment does not limit the length of the target speech data; for example, the target speech data can be one or two paragraphs.

[0071] It is understandable that the target voice data can be obtained through recording or other means according to actual needs. For example, the telephone call voice data in people's daily life, or the conference recording data, etc. can all be used as target voice data. After obtaining the target voice data, this embodiment is used to process the target voice data to select valuable data for manual labeling.

[0072] S102: Performing K-layer unsupervised clustering processing on the target speech data to obtain clustered target speech data; wherein K is a positive integer greater than 0.

[0073] In this embodiment, after obtaining the target speech data to be processed through step S101, in order to improve the processing effect of the target speech data, the target speech data can be further subjected to K (K is a positive integer greater than 0) layers of unsupervised clustering processing to obtain the clustered target speech data, which is used to execute the subsequent step S103 to ensure the balance of the target speech data selected as sample data in the subsequent steps.

[0074] In one optional implementation, K may be set to 2, and the implementation steps of step S102 may include the following steps AC:

[0075] Step A: Using the pre-trained model corresponding to the first speech processing task, extract features from the target speech data to obtain a feature vector of the target speech data.

[0076] In this implementation, after obtaining the target speech data to be processed, it is first necessary to frame the target speech data to obtain a corresponding speech frame sequence, and then pre-emphasize the framed speech frame sequence; and then use the pre-trained model corresponding to the first speech processing task to extract the acoustic features of each speech frame in turn (for example, it can be a high-level representative feature vector such as the feature vector calculated by the extracted penultimate layer of the network), wherein the acoustic feature refers to the feature data used to characterize the voiceprint information of the corresponding speech frame, for example, it can be a Mel-scale Frequency Cepstral Coefficient (MFCC) feature or a Log Mel filter bank (FBANK) feature, etc., as the feature vector of each speech frame sequence in the target speech data, for executing the subsequent step B.

[0077] The first speech processing task may be a task that requires the use of processed target speech data, such as a speech recognition task that requires the use of processed target speech data for speech recognition. Accordingly, the pre-trained model corresponding to the first speech processing task refers to a speech recognition model trained using labeled general data, or an initial speech recognition model that has only undergone parameter initialization, such as a convolutional neural network (CNN) or a recurrent neural network (RNN).

[0078] An optional implementation method is that when the target voice data packet contains voice data sequences of different lengths, the sequence lengths of the acoustic features extracted from the voice data sequences (voice data or video data) of different lengths (such as the feature vectors calculated by the penultimate layer of the network) are also inconsistent. At this time, in order to execute the subsequent step B to implement clustering processing of the target voice data, the pre-trained model corresponding to the first voice processing task is used to extract features of the voice data sequences of different lengths in the target voice data respectively. After obtaining the feature vectors corresponding to the voice data sequences of different lengths, it is further necessary to calculate the average value of the feature vectors corresponding to the voice data sequences of different lengths as the feature vector of the entire target voice data for executing the subsequent step B.

[0079] Step B: Using a preset unsupervised clustering algorithm, cluster the feature vectors of the target speech data to obtain N types of target speech data; wherein N is a positive integer greater than 0.

[0080] In this implementation, after obtaining the feature vector of the target speech data through step A, the preset unsupervised clustering algorithm can be further used to cluster the feature vector of the target speech data to obtain N (N is a positive integer greater than 0) types of target speech data, that is, the feature vectors of all target speech data are divided into N categories.

[0081] Among them, the preset unsupervised clustering algorithm can be any existing or future unsupervised clustering algorithm, which can be selected according to actual conditions. The embodiment of this application is not limited. For example, the preset unsupervised clustering algorithm can be selected as K-means clustering or hierarchical clustering.

[0082] Step C: Using the pre-trained model or preset unsupervised clustering algorithm corresponding to the second speech processing task, the target speech data of each category in the N categories of target speech data are clustered again into M categories of target speech data to obtain NM categories of target speech data; wherein M is a positive integer greater than 0; the second speech processing task and the first speech processing task are similar tasks or related tasks.

[0083] In this implementation, after clustering the feature vectors of the target speech data through step B to obtain N types of target speech data, the pre-trained model corresponding to the second speech processing task or the preset unsupervised clustering algorithm can be further used. After extracting the feature vectors of each target speech data in each type of target speech data in turn, the feature vectors of each target speech data are clustered again, and each type of target speech data in the N types of target speech data is clustered into M (M is a positive integer greater than 0) types of target speech data, that is, all target speech data are divided into N*M types, and NM types of target speech data are obtained.

[0084] Among them, the second speech processing task can be a similar or related task to the first speech processing task. For example, if the first speech processing task is a speech recognition task, then the second speech processing task can be a speaker classification task accordingly. The feature vector of the extracted target speech data is the speaker-related feature (such as i-vector or x-vector), and the pre-trained model corresponding to the second speech processing task should also refer to a speaker classification model trained using labeled general data, or an initial speaker classification model that has only been initialized with parameters, etc.

[0085] S103: performing uncertainty sampling on the clustered target speech data, and selecting a preset number of target speech data from each type of target speech data.

[0086] In this embodiment, after obtaining the clustered target speech data through step S102, the uncertainty sampling method that exists or will appear in the future can be further used to perform uncertainty sampling on the clustered target speech data. For example, for each clustered target speech data, a speech recognition model can be used to directly calculate the posterior probability of the model output and perform uncertainty sampling. The greater the uncertainty, the more valuable the corresponding target speech data is to be selected for annotation training. Then, the target speech data in each type of target speech data can be sorted according to uncertainty, and the first preset number of target speech data with the highest ranking can be selected from each type of target speech data, thereby ensuring the balance of the overall target speech data. At the same time, within each category, the target speech data with the highest uncertainty ranking can also be selected, thereby also ensuring the annotation value of the target speech data within each category. Among them, the specific value of the first preset number can be set according to actual conditions, and the embodiment of the present application does not limit this.

[0087] Specifically, an optional implementation method is that the implementation process of this step S103 can specifically include: first calculating the entropy value of the feature vector of the clustered target speech data, and then arranging the target speech data in each category of target speech data from large to small according to the entropy value, and selecting the first preset number of target speech data with the highest ranking from each category of arranged target speech data.

[0088] In this implementation, after obtaining the clustered target speech data, uncertainty sampling can be further performed on the clustered target speech data. For example, for each clustered target speech data, a speech recognition model can be used to directly calculate the posterior probability of the model output and perform entropy sampling. The sampling formula is as follows:

[0089]

[0090] Among them, the above formula (1) takes into account the probability information entropy of all categories of x predicted by the speech recognition model, Indicates the target speech data with the largest information entropy.

[0091] On this basis, for the obtained NM types of target speech data, the target speech data in each type of target speech data can be arranged from large to small according to the entropy value, and a first preset number of target speech data with the highest ranking can be selected from each type of target speech data after arrangement. In this way, by selecting only a portion of the data (i.e., the first preset number of target speech data) for annotation training, the same effect as performing annotation training on all the data can be achieved, thereby significantly reducing the pressure of data annotation.

[0092] Another optional implementation is that the implementation process of step S103 may specifically include: performing minimum confidence sampling or edge sampling on the clustered target speech data, and selecting a first preset number of target speech data from each type of target speech data. The formula for minimum confidence sampling is as follows:

[0093]

[0094] in, Indicates the category of the target speech data for which the model predicts the maximum probability of x; Indicates the target speech data with the minimum posterior probability of the aforementioned category.

[0095] The formula for edge sampling is as follows:

[0096]

[0097] in, and They represent the category of the target speech data with the highest probability predicted by the model and the category of the target speech data with the second highest probability, respectively; Indicates the target speech data with the smallest difference in the probabilities of the two aforementioned categories.

[0098] S104: Use a first preset amount of target speech data as sample data to train the speech processing model, and calculate the uncertainty of each sample data during the training process so as to update the model in a direction of reducing the model uncertainty, thereby obtaining a speech processing model that has learned the information of the target speech data.

[0099] In this embodiment, after selecting a first preset number of target speech data from each type of target speech data through step S103, the first preset number of target speech data can be further used as sample data to train a speech processing model (such as a speech recognition model, etc.), and the uncertainty of each sample data is calculated during the training process, so as to update the model in the direction of reducing the model uncertainty, and obtain a speech processing model that has learned the information of the target speech data, which is used to perform the subsequent step S105.

[0100] Specifically, one optional implementation method is that since the first preset number of target speech data (i.e., sample data) selected are all data with relatively high uncertainty, in this case, a pre-set objective function can be used during the training process to calculate the uncertainty of each sample data, so as to update the model in the direction of reducing the model uncertainty, so that the model's judgment on the sample data changes from uncertainty to certainty. This can quickly improve the model's ability to learn the target speech data of the category to which the sample data belongs, so that different types of data can be selected in the subsequent process of selecting sample data.

[0101] The preset objective function may be set according to actual conditions, and the present embodiment does not limit this. For example, the objective function may be used to perform entropy sampling on the sample data. The specific formula is as follows:

[0102] L=arg min x -∑ i P θ (y i |x)*lnP θ (y i |x) (4)

[0103] It can be seen that when the value of the objective function L no longer decreases or remains substantially unchanged, the training of the speech processing model is completed, and a speech processing model that has learned the information of the target speech data is obtained.

[0104] S105: Using the speech processing model that has learned the information of the target speech data, repeatedly perform K-layer unsupervised clustering processing and subsequent steps on the target speech data until a second preset number of target speech data are selected from each type of target speech data to obtain the processing results of the target speech data.

[0105] In this embodiment, after obtaining the speech processing model that has learned the information of the target speech data through step S104, the above steps S102-S103 can be further repeated until a second preset number of target speech data are selected from each type of target speech data, and the selected target speech data can be collectively used as the processing result of the target speech data.

[0106] Next, to facilitate understanding of the voice data processing method provided in the embodiments of the present application, this application will provide a detailed introduction through specific examples.

[0107] For example: After obtaining 10,000 target speech data to be labeled, a speech recognition model can be trained in advance using the existing labeled general data. Since the application scenarios of the general data and the current target speech data to be labeled are inconsistent, directly using this speech recognition model cannot achieve a good recognition effect. The labeling budget is 2,000 target speech data, that is, 2,000 target speech data need to be selected from the 10,000 target speech data for manual labeling. At this point, the 10,000 target speech data can be processed through the above steps S101-S105, and 2,000 target speech data can be selected from them. The specific processing process is as follows:

[0108] First, a speech recognition model is trained using existing labeled general data to perform feature extraction on 10,000 target speech data. That is, the feature vector of the second-to-last layer of the model is extracted, and the average value of the feature vector for all speech frames is calculated as the feature vector of these 10,000 target speech data. K-means is then used to cluster them into 20 categories.

[0109] Then, a speaker classification model trained with general data is used to extract speaker-related i-vector vectors, and the clustered 20 categories of target speech data sets are clustered again. That is, each category of target speech data set is clustered into 10 categories using i-vector vectors using K-means. In this way, the 10,000 target speech data can be finally clustered into 200 (i.e., 20*10=200) categories of target speech data.

[0110] Next, uncertainty sampling is performed on these 200 categories of target speech data. That is, the target speech data in each category are sampled in descending order according to the entropy value corresponding to their feature vectors, and the target speech data in each category are sampled with the maximum entropy value in turn, and the sampling is performed in a round-robin manner until 1,000 target speech data are sampled.

[0111] Furthermore, the obtained 1,000 target speech data can be used as sample data for manual labeling, and the speech recognition model can be trained. During the training process, the uncertainty of each sample data can be calculated so that the model can be updated in the direction of reducing the model uncertainty, and a speech recognition model that has learned the information of these 1,000 target speech data can be obtained.

[0112] Finally, the trained speech recognition model can be used to repeatedly perform the clustering and uncertainty adoption steps until 1,000 more target speech data items are sampled. At this point, the total of 2,000 sampled target speech data items can be used as the selected 2,000 target speech data items. The specific implementation process can be found in the description of steps S101-S105 above and will not be repeated here.

[0113] In summary, the present embodiment provides a speech data processing method, which first obtains the target speech data to be processed and performs K-layer unsupervised clustering processing on it to obtain the clustered target speech data; wherein K is a positive integer greater than 0; then uncertainty sampling is performed on the clustered target speech data, and a first preset number of target speech data is selected from each type of target speech data. Then, the first preset number of target speech data is used as sample data to train the speech processing model, and the uncertainty of each sample data is calculated during the training process, so as to update the model in the direction of reducing the model uncertainty, thereby obtaining a speech processing model that has learned the information of the target speech data. The speech processing model can then be used to repeatedly perform K-layer unsupervised clustering processing and uncertainty sampling on the target speech data until a second preset number of target speech data is selected from each type of target speech data to obtain the processing result of the target speech data.

[0114] It can be seen that since the embodiment of the present application first ensures the balance of the target speech data through unsupervised clustering, and then selects the target speech data with greater uncertainty through uncertainty sampling for training the speech processing model, it can achieve the purpose of selecting more valuable data for training and reducing the cost of manual labeling without increasing the training cost of the speech processing model. In addition, by updating the model in the direction of reducing uncertainty during the model training process, the model can also quickly learn the information of the type of speech data to which the sample data belongs, which facilitates the speech data processing again, thereby improving the processing effect of the speech data.

[0115] Second embodiment

[0116] This embodiment will introduce a voice data processing device. For related content, please refer to the above method embodiment.

[0117] See also Figure 2 , is a schematic diagram of the composition of a voice data processing device provided in this embodiment, the device 200 includes:

[0118] An acquisition unit 201 is configured to acquire target speech data to be processed;

[0119] The clustering unit 202 is configured to perform K-layer unsupervised clustering processing on the target speech data to obtain clustered target speech data; wherein K is a positive integer greater than 0;

[0120] The sampling unit 203 is configured to perform uncertainty sampling on the clustered target speech data, and select a first preset number of target speech data from each type of target speech data;

[0121] a training unit 204 configured to train a speech processing model using the first preset amount of target speech data as sample data, and calculate uncertainty of each sample data during the training process so as to update the model in a direction of reducing the model uncertainty, thereby obtaining a speech processing model that has learned information about the target speech data;

[0122] The processing unit 205 is used to use the speech processing model that has learned the information of the target speech data to repeatedly perform K-layer unsupervised clustering processing and subsequent steps on the target speech data until a second preset number of target speech data are selected from each type of target speech data to obtain the processing result of the target speech data.

[0123] In one implementation of this embodiment, K is 2; the clustering unit 202 includes:

[0124] an extraction subunit, configured to perform feature extraction on the target speech data using a pre-trained model corresponding to the first speech processing task, to obtain a feature vector of the target speech data;

[0125] A first clustering subunit is configured to perform clustering processing on the feature vectors of the target speech data using a preset unsupervised clustering algorithm to obtain N types of target speech data, where N is a positive integer greater than 0;

[0126] The second clustering subunit is used to use the pre-trained model or preset unsupervised clustering algorithm corresponding to the second speech processing task to cluster the target speech data of each category in the N categories of target speech data into M categories of target speech data again to obtain NM categories of target speech data; M is a positive integer greater than 0; the second speech processing task and the first speech processing task are similar tasks or related tasks.

[0127] In one implementation of this embodiment, when the target speech data includes speech data sequences of different lengths, the extraction subunit includes:

[0128] an obtaining subunit, configured to perform feature extraction on speech data sequences of different lengths in the target speech data using a pre-trained model corresponding to the first speech processing task, and obtain feature vectors corresponding to the speech data sequences of different lengths;

[0129] The first calculation subunit is configured to calculate an average value of the feature vectors corresponding to the speech data sequences of different lengths as the feature vector of the target speech data.

[0130] In an implementation of this embodiment, the first speech processing task is a speech recognition task; and the second speech processing task is a speaker classification task.

[0131] In one implementation of this embodiment, the sampling unit 203 includes:

[0132] A second calculation subunit, configured to calculate an entropy value of a feature vector of the clustered target speech data;

[0133] The selection subunit is used to arrange the target speech data in each category of target speech data from large to small according to the entropy value, and select a first preset number of target speech data with a top ranking from each category of arranged target speech data.

[0134] In one implementation of this embodiment, the sampling unit 203 is specifically configured to:

[0135] The clustered target speech data are subjected to minimum confidence sampling or edge sampling, and a first preset number of target speech data are selected from each type of target speech data.

[0136] In one implementation of this embodiment, the training unit 204 is specifically configured to:

[0137] Training a speech processing model using the first preset amount of target speech data as sample data, and during the training process, calculating the uncertainty of each sample data using a given objective function so as to update the model in a direction that reduces the model uncertainty, thereby obtaining a speech processing model that has learned information about the target speech data;

[0138] The objective function is used to perform entropy sampling on the sample data.

[0139] Furthermore, an embodiment of the present application also provides a voice data processing device, comprising: a processor, a memory, and a system bus;

[0140] The processor and the memory are connected via the system bus;

[0141] The memory is used to store one or more programs, and the one or more programs include instructions. When the instructions are executed by the processor, the processor executes any one of the implementation methods of the above-mentioned voice data processing method.

[0142] Furthermore, an embodiment of the present application also provides a computer-readable storage medium, in which instructions are stored. When the instructions are executed on a terminal device, the terminal device executes any one of the implementation methods of the above-mentioned voice data processing method.

[0143] Furthermore, an embodiment of the present application also provides a computer program product, which, when running on a terminal device, enables the terminal device to execute any one of the implementation methods of the above-mentioned voice data processing method.

[0144] Through the description of the above embodiments, it can be seen that those skilled in the art can clearly understand that all or part of the steps in the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network communication device such as a media gateway, etc.) to execute the methods described in each embodiment of the present application or certain parts of the embodiments.

[0145] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Reference can be made to the common and similar parts between the various embodiments. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the description of the methods.

[0146] It should also be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or device comprising the element.

[0147] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present application. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application is not limited to the embodiments shown herein, but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for processing speech data, characterized in that: include: Obtaining target voice data to be processed; Performing K-layer unsupervised clustering processing on the target speech data to obtain clustered target speech data; The K is a positive integer greater than 0; Performing uncertainty sampling on the clustered target speech data, and selecting a first preset number of target speech data from each type of target speech data; Using the first preset amount of target speech data as sample data, training the speech processing model, and calculating the uncertainty of each sample data during the training process, so as to update the model in a direction in which the model uncertainty is reduced, and obtain a speech processing model that has learned the information of the target speech data; Using the speech processing model that has learned the information of the target speech data, repeatedly performing K-layer unsupervised clustering processing on the target speech data and subsequent steps until a second preset number of target speech data is selected from each type of target speech data to obtain a processing result of the target speech data; The step of performing K-layer unsupervised clustering processing on the target speech data to obtain clustered target speech data includes: Using a pre-trained model corresponding to the first speech processing task, extracting features from the target speech data to obtain a feature vector of the target speech data; Using a preset unsupervised clustering algorithm, clustering the feature vectors of the target speech data to obtain N types of target speech data, where N is a positive integer greater than 0; Using the pre-trained model or preset unsupervised clustering algorithm corresponding to the second speech processing task, the target speech data of each category in the N categories of target speech data are clustered again into M categories of target speech data to obtain NM categories of target speech data; M is a positive integer greater than 0; the second speech processing task and the first speech processing task are similar tasks or related tasks.

2. The method according to claim 1, characterized in that When the target speech data includes speech data sequences of different lengths; the use of the pre-trained model corresponding to the first speech processing task to extract features from the target speech data to obtain a feature vector of the target speech data includes: Using the pre-trained model corresponding to the first speech processing task, feature extraction is performed on speech data sequences of different lengths in the target speech data to obtain feature vectors corresponding to the speech data sequences of different lengths; The average value of the feature vectors corresponding to the speech data sequences of different lengths is calculated as the feature vector of the target speech data.

3. The method according to claim 1 or 2, characterized in that: The first speech processing task is a speech recognition task; the second speech processing task is a speaker classification task.

4. The method according to claim 1, characterized in that: The step of performing uncertainty sampling on the clustered target speech data and selecting a first preset number of target speech data from each type of target speech data includes: Calculating the entropy value of the feature vector of the clustered target speech data; The target speech data in each category of target speech data are arranged from large to small according to the entropy value, and a first preset number of target speech data ranked at the top are selected from each category of the arranged target speech data.

5. The method according to claim 1, characterized in that The step of performing uncertainty sampling on the clustered target speech data and selecting a first preset number of target speech data from each type of target speech data includes: The clustered target speech data are subjected to minimum confidence sampling or edge sampling, and a first preset number of target speech data are selected from each type of target speech data.

6. The method according to claim 1, characterized in that The method uses the first preset amount of target speech data as sample data to train the speech processing model, and calculates the uncertainty of each sample data during the training process so as to update the model in a direction in which the model uncertainty is reduced, thereby obtaining a speech processing model that has learned the information of the target speech data, including: Using the first preset number of target speech data as sample data, training the speech processing model, and during the training process, using a given objective function to calculate the uncertainty of each sample data, so as to update the model in a direction in which the model uncertainty is reduced, thereby obtaining a speech processing model that has learned the information of the target speech data; The objective function is used to perform entropy sampling on the sample data.

7. The method according to claim 1, characterized in that The K is 2.

8. A voice data processing device, characterized in that: include: An acquisition unit, used for acquiring target voice data to be processed; A clustering unit, used for performing K-layer unsupervised clustering processing on the target speech data to obtain clustered target speech data; wherein K is a positive integer greater than 0; A sampling unit, configured to perform uncertainty sampling on the clustered target speech data, and select a first preset number of target speech data from each type of target speech data; A training unit, configured to use the first preset number of target speech data as sample data to train a speech processing model, and calculate the uncertainty of each sample data during the training process, so as to update the model in a direction in which the uncertainty of the model is reduced, thereby obtaining a speech processing model that has learned the information of the target speech data; A processing unit, configured to repeatedly perform K-layer unsupervised clustering processing and subsequent steps on the target speech data using the speech processing model that has learned the information of the target speech data, until a second preset number of target speech data is selected from each type of target speech data, thereby obtaining a processing result of the target speech data; The clustering unit comprises: An extraction subunit, configured to perform feature extraction on the target speech data using a pre-trained model corresponding to the first speech processing task, to obtain a feature vector of the target speech data; A first clustering subunit is used to perform clustering processing on the feature vectors of the target speech data using a preset unsupervised clustering algorithm to obtain N types of target speech data, where N is a positive integer greater than 0; The second clustering subunit is used to cluster the target speech data of each category in the N categories of target speech data into M categories of target speech data again by using a pre-trained model or a preset unsupervised clustering algorithm corresponding to the second speech processing task, so as to obtain NM categories of target speech data; M is a positive integer greater than 0; and the second speech processing task and the first speech processing task are similar tasks or related tasks.

9. A voice data processing device, characterized in that: include: Processor, memory, system bus; The processor and the memory are connected via the system bus; The memory is used to store one or more programs, wherein the one or more programs include instructions, and when the instructions are executed by the processor, the processor executes the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, and when the instructions are executed on a terminal device, the terminal device executes the method according to any one of claims 1 to 7.

Images