Processing method of training sample, voice quality inspection method and device

Abstract

The application provides a processing method of training samples, a speech quality inspection method and device. The processing method comprises: performing classification prediction on a sample data set by a classification prediction model to obtain a predicted type and a confidence degree corresponding to each sample data; obtaining a real type of each sample data, determining sample data with a mismatched real type and predicted type as error sample data and counting the number; obtaining confidence distribution data of the sample data set of each real type; calculating a balance degree score of the sample data set of each real type according to the number of error sample data and the confidence distribution data; determining a real type with a balance degree score less than a preset balance degree threshold as a target type, and performing data augmentation processing on the sample data set of the target type. The method determines sample data of the target type according to classification prediction and confidence distribution data and performs data augmentation processing, thereby saving the cost of global data expansion and improving the generalization ability of the model.

Description

Technical Field

[0001] This application relates to the field of data processing, and in particular to a method for processing training samples, a voice quality inspection method, and an apparatus. Background Technology

[0002] Data diversity has received increasing attention in the field of machine learning in recent years. When the diversity distribution of data does not cover the actual scenario, there is a significant offset between the data diversity distribution to be predicted and the data diversity distribution used for training, which leads to poor model performance.

[0003] In related technologies, it is difficult to solve the problem of poor model performance caused by data diversity by simply improving the model's generalization ability. Therefore, the main methods to solve this problem include the following two: manual construction, screening, and labeling of samples. This method is effective, but it requires a lot of manpower and resources and needs to be completed by professionals in the field; data augmentation. This method is simple and easy to operate and can solve the problem of high manpower and resource costs, but if used improperly, it can easily introduce noise and affect the model's performance. Both of these methods aim to improve the model's generalization ability by increasing the diversity of training data, but neither of them conducts scientific analysis of the original data. In the original data, some categories have sufficient diversity and do not need to be increased; some categories lack diversity and need to have their diversity enriched. Blindly performing global data augmentation will waste resources (time cost, machine cost, etc.) and may even have the opposite effect. Summary of the Invention

[0004] This application provides a method for processing training samples, a method and apparatus for voice quality inspection, which are used to screen out sample data that needs to be augmented based on classification prediction and confidence analysis, thereby performing targeted data augmentation on the selected data, saving costs and improving efficiency.

[0005] Firstly, this application provides a method for processing training samples, including:

[0006] The sample dataset is classified and predicted by a classification prediction model to obtain the prediction type and the confidence level of each prediction type for each sample data.

[0007] Obtain the true type of each sample data, identify the sample data whose true type and predicted type do not match as error sample data, and calculate the number of error sample data contained in each true type sample dataset;

[0008] For each real type, obtain the confidence distribution data of the sample dataset of the real type; where multiple confidence intervals are pre-set, the confidence distribution data is used to characterize the number of sample data of the real type that fall in each confidence interval;

[0009] Calculate the balance score for each real-type sample dataset based on the number of error samples in each real-type sample dataset and the confidence distribution data for each real-type sample dataset.

[0010] The true types whose balance scores are less than the preset balance threshold are identified as the target types, and data augmentation processing is performed on the sample datasets of the target types.

[0011] Secondly, this application provides a training sample processing apparatus, comprising:

[0012] The classification prediction module is used to perform classification prediction on the sample dataset using a classification prediction model, and to obtain the prediction type and the confidence level of the prediction type for each sample data.

[0013] The error statistics module is used to obtain the true type of each sample data, identify sample data whose true type and predicted type do not match as error sample data, and calculate the number of error sample data contained in each true type sample dataset.

[0014] The distribution acquisition module is used to acquire the confidence distribution data of the sample dataset of each real type; multiple confidence intervals are preset, and the confidence distribution data is used to represent the number of real type sample data located in each confidence interval;

[0015] The balance calculation module is used to calculate the balance score of each real type of sample dataset based on the number of error sample data contained in each real type of sample dataset and the confidence distribution data of each real type of sample dataset.

[0016] The data augmentation module is used to identify the true types whose balance scores are less than a preset balance threshold as the target types, and to perform data augmentation processing on the sample dataset of the target types.

[0017] Thirdly, this application provides a voice quality inspection method, including:

[0018] Obtain the call text data from the agent terminal;

[0019] Input the call text data into the voice quality inspection model;

[0020] The quality inspection result of the call text data is determined based on the classification result of the voice quality inspection model;

[0021] The voice quality inspection model is trained using sample data augmented by the above method; and the length of the sample data is greater than a preset length threshold.

[0022] Fourthly, this application provides a voice quality inspection device, comprising:

[0023] The acquisition module is suitable for acquiring call text data from agent terminals;

[0024] The input module is adapted to input the call text data into the voice quality inspection model;

[0025] The quality inspection module is adapted to determine the quality inspection result of the call text data based on the classification result of the voice quality inspection model.

[0026] The voice quality inspection model is trained using sample data augmented by the above method; and the length of the sample data is greater than a preset length threshold.

[0027] Fifthly, this application provides an electronic device comprising: at least one processor; a memory communicatively connected to the at least one processor; and one or more I / O interfaces connected between the at least one processor and the memory; wherein the memory stores one or more computer programs executable by the at least one processor, the one or more computer programs being executed by the at least one processor to enable the at least one processor to perform the above-described method.

[0028] Sixthly, this application provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the above-described method when executed by a processor.

[0029] In the training sample processing method provided in this application, a classification prediction model is used to classify and predict the sample dataset to obtain the predicted type and confidence level of each sample data. The true type of each sample data is obtained, and sample data whose true type and predicted type do not match are identified as error sample data. The number of error sample data contained in the sample dataset of each true type is calculated. For each true type, the confidence level distribution data of the sample dataset of the true type is obtained. Multiple confidence level intervals are preset, and the confidence level distribution data is used to represent the number of true type sample data located in each confidence level interval. Based on the number of error sample data contained in the sample dataset of each true type and the confidence level distribution data of the sample dataset of each true type, the balance score of the sample dataset of each true type is calculated. The true type whose balance score is less than a preset balance threshold is identified as the target type, and data augmentation processing is performed on the sample dataset of the target type. Therefore, this method obtains the balance score of each real-type sample data based on classification prediction and confidence distribution data. Using this metric, which differs from related technologies, it quickly and effectively assesses the data diversity of the sample dataset, avoiding the increased time consumption of blind model training. This allows for the selection of sample datasets with low data diversity, identifying them as requiring data augmentation. Then, a scientific method is used to perform data augmentation on these sample datasets to enrich their data diversity, improve the model's generalization ability, save on the cost of global data augmentation, and avoid the resource waste or even counterproductive effects that can result from blindly performing global data augmentation. Furthermore, based on the above-described method for judging and enriching the data diversity of sample datasets, this method also features high computational speed and a high degree of automation.

[0030] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this application, nor is it intended to limit the scope of this application. Other features of this application will become readily apparent from the following description. Attached Figure Description

[0031] The accompanying drawings are provided to further illustrate the present application and form part of the specification. They are used together with the embodiments of the present application to explain the application and do not constitute a limitation thereof. The above and other features and advantages will become more apparent to those skilled in the art from the detailed example embodiments described with reference to the accompanying drawings, in which:

[0032] Figure 1 A flowchart illustrating a method for processing training samples provided in an embodiment of this application;

[0033] Figure 2A flowchart illustrating a method for processing training samples, provided in yet another embodiment of this application;

[0034] Figure 3 A flowchart illustrating a method for processing training samples, provided as a specific example of an embodiment of this application;

[0035] Figure 4 A structural block diagram of a training sample processing device provided in an embodiment of this application;

[0036] Figure 5 This is a structural block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0037] To enable those skilled in the art to better understand the technical solutions of this application, exemplary embodiments of this application are described below in conjunction with the accompanying drawings, including various details of the embodiments of this application to aid understanding. These should be considered merely exemplary. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this application. Similarly, for clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0038] Where there is no conflict, the various embodiments of this application and the features thereof may be combined with each other.

[0039] As used herein, the term “and / or” includes any and all combinations of one or more related enumerated entries.

[0040] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the application. As used herein, the singular forms “a” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that when the terms “comprising” and / or “made of” are used in this specification, the presence of the stated feature, integral, step, operation, element, and / or component is specified, but the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or groups thereof is not excluded. Terms such as “connected” or “linked” are not limited to physical or mechanical connections but can include electrical connections, whether direct or indirect.

[0041] Unless otherwise specified, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and this application, and will not be interpreted as having an idealized or overly formal meaning, unless expressly so defined herein.

[0042] The method provided in the embodiments of this application can be executed by electronic devices such as terminal devices or servers. The terminal device can be an in-vehicle device, user equipment (UE), mobile device, user terminal, terminal, cellular phone, cordless phone, personal digital assistant (PDA), handheld device, computing device, in-vehicle device, wearable device, etc. The server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. Specifically, the method can be implemented by a processor calling a computer program stored in memory.

[0043] In related technologies, the diversity of training sample data is mainly increased through two methods: manual construction, screening, and labeling of samples, and data augmentation, thereby improving the model training effect. However, the former requires a lot of manpower and resources and needs to be completed by professionals in the field; the latter, if used improperly, can easily introduce noise and affect the model's performance. To solve the above problems, this application provides a method for processing training samples. This method obtains the balance score of each real type of sample data based on classification prediction and confidence distribution data, thereby determining the sample data that needs data augmentation, and performing data augmentation processing on this part of the data, saving the cost required for global data augmentation and improving the model's generalization ability.

[0044] Figure 1 A flowchart illustrating a training sample processing method provided in an embodiment of this application. (Refer to...) Figure 1 The method includes:

[0045] Step S110: Classify and predict the sample dataset using a classification prediction model to obtain the prediction type and confidence level of each sample data.

[0046] In one specific implementation, each sample data is fed into a classification prediction model. Based on the model's output, the corresponding prediction type and the confidence level of that prediction type are obtained, thereby achieving classification prediction for the sample data. The classification prediction model is pre-trained and used to perform classification predictions on the input sample dataset.

[0047] In statistics, a confidence interval for a probability sample is an interval estimate of a population parameter within that sample. The confidence interval shows the degree to which the true value of this parameter has a certain probability of falling within the range of the measured result. The confidence interval provides the range of confidence regarding the measured value of the measured parameter, i.e., the "certain probability" mentioned earlier; this probability is called the confidence level, or confidence degree. In this embodiment, the confidence degree is used to represent the probability that the classification prediction result corresponding to the sample data is the true type.

[0048] In one specific implementation, a base model is used in conjunction with K-Fold cross-validation to obtain pseudo-labels and confidence scores for multiple sample data. The pseudo-label refers to the result with the highest probability among the classification predictions for each sample data as its classification label. The probability corresponding to the result with the highest probability (i.e., the pseudo-label) is the confidence score for the predicted type of that sample data. The base model can be selected from LR (Logistic Regression), SVM (Support Vector Machine), BERT (Bidirectional Encoder Representations from Transformer, a pre-trained language representation model), etc., as chosen by those skilled in the art in implementing this step; this application does not impose any restrictions.

[0049] Step S120: Obtain the true type of each sample data, identify the sample data whose true type and predicted type do not match as error sample data, and calculate the number of error sample data contained in each true type sample dataset.

[0050] In this process, the true types contained in the sample dataset are pre-defined manually, and the true type of each sample data is pre-labeled manually. For the classification prediction results obtained based on the classification prediction model in step S110, the number of error sample data where the pseudo label is inconsistent with the true type label under each true type label is counted, which is the number of error sample data contained in the sample dataset of each true type.

[0051] Step S130: For each real type, obtain the confidence distribution data of the sample dataset of the real type; wherein, multiple confidence intervals are pre-set, and the confidence distribution data is used to characterize the number of sample data of the real type located in each confidence interval.

[0052] In one specific implementation, at least two confidence intervals are set, and the upper limit of the first confidence interval does not exceed the lower limit of the second confidence interval; for example, if the confidence intervals are set to [0, α1) and [α1, α2), then the confidence distribution data is used to characterize the number of true type sample data located in the intervals [0, α1) and [α1, α2), respectively.

[0053] Step S140: Calculate the balance score of each real type of sample dataset based on the number of error sample data contained in each real type of sample dataset and the confidence distribution data of each real type of sample dataset.

[0054] The balance score is used to determine the data diversity of the sample dataset. It is calculated based on the number of error sample data in each true type of sample dataset calculated in step S120 and the confidence distribution data of each true type of sample dataset obtained in step S130.

[0055] Step S150: Determine the true type whose balance score is less than the preset balance threshold as the target type, and perform data augmentation processing on the sample dataset of the target type.

[0056] If the balance score is less than the preset balance threshold, it indicates that the data diversity of this real type is low, and therefore it is identified as the target type that needs data augmentation, and data augmentation processing is performed. The preset balance threshold can be set by those skilled in the art when implementing this step, depending on the specific circumstances; this application does not impose any restrictions on it.

[0057] The purpose of data augmentation is to diversify the dataset as much as possible, so that the trained model has a stronger generalization ability. This can be achieved through synonym replacement, random punctuation, and oversampling.

[0058] In summary, the training sample processing method provided in this embodiment involves classifying and predicting the sample dataset using a classification prediction model to obtain the predicted type and confidence level of each sample data. The true type of each sample data is obtained, and sample data whose true type and predicted type do not match are identified as error sample data. The number of error sample data contained in each true type's sample dataset is calculated. For each true type, the confidence level distribution data of the true type's sample dataset is obtained. Multiple confidence level intervals are pre-set, and the confidence level distribution data represents the number of true type sample data located in each confidence level interval. Based on the number of error sample data contained in each true type's sample dataset and the confidence level distribution data of each true type's sample dataset, the balance score of each true type's sample dataset is calculated. True types with balance scores less than a preset balance threshold are identified as target types, and data augmentation processing is performed on the target type's sample dataset. Therefore, this method obtains the balance score of each real-type sample dataset based on classification prediction and confidence distribution data. This metric, distinct from related technologies, quickly and effectively assesses the data diversity of the sample dataset, avoiding the increased time consumption of blind model training. This allows for the selection of sample datasets with low data diversity, identifying them as requiring data augmentation. Then, scientific methods are used to perform data augmentation on these datasets to enrich their data diversity, improve the model's generalization ability, save on the cost of global data augmentation, and avoid the resource waste or even counterproductive effects that can result from blindly performing global data augmentation. Furthermore, based on the above-described method for judging and enriching the data diversity of sample datasets, this method also features high computational speed and a high degree of automation.

[0059] Figure 2 This is a flowchart illustrating a method for processing training samples, provided in yet another embodiment of this application. (Refer to...) Figure 2 The method includes:

[0060] Step S210: Classify and predict the sample dataset using a classification prediction model to obtain the prediction type and confidence level of each sample data.

[0061] In one specific implementation, each sample data is fed into a classification prediction model, and the corresponding prediction type and the confidence level of the prediction type are obtained based on the model output, thereby realizing the classification prediction of the sample data.

[0062] In statistics, a confidence interval for a probability sample is an interval estimate of a population parameter within that sample. The confidence interval shows the degree to which the true value of this parameter has a certain probability of falling within the range of the measured result. The confidence interval provides the range of confidence regarding the measured value of the measured parameter, i.e., the "certain probability" mentioned earlier; this probability is called the confidence level, or confidence degree. In this embodiment, the confidence degree is used to represent the probability that the classification prediction result corresponding to the sample data is the true type.

[0063] In one specific implementation, a base model is used in conjunction with K-Fold cross-validation to obtain pseudo-labels and confidence scores for multiple sample data. The pseudo-label refers to the result with the highest probability among the classification predictions for each sample data as its classification label. The probability corresponding to the result with the highest probability (i.e., the pseudo-label) is the confidence score for the predicted type of that sample data. The base model can be selected from LR (Logistic Regression), SVM (Support Vector Machine), BERT (Bidirectional Encoder Representations from Transformer, a pre-trained language representation model), etc., as chosen by those skilled in the art in implementing this step; this application does not impose any restrictions.

[0064] Step S220: Obtain the true type of each sample data, identify the sample data whose true type and predicted type do not match as error sample data, and calculate the number of error sample data contained in each true type sample dataset.

[0065] In this process, the true types contained in the sample dataset are pre-defined manually, and the true type of each sample data is pre-labeled manually. For the classification prediction results obtained based on the classification prediction model in step S210, the number of error sample data where the pseudo label is inconsistent with the true type label under each true type label is counted, which is the number of error sample data contained in the sample dataset of each true type.

[0066] Step S230: For each real type, obtain the confidence distribution data of the sample dataset of the real type.

[0067] In one specific implementation, three confidence intervals are pre-set, and the confidence distribution data is used to characterize the number of true type sample data located in each confidence interval. Then, step S230 specifically includes the following methods:

[0068] First, obtain the first number of real-type sample data that falls within the first confidence interval;

[0069] Secondly, obtain the second number of real-type sample data that fall within the second confidence interval;

[0070] Next, obtain the third number of real-type sample data that fall within the third confidence interval;

[0071] Finally, based on the first, second, and third quantities, as well as the total amount of sample data of the true type, the confidence distribution data of the sample dataset of the true type is obtained, that is, the number of sample data of the true type located in the first confidence interval, the second confidence interval, and the third confidence interval, respectively.

[0072] The first confidence interval, the second confidence interval, and the third confidence interval are divided by the endpoints of the first and second intervals. The first confidence interval is less than the endpoint of the second interval, the second confidence interval is greater than or equal to the endpoint of the second interval and less than the endpoint of the first interval, and the third confidence interval is greater than or equal to the endpoint of the first interval. The endpoint of the first interval is greater than the endpoint of the second interval, and the sum of the endpoints of the first and second intervals equals a preset value. For example, if the endpoint of the first interval is set to α1 and the endpoint of the second interval is set to α2, then the first confidence interval is [0, α2), the second confidence interval is [α2, α1), and the third confidence interval is [α1, 1]. Here, α1 > α2 > b, and α1 + α2 = α. a and b are preset values, which can be set by those skilled in the art according to specific circumstances when implementing this step, and this application does not impose any restrictions here.

[0073] Step S240: Calculate the accuracy parameter and confidence parameter for each true type.

[0074] Among them, the accuracy parameter is used to characterize the proportion of non-error sample data in the sample dataset of each true type, that is, the prediction accuracy of the classification prediction model for the true type, and the confidence parameter is used to characterize the diversity of sample data in the true type. In a specific implementation, step S240 includes:

[0075] Based on the number of error samples in each true type's dataset and the total number of samples in each true type's dataset, calculate the accuracy parameter for each true type; specifically, calculate the accuracy parameter using the following formula:

[0076]

[0077] Among them, X i Let N be the accuracy parameter for the i-th true type. iLet n be the total number of samples in the i-th real-type sample dataset. i Let be the total amount of error sample data contained in the i-th real type sample dataset.

[0078] Based on the confidence distribution data, the first interval endpoint, and the second interval endpoint of the sample dataset for each real type, calculate the confidence parameter corresponding to each real type; specifically, calculate the confidence parameter using the following formula:

[0079]

[0080] Among them, Y i Let N be the confidence parameter for the i-th true type. i Let be the total number of samples in the i-th real-type sample dataset, α1 be the endpoint of the first interval, α2 be the endpoint of the second interval, α1>α2>b, and α1+α2=a; k i p i q i It is confidence distribution data, k i Let p be the first number of samples of the i-th true type that fall within the first confidence interval. i Let q be the second number of samples of the i-th true type that fall within the second confidence interval. i Let be the third number of samples of the i-th true type that are located in the third confidence interval.

[0081] Step S250: Calculate the balance score of each real-type sample dataset based on the accuracy parameter and the confidence parameter.

[0082] The balance score is used to determine the data diversity of the sample dataset. Specifically, the balance score is calculated using the following formula:

[0083]

[0084] Where, ω i For the data diversity of the i-th real-type sample dataset, X i Y is the accuracy parameter for the i-th true type. i The confidence parameter for the i-th true type has been calculated in step S240, and m is the number of true types.

[0085] Step S260: Determine the true type whose balance score is less than the preset balance threshold as the target type, and perform data augmentation processing on the sample dataset of the target type.

[0086] In one specific implementation, the preset balance threshold is characterized by the diversity of the overall sample data. Specifically, the preset balance threshold is calculated using the following formula:

[0087]

[0088]

[0089]

[0090] Where X is the accuracy parameter of the entire sample data set, Y is the confidence parameter of the entire sample data set, ω is the data diversity of the entire sample data set; N is the total number of samples in the entire sample data set, n is the total number of error samples in the entire sample data set; α1 is the endpoint of the first interval, α2 is the endpoint of the second interval, α1>α2>b, and α1+α2=a; k, p, q are confidence distribution data, k is the first number of samples in the entire sample data set located in the first confidence interval, p is the second number of samples in the entire sample data set located in the second confidence interval, and q is the third number of samples in the entire sample data set located in the third confidence interval.

[0091] Wherein, if the balance score is less than the preset balance threshold, i.e., ω i If the value is >ω, it indicates that the data diversity of this real type is low, so it is identified as the target type that needs to be augmented, and data augmentation processing is performed.

[0092] In one specific implementation, performing data augmentation on a sample dataset of the target type can be achieved in the following two ways:

[0093] Obtain error sample data from the sample dataset of the target type, and perform data augmentation processing on the error sample data;

[0094] Obtain non-error sample data from the target type's sample dataset, and extract non-error sample data with a confidence level lower than a preset confidence threshold as target sample data; perform data augmentation processing on the target sample data.

[0095] The two methods described above perform data augmentation processing on error sample data and non-error sample data that are more likely to be error sample data in the sample dataset of the target type.

[0096] In one specific implementation, data augmentation processing on a given sample data includes: synonym replacement, random punctuation, and oversampling. Synonym replacement involves randomly selecting several non-stop words in the sentence and replacing each selected word with a randomly chosen synonym. Random punctuation involves randomly selecting a punctuation mark and inserting it at any position in the sentence. Oversampling involves sampling the data with a sample size far exceeding twice the size of the original sample data set or the entire sample data set.

[0097] In summary, in the training sample processing method provided in this embodiment, a classification prediction model is used to classify and predict the sample dataset to obtain the predicted type and confidence level of each sample data. The true type of each sample data is obtained, and sample data whose true type and predicted type do not match are identified as error sample data. The number of error sample data contained in the sample dataset of each true type is calculated. For each true type, the confidence level distribution data of the sample dataset of the true type is obtained. Multiple confidence level intervals are pre-set, and the confidence level distribution data is used to characterize the number of true type sample data located in each confidence level interval. An accuracy parameter is calculated based on the number of error sample data contained in the sample dataset of each true type, and a confidence level parameter is calculated based on the confidence level distribution data of the sample dataset of each true type. Based on this, the balance score of the sample dataset of each true type is calculated. True types with balance scores less than a preset balance threshold are identified as target types, and data augmentation processing is performed on the sample dataset of the target types. Therefore, this method obtains the accuracy and confidence parameters for each real-type sample data based on classification prediction and confidence distribution data. These two parameters measure different dimensions and are combined to measure data diversity, avoiding the problem of only one parameter performing well due to overfitting, which could lead to subjective results. This measurement standard, different from related technologies, quickly and effectively obtains the data diversity of sample datasets, avoiding the increased business cycle time caused by blind model training. It then filters out sample datasets with low data diversity, identifying them as requiring data augmentation. Scientific methods are then used to perform data augmentation on these sample datasets to enrich their data diversity, improve the model's generalization ability, save the cost of global data augmentation, and avoid the resource waste or even counterproductive effects that can result from blindly performing global data augmentation. Furthermore, based on the above method for judging and enriching the data diversity of sample datasets, this method also features high computational speed and a high degree of automation.

[0098] To facilitate understanding, a specific example will be used below to describe in detail the specific implementation details of this embodiment.

[0099] Data diversity has received increasing attention in the field of machine learning in recent years, becoming a major challenge for machine learning models in practical deployment. When the diversity distribution of data does not cover the actual scenario, there is a significant deviation between the diversity distribution of data to be predicted and the diversity distribution of data used for training, which leads to poor model performance.

[0100] Data diversity issues lead to poor model performance, which is difficult to address by simply improving the model's generalization ability, as current machine learning methods are primarily based on the assumption of independent and identically distributed (IOD) data. With a limited amount of observable training data under a real distribution, a well-trained model will experience a drop in accuracy when encountering unobserved samples that conform to the same distribution during prediction. While choosing appropriate algorithms, cross-validation, and regularization can effectively improve generalization in such cases, the fundamental problem of data diversity lies in the significant difference between the real-world data distribution and the actual distribution in real-world scenarios. Therefore, simply improving generalization ability is insufficient to effectively enhance model performance.

[0101] Existing technologies address the aforementioned challenges by: manually constructing, screening, and labeling samples, which yields good results but requires significant human and material resources and specialists in the field; and data augmentation, which is simple and easy to implement, addressing the resource constraints, but can introduce noise if used improperly, affecting model performance, and has the following drawbacks: the semantics of the augmented sentences are difficult to control and easily change, affecting the model's generalization effect; and the inability to specifically increase diversity can lead to suboptimal overall performance.

[0102] Figure 3 This is a flowchart illustrating a method for processing training samples, provided as a specific example of an embodiment of this application. (Refer to...) Figure 3 The method includes:

[0103] Step 1: Classify the raw labeled data;

[0104] Input the original labeled data, assuming there are N sample data points, namely text_1, text_2, ..., text_N; and m categories, namely C1, C2, ..., C6. m The number of texts contained in each category are N1, N2, ..., N. m Where N = N1 + N2 + ... + N m .

[0105] Step 2: Obtain the pseudo-labels and confidence levels of the labeled data;

[0106] Using a base model (LR, SVM, BERT, etc.) combined with K-Fold (K-fold cross-validation), pseudo-labels and confidence scores of N data points are obtained (the pseudo-labels are equivalent to the prediction types obtained based on the model output in step S210 of the above embodiment), as shown in Table 1.

[0107] Table 1. Pseudo-labels and confidence levels

[0108]

[0109] Step 3: Data statistics;

[0110] Based on the results in Table 1, statistical analysis was performed. Of the N samples, n samples had inconsistent true and false labels; among the N data points, k, p, and q samples had false label confidence levels γ≥α1, α2≤γ<α1, and γ<α2, respectively; the categories were C1, C2, ..., C... m The corresponding instances where the real and pseudo labels are inconsistent are n1, n2, ..., n m Items; Categories C1, C2, ..., C m The corresponding N1, N2, ..., N m In the data, the pseudo-label confidence scores γ≥α1, α2≤γ<α1, and γ<α2 are k1, p1, q1, k2, p2, q2, ..., k m ,p m ,q m Article. Where n = n1 + n2 + ... + n m ; α1>α2>0.5, and α1+α2=1.5, the value of α1 is generally 0.85, and the value of α2 is generally 0.65.

[0111] Step 4: Calculate overall diversity;

[0112] Overall diversity is used to measure the data diversity of the entire labeled dataset. As described in step S260 of the above embodiment, it is used to characterize a preset balance threshold. The formula for calculating overall diversity ω is:

[0113]

[0114]

[0115]

[0116] Here, X is the accuracy parameter, which reflects the strength of the fault tolerance of the labeled data on the model through the recognition results (pseudo-labels) of the base model, representing the quality of the labeled data diversity; Y is the confidence parameter, which reflects the strength of the generalization ability of the labeled data on the model through the confidence of the pseudo-labels and the number of samples included within a certain threshold confidence range, also representing the quality of the labeled data diversity; the overall diversity ω is calculated by combining X and Y, where X and Y represent different measurement dimensions, and the excellence of the overall diversity ω can only be proved when both X and Y perform well (because if only X and Y perform well, it may be due to overfitting or other reasons, and cannot objectively reflect the data diversity).

[0117] Step 5: Calculate inter-class diversity;

[0118] Inter-class diversity is used to measure the data diversity of each category of labeled sample data. As described in steps S240 and S250 of the above embodiment, inter-class diversity is determined by the balance score, where the i-th category C... i diversity ω i The formula for calculating the (balance score) is:

[0119]

[0120] Among them, X i The accuracy parameter for the i-th true type is calculated using the following formula: Y i The confidence parameter for the i-th true type is calculated using the following formula:

[0121] Step Six: Diversity Assessment;

[0122] The i-th (1≤i≤m) category C i diversity ω i The criteria for determining whether the requirements are met are as follows:

[0123] ω i >ω

[0124] If ω i If the judgment criteria are met, it means that ω in the original data... i The diversity is sufficient, and no additional data is needed; otherwise, it indicates that ω i The diversity is insufficient, and the diversity of this category needs to be enriched.

[0125] Step Seven: Enrich Diversity;

[0126] For categories lacking diversity, the following methods can be used to increase inter-class diversity:

[0127] For sample data that is misidentified (where the real label and pseudo label are inconsistent) and has low confidence (but the real label and pseudo label are consistent): ① Increase the number of samples while ensuring that the semantics remain unchanged by using methods such as synonym replacement, random punctuation, and oversampling; ② Increase the weight of these samples and categories during model training.

[0128] In summary, this specific example of a training data processing method detects sample diversity and improves the model's generalization ability by statistically analyzing the pseudo-labels of the labeled dataset, designing evaluation methods for overall diversity and inter-class diversity, establishing diversity judgment rules, and employing diversity enrichment methods. Specifically, for diversity evaluation, a novel method for measuring sample diversity is provided, which can quickly and effectively calculate overall and inter-class diversity, avoiding blind model training and increasing business cycle time. For diversity judgment rules, categories with insufficient inter-class diversity can be evaluated and identified. For diversity enrichment, a scientific method for enriching inter-class diversity is provided, improving the model's generalization ability and solving the problem of blindly expanding data globally, which wastes resources (time cost, machine cost, etc.) and can even have adverse effects (easily introduces noise, causing data pollution; the semantics of expanded data are difficult to control; and it cannot achieve global optimization). Furthermore, this method features fast computation speed and a high degree of automation.

[0129] The methods provided in the specific examples above can be applied to the agent dialogue classification quality inspection scenario. In this scenario, model training is needed to accurately classify agent dialogue to improve service quality. The identified categories include inquiries about identity, threats and intimidation, insults, unprofessional language, and other business-related categories. In this scenario, the text for each category has the following characteristics: the text is relatively long (a single sentence generally cannot accurately express the semantics, requiring the splicing of surrounding information); the text has strong business relevance; and the semantics (category or intent) of the text cannot be accurately expressed by a few words. In this scenario, statistical methods based on words, sentences, or other dimensions cannot objectively reflect the diversity of the data. The above method measures data diversity from a semantic perspective. Before model training, the business obtains corresponding dialogue data for each category from historical call data and labels these dialogue data to obtain the original labeled dataset. Statistics are performed on the overall labeled data and the text data for each category, including the total number of samples, the number of categories, and the number of samples in each category. Based on the labeled dataset, the model is trained and tested using a base model and cross-validation to obtain pseudo-labels and confidence levels for the labeled data. Based on the sample's true label, pseudo label, and confidence level, data statistics are performed separately for the overall data and categories, including the number of entries where the true label and pseudo label are inconsistent, and the number of samples included in different confidence levels. Based on the above statistical data and diversity calculation formula, the overall diversity and inter-class diversity of the labeled data are calculated. Categories with insufficient inter-class diversity are identified through diversity judgment rules. For categories with insufficient diversity, the inter-class diversity problem is improved or resolved by increasing the number of samples and the weights of samples and categories. The corrected labeled dataset is then used to train the model for agent dialogue classification.

[0130] Furthermore, another embodiment of this application provides a voice quality inspection method, which applies the above-mentioned training sample processing method to a call center dialogue classification quality inspection scenario. In this scenario, model training is needed to accurately classify call center dialogue, thereby improving the service quality of the call center. The identified categories include identity inquiries, threats and intimidation, insults, unprofessional language, and other business-related categories. In this scenario, the text for each category has the following characteristics: the text is relatively long (a single sentence generally cannot accurately express the semantics, requiring the splicing of information from adjacent sentences); the text has strong business relevance; and the semantics (category or intent) of the text cannot be accurately expressed by a few words. In this scenario, statistical methods based on words, sentences, and other dimensions cannot objectively reflect the diversity of the data. This method measures the diversity of data from the semantic perspective of the text, specifically including the following steps:

[0131] Step 1: Obtain the call text data from the agent terminal.

[0132] In one specific implementation, before model training, the business obtains the corresponding dialogue data for each category from historical call data and labels these dialogue data accordingly to obtain the original labeled sample dataset.

[0133] Step 2: Input the call text data into the voice quality inspection model.

[0134] The voice quality inspection model is trained on sample data after augmentation processing using the above method; and the length of the sample data is greater than a preset length threshold; the preset length threshold is set by those skilled in the art according to the specific circumstances of the scenario when implementing this method, and is not restricted here.

[0135] Based on the training sample processing method provided in the above embodiments, the above speech quality inspection model is specifically trained in the following manner:

[0136] (1) Data preprocessing: Statistical analysis of the overall labeled data and the text data of each category, including the total number of samples, the number of categories, and the number of samples in each category;

[0137] (2) Obtain pseudo-labels and confidence scores (pseudo-labels are equivalent to the prediction type obtained based on the model output in step S210 of the above embodiment): Based on the labeled sample dataset, use the basic model and cross-validation to train and test the model, and obtain the pseudo-labels and confidence scores of the labeled data.

[0138] (3) Data statistics: Based on the true labels, pseudo labels and their confidence levels of the samples, data statistics are performed separately for the whole and the categories, including the number of items where the true labels and pseudo labels are inconsistent, and the number of samples included in different confidence level ranges;

[0139] (4) Diversity calculation: Based on the above statistical data and diversity calculation formula, calculate the overall diversity and inter-class diversity of the labeled data; wherein, the overall diversity is used to measure the data diversity of the entire set of labeled data, as described in step S260 of the above embodiment, and is used to characterize the preset balance threshold; the inter-class diversity is used to measure the data diversity of each category of sample data of the labeled data, as described in steps S240 and S250 of the above embodiment, and the inter-class diversity is judged by the balance score;

[0140] (5) Diversity judgment: Based on the diversity judgment rules, identify the categories with insufficient inter-class diversity;

[0141] (6) Enriching diversity: For categories with insufficient diversity, data augmentation is performed by increasing the number of samples and the weights of samples and categories to improve or solve the problem of inter-class diversity;

[0142] (7) Model training: The corrected labeled sample dataset is used as sample data for model training, thereby training a voice quality inspection model for agent dialogue classification.

[0143] Step 3: Determine the quality inspection results of the call text data based on the classification results of the voice quality inspection model.

[0144] It is understood that the various method embodiments mentioned above in this application can be combined with each other to form combined embodiments without violating the principle and logic. Due to space limitations, this application will not elaborate further. Those skilled in the art will understand that in the above methods of specific implementation, the specific execution order of each step should be determined by its function and possible internal logic.

[0145] Figure 4 This is a structural block diagram of a training sample processing device provided in an embodiment of this application. (Refer to...) Figure 4 This application provides a training sample processing apparatus 40, which includes:

[0146] The classification prediction module 41 is used to perform classification prediction on the sample dataset through the classification prediction model, and obtain the prediction type and the confidence level of the prediction type for each sample data.

[0147] Error statistics module 42 is used to obtain the true type of each sample data, identify sample data whose true type and predicted type do not match as error sample data, and calculate the number of error sample data contained in each true type sample dataset.

[0148] The distribution acquisition module 43 is used to acquire the confidence distribution data of the sample dataset of each real type; wherein, multiple confidence intervals are preset, and the confidence distribution data is used to characterize the number of real type sample data located in each confidence interval;

[0149] The balance calculation module 44 is used to calculate the balance score of each real type of sample dataset based on the number of error sample data contained in each real type of sample dataset and the confidence distribution data of each real type of sample dataset.

[0150] The data augmentation module 45 is used to identify the real type whose balance score is less than the preset balance threshold as the target type, and to perform data augmentation processing on the sample dataset of the target type.

[0151] Optionally, the distribution acquisition module 43 is specifically used for:

[0152] Obtain the first number of real-type sample data that falls within the first confidence interval;

[0153] Obtain the second number of real-type sample data points that fall within the second confidence interval;

[0154] Obtain the third number of real-type sample data points located in the third confidence interval;

[0155] Based on the first quantity, the second quantity, the third quantity, and the total amount of sample data of the real type, the confidence distribution data of the sample dataset of the real type is obtained;

[0156] The first confidence interval, the second confidence interval, and the third confidence interval are divided by the endpoints of the first interval and the second interval; and the first confidence interval is less than the endpoint of the second interval, the second confidence interval is greater than or equal to the endpoint of the second interval and less than the endpoint of the first interval, and the third confidence interval is greater than or equal to the endpoint of the first interval; wherein the endpoint of the first interval is greater than the endpoint of the second interval, and the sum of the endpoints of the first interval and the endpoint of the second interval is equal to a preset value.

[0157] Optionally, the balance calculation module 44 is specifically used for:

[0158] Calculate the accuracy parameter for each real type based on the number of error samples in each real type's sample dataset and the total number of samples in each real type's sample dataset.

[0159] Calculate the confidence parameter for each real type based on the confidence distribution data, the first interval endpoint, and the second interval endpoint of the sample dataset for each real type.

[0160] Based on the accuracy parameter and the confidence parameter, the balance score of each real type of sample dataset is calculated.

[0161] Optionally, the data augmentation module 45 is specifically used for:

[0162] Obtain error sample data contained in the sample dataset of the target type;

[0163] Perform data augmentation processing on the error sample data.

[0164] The specific structure and working principle of each of the above modules can be found in the description of the corresponding part of the above method embodiment, and will not be repeated here.

[0165] In addition, another embodiment of this application provides a voice quality inspection device, including:

[0166] The acquisition module is suitable for acquiring call text data from agent terminals;

[0167] The input module is adapted to input the call text data into the voice quality inspection model;

[0168] The quality inspection module is adapted to determine the quality inspection result of the call text data based on the classification result of the voice quality inspection model.

[0169] The voice quality inspection model is trained using sample data augmented by the above method; and the length of the sample data is greater than a preset length threshold.

[0170] Figure 5 This is a structural block diagram of an electronic device provided in an embodiment of this application. (Refer to...) Figure 5 This application provides an electronic device 50, which includes:

[0171] At least one processor 501; a memory 502 communicatively connected to at least one processor 501; and one or more I / O interfaces 503 connected between at least one processor 501 and memory 502; wherein memory 502 stores one or more computer programs executable by at least one processor 501, the one or more computer programs being executed by at least one processor 501 to enable at least one processor 501 to perform the above-described method.

[0172] This application also provides a computer-readable storage medium storing a computer program thereon, wherein the computer program implements the above-described method when executed by a processor. The computer-readable storage medium may be a volatile or non-volatile computer-readable storage medium.

[0173] This application also provides a computer program product, including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code, wherein when the computer-readable code is run in the processor of an electronic device, the processor in the electronic device executes the above-described method.

[0174] Those skilled in the art will understand that all or some of the steps, systems, and apparatuses disclosed above, and their functional modules / units, can be implemented as software, firmware, hardware, or suitable combinations thereof. In hardware implementations, the division between functional modules / units mentioned above does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit (ASIC). Such software can be distributed on a computer-readable storage medium, which may include computer storage media (or non-transitory media) and communication media (or transient media).

[0175] As is known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable program instructions, data structures, program modules, or other data). Computer storage media includes, but is not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), static random access memory (SRAM), flash memory or other memory technologies, portable compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer. Furthermore, it is known to those skilled in the art that communication media typically contain computer-readable program instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0176] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0177] The computer program instructions used to perform the operations of this application may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuits, such as programmable logic circuits, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), are personalized by utilizing state information from the computer-readable program instructions. These electronic circuits can execute the computer-readable program instructions to implement various aspects of this application.

[0178] The computer program product described herein can be implemented specifically through hardware, software, or a combination thereof. In one alternative embodiment, the computer program product is specifically embodied in a computer storage medium; in another alternative embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0179] Various aspects of this application are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0180] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0181] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0182] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0183] Example embodiments have been disclosed herein, and while specific terminology has been used, it is for general illustrative purposes only and should not be construed as limiting. In some instances, it will be apparent to those skilled in the art that features, characteristics, and / or elements described in conjunction with particular embodiments may be used alone, or in combination with features, characteristics, and / or elements described in conjunction with other embodiments, unless otherwise expressly indicated. Therefore, those skilled in the art will understand that various changes in form and detail may be made without departing from the scope of this application as set forth by the appended claims.

Claims

1. A method for processing training samples, characterized in that, include: The sample dataset is classified and predicted by a classification prediction model to obtain the prediction type and the confidence level of each prediction type for each sample data. Obtain the true type of each sample data, identify the sample data whose true type and predicted type do not match as error sample data, and calculate the number of error sample data contained in each true type sample dataset; For each real type, obtain the confidence distribution data of the sample dataset of that real type; wherein, multiple confidence intervals are pre-set, and the confidence distribution data is used to characterize the number of sample data of the real type located in each confidence interval; The balance score of each real-type sample dataset is calculated based on the number of error samples in each real-type sample dataset and the confidence distribution data of each real-type sample dataset. The true type whose balance score is less than the preset balance threshold is identified as the target type, and data augmentation processing is performed on the sample dataset of the target type.

2. The method according to claim 1, characterized in that, The step of obtaining the confidence distribution data of the sample dataset for each real type includes: Obtain the first number of sample data of the real type that fall within the first confidence interval; Obtain the second number of sample data of the real type that fall within the second confidence interval; Obtain the third number of sample data of the real type that fall within the third confidence interval; Based on the first quantity, the second quantity, the third quantity, and the total amount of sample data of the real type, the confidence distribution data of the sample dataset of the real type is obtained; The first confidence interval, the second confidence interval, and the third confidence interval are divided by the first interval endpoint and the second interval endpoint; and the first confidence interval is less than the second interval endpoint, the second confidence interval is greater than or equal to the second interval endpoint and less than the first interval endpoint, and the third confidence interval is greater than or equal to the first interval endpoint; wherein the first interval endpoint is greater than the second interval endpoint, and the sum of the first interval endpoint and the second interval endpoint is equal to a preset value.

3. The method according to claim 2, characterized in that, The step of calculating the balance score of each real-type sample dataset based on the number of error samples in each real-type sample dataset and the confidence distribution data of each real-type sample dataset includes: Calculate the accuracy parameter for each real type based on the number of error samples in each real type's sample dataset and the total number of samples in each real type's sample dataset. Based on the confidence distribution data of the sample dataset of each real type, the first interval endpoints, and the second interval endpoints, calculate the confidence parameter corresponding to each real type; Based on the accuracy parameter and the confidence parameter, the balance score of each real-type sample dataset is calculated.

4. The method according to claim 1, characterized in that, The data augmentation process performed on the sample dataset of the target type includes: Obtain the error sample data contained in the sample dataset of the target type; Perform data augmentation processing on the error sample data.

5. The method according to claim 1, characterized in that, The data augmentation process performed on the sample data of the target type includes: Obtain non-error sample data contained in the sample data of the target type, and extract non-error sample data with a confidence level lower than a preset confidence threshold as target sample data; Perform data augmentation processing on the target sample data.

6. A voice quality inspection method, characterized in that, include: Obtain the call text data from the agent terminal; Input the call text data into the voice quality inspection model; The quality inspection result of the call text data is determined based on the classification result of the voice quality inspection model; The voice quality inspection model is trained using sample data augmented by the method described in any one of claims 1-5; and the length of the sample data is greater than a preset length threshold.

7. A training sample processing apparatus, characterized in that, include: The classification prediction module is used to perform classification prediction on the sample dataset through a classification prediction model, and obtain the prediction type and the confidence level of the prediction type for each sample data. The error statistics module is used to obtain the true type of each sample data, identify sample data whose true type and predicted type do not match as error sample data, and calculate the number of error sample data contained in each true type sample dataset. The distribution acquisition module is used to acquire the confidence distribution data of the sample dataset of each real type; wherein, multiple confidence intervals are preset, and the confidence distribution data is used to characterize the number of sample data of the real type located in each confidence interval; The balance calculation module is used to calculate the balance score of each real type of sample dataset based on the number of error sample data contained in each real type of sample dataset and the confidence distribution data of each real type of sample dataset. The data augmentation module is used to identify the true types whose balance scores are less than a preset balance threshold as target types, and to perform data augmentation processing on the sample dataset of the target type.

8. A voice quality inspection device, characterized in that, include: The acquisition module is suitable for acquiring call text data from agent terminals; The input module is adapted to input the call text data into the voice quality inspection model; The quality inspection module is adapted to determine the quality inspection result of the call text data based on the classification result of the voice quality inspection model. The voice quality inspection model is trained using sample data augmented by the method described in any one of claims 1-5; and the length of the sample data is greater than a preset length threshold.

9. An electronic device, characterized in that, include: At least one processor; A memory that is communicatively connected to the at least one processor; as well as One or more I / O interfaces are connected between the at least one processor and the memory; wherein, The memory stores one or more computer programs that can be executed by the at least one processor. One or more of the computer programs are executed by the at least one processor to enable the at least one processor to perform the method as described in any one of claims 1-6.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, The computer program, when executed by a processor, implements the method as described in any one of claims 1-6.

Images