Optimization method and device of verbal skill classification model, computer equipment and storage medium
By enhancing fixed-point data for samples with wrong classification results in the speech classification model, generating new sample sets, and combining the initial sample set for model training, the problem of insufficient training data in the existing technology is solved and the speech classification is improved Model performance and accuracy.
Patent Information
- Application Number
- CN202411480116.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-22
- Publication Date
- 2025-02-07
AI Technical Summary
There is a lack of a method in the prior art to perform fixed-point data enhancement for samples with incorrect classification results in the speech classification model, resulting in insufficient training data and affecting model performance.
By obtaining the initial sample set and benchmark classification results, using the speech classification model to classify, filter out the quality inspection text with incorrect classification results, and enhance it data to form a new sample set, and finally the initial sample set and the new sample set are added. Combining sets to train models to optimize speech classification models.
A new sample set was generated that was highly similar to the text style and content of the quality inspection text, which improved the quality and diversity of the training data, improved the recognition and understanding ability of the speech classification model, enhanced the classification accuracy, and formed continuous optimization. and the cycle of lifting.
Smart Images
Figure CN119397019A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of natural language processing, and in particular to an optimization method, device, computer equipment and storage medium for a speech classification model. Background Art
[0002] In conversation scenarios, the diversity and complexity of communication speech require speech classification models to have high coverage and recognition accuracy. When the amount of training data is insufficient, the performance of the speech classification model will be affected and cannot meet actual business needs.
[0003] Existing technologies usually increase the amount of training data through data augmentation. However, traditional data augmentation methods have certain limitations. For example, the existing method of adding, deleting, modifying and checking sentences to generate new sentences, or the method of extracting new sentences from the source database using similarity calculation, are all indiscriminate data augmentation for the original samples.
[0004] Therefore, in the prior art, there is a lack of a method that can perform fixed-point data enhancement on samples with incorrect classification results to increase the amount of data to optimize the speech classification model. Summary of the invention
[0005] The present invention provides a method, device, computer equipment and storage medium for optimizing a speech classification model. By enhancing the fixed-point data of samples with erroneous classification results, richer virtual speech that is closer to the real scene is generated to increase the amount of training data, thereby optimizing the speech classification model.
[0006] In a first aspect, a method for optimizing a speech classification model is provided, including:
[0007] Acquire an initial sample set consisting of a plurality of speech texts, and determine a baseline classification result of each speech text in response to manual labeling of each speech text;
[0008] Use the speech classification model to classify the initial sample set and obtain the model classification results of each speech text;
[0009] According to the model classification results and the benchmark classification results, filter out the quality inspection texts with incorrect classification results;
[0010] Perform data enhancement on the quality inspection text to form a new sample set;
[0011] The initial sample set and the newly added sample set are used to train the dialogue classification model to obtain an optimized dialogue classification model.
[0012] In a second aspect, a device for optimizing a speech classification model is provided, comprising:
[0013] A benchmark classification result determination unit, used to obtain an initial sample set consisting of a plurality of speech texts, and determine a benchmark classification result of each speech text in response to manual labeling of each speech text;
[0014] A model classification result determination unit is used to classify the initial sample set using the speech classification model to obtain the model classification result of each speech text;
[0015] A quality inspection text screening unit is used to screen quality inspection texts with incorrect classification results based on the model classification results and the benchmark classification results;
[0016] A new sample set forming unit is added to perform data enhancement on the quality inspection text to form a new sample set;
[0017] The model optimization unit is used to train the speech classification model using the initial sample set and the newly added sample set to obtain an optimized speech classification model.
[0018] In a third aspect, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the above-mentioned method for optimizing the speech classification model when executing the computer program.
[0019] In a fourth aspect, a computer-readable storage medium is provided, which stores a computer program, and when the computer program is executed by a processor, the steps of the optimization method of the above-mentioned speech classification model are implemented.
[0020] In the scheme implemented by the optimization method, device, computer equipment and storage medium of the above-mentioned speech classification model, an initial sample set consisting of multiple speech texts is first obtained, and the baseline classification result of each speech text is determined in response to manual labeling of each speech text; then the initial sample set is classified using the speech classification model to obtain the model classification result of each speech text; then, based on the model classification result and the baseline classification result, the quality inspection texts with incorrect classification results are screened, and the quality inspection texts are data enhanced to form a new sample set; finally, the initial sample set and the new sample set are used to train the speech classification model to obtain the optimized speech classification model. In the present invention, fixed-point data enhancement is performed on quality inspection texts with incorrect classification results, so as to generate a new sample set that is highly similar to the quality inspection text in style and content, thereby improving the quality and diversity of training data; the speech classification model is optimized and trained by introducing the new sample set, so that the optimized speech classification model can better recognize and understand communication speech in different scenarios, thereby improving the accuracy of speech classification; the new sample set is combined with the initial sample set to iteratively train the speech classification model, thereby forming a cyclic process that can continuously optimize and improve the speech classification model, so that the speech classification model can continuously learn from the new sample set and continuously improve its performance; the output result of the optimized speech classification model is more accurate, the model performance is better, it can adapt to complex and changeable speech scenarios, can help users communicate more effectively, and improve the user experience. BRIEF DESCRIPTION OF THE DRAWINGS
[0021] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.
[0022] Figure 1 Schematic diagram of the application environment of the optimization method of the speech classification model in one embodiment of the present invention;
[0023] Figure 2 It is a flowchart of an optimization method of a speech classification model in one embodiment of the present invention;
[0024] Figure 3 It is a flowchart of an optimization method of a speech classification model in another embodiment of the present invention;
[0025] Figure 4 It is a structural schematic diagram of an optimization device for a speech classification model in one embodiment of the present invention;
[0026] Figure 5 is a schematic diagram of the structure of a computer device in one embodiment of the present invention;
[0027] Figure 6 It is a schematic diagram of the structure of a computer device in another embodiment of the present invention. DETAILED DESCRIPTION
[0028] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0029] The optimization method of the speech classification model provided by the embodiment of the present invention can be applied to Figure 1 In an application environment, a client communicates with a server through a network.
[0030] The client is used to indicate the party that has the authority to label the benchmark classification results. The client can usually be a specific business person or developer. The client can be, but is not limited to, a device with a display and input device such as a personal computer, a laptop, a smart phone, a tablet computer, and a portable wearable device. The person can receive an initial sample set consisting of multiple speech texts through the client, and feed back the results of each manually labeled speech text to the server.
[0031] The server can be implemented with an independent server or a service platform composed of multiple servers. The server first obtains an initial sample set consisting of multiple speech texts, transmits the initial sample set to the client, and determines the benchmark classification result based on the labeling result of each speech text fed back by the client; then the speech classification model is used to classify the initial sample set to obtain the model classification result of each speech text; then, based on the model classification result and the benchmark classification result, the quality inspection texts with incorrect classification results are filtered out, and the quality inspection texts are data enhanced to form a new sample set; finally, the initial sample set and the new sample set are used to train the speech classification model to obtain the optimized speech classification model.
[0032] In the present invention, fixed-point data enhancement is performed on quality inspection texts with incorrect classification results, so as to generate a new sample set that is highly similar to the quality inspection text in style and content, thereby improving the quality and diversity of training data; the speech classification model is optimized and trained by introducing the new sample set, so that the optimized speech classification model can better recognize and understand communication speech in different scenarios, thereby improving the accuracy of speech classification; the new sample set is combined with the initial sample set to iteratively train the speech classification model, thereby forming a cyclic process that can continuously optimize and improve the speech classification model, so that the speech classification model can continuously learn from the new sample set and continuously improve its performance; the output result of the optimized speech classification model is more accurate, the model performance is better, it can adapt to complex and changeable speech scenarios, can help users communicate more effectively, and improve the user experience.
[0033] The present invention is described in detail below through specific embodiments.
[0034] The purpose of the present invention is to optimize the dialogue classification model. The application scenarios of the dialogue classification model are briefly introduced.
[0035] The speech classification model can be applied to various scenarios, various intelligent customer services or intelligent assistants and other intelligent question-answering engines. Intelligent question-answering engines are usually implemented through the background. The background can receive the user speech of the front-end connected to it in real time, classify the user speech through the speech classification model to determine the classification result, and call the corresponding intelligent assistant to give accurate feedback based on the classification result.
[0036] The speech classification model can be applied to different fields. When the speech classification model is applied to different fields, it can have different target classification results according to different classification standards.
[0037] For example, the speech classification model can be a model that classifies results based on business. For example, for group business, the speech classification model classifies the received user speech into insurance business, medical business, shopping business, etc., and then, based on the classification results of the speech classification model, transfers users of insurance business to the corresponding insurance smart assistant, transfers users of medical business to the corresponding medical smart assistant, and transfers users of shopping business to the corresponding shopping smart assistant, so as to provide feedback on the user speech.
[0038] For another example, the speech classification model can be a model with emotion as the target classification result. For example, for insurance business, the speech classification model classifies the received user speech into positive emotion and negative emotion, and then based on the classification result of the speech classification model, transfers users with positive emotion to regular intelligent customer service and transfers users with negative emotion to complaint intelligent customer service to provide feedback on the user speech.
[0039] For another example, for medical services, the speech classification model can be a model with disease as the target classification result. For example, the speech classification model classifies the received user speech into respiratory diseases, gastrointestinal diseases, cardiovascular and cerebrovascular diseases, endocrine diseases, etc., and then transfers the user to the corresponding consultation intelligent assistant based on the classification result of the speech classification model to provide feedback on the user's speech.
[0040] The above examples are only for illustration and are not intended to limit the scope of the present invention. The present invention does not specifically limit the speech classification model, that is, no matter which specific field or classification standard the speech classification model is applied to, it can be optimized using the method provided by the present invention.
[0041] See also Figure 2 As shown, Figure 2 A flowchart of a method for optimizing a speech classification model provided by an embodiment of the present invention includes the following steps:
[0042] Step S210: Acquire an initial sample set consisting of a plurality of speech texts, and determine a baseline classification result for each speech text in response to manual labeling of each speech text.
[0043] The initial sample set may be a collection of user speech texts that are consistent with the application domain of the speech classification model. For example, when the speech classification model is for group business, the initial sample set may include user speech texts covering all types of business of the group. For another example, when the speech classification model is for insurance business, the initial sample set may include user speech texts related to insurance business.
[0044] The initial sample set may be a collection of user speech texts within a specific time range, and the specific time range may be flexibly determined according to actual conditions. For example, when the speech classification model is optimized and updated for the first time, the initial sample set may be a collection of user speech texts within the latest period of time (such as a week, a month, etc.). For another example, when the speech classification model is continuously optimized and updated, the initial sample set may be a collection of user speech texts within the range from the last optimization of the speech classification model to the current time.
[0045] The user speech may be a text speech or a voice speech. In the case where the user speech is a voice speech, the voice speech may be converted into a text speech to serve as the user speech text. The method of converting the voice speech into the text speech is not specifically limited.
[0046] After the server obtains the initial sample set consisting of multiple speech texts, it can provide the initial sample set to the client, so that the client's business personnel or developers can mark the classification results of each speech text. After receiving the marking results returned by the client, the server determines the benchmark classification result of each speech text based on the marking results. The benchmark classification result is the correct classification result of each speech text based on manual marking.
[0047] For example, the server obtains a text of speech "How much did you say seat insurance cost per person last time?" and provides it to the client. The business personnel labels the text of speech "insurance" through the client and feeds it back to the server. Based on the labeling result, the server determines that the baseline classification result of "How much did you say seat insurance cost per person last time?" is "insurance business."
[0048] Step S220, using the speech classification model to classify the initial sample set, and obtain the model classification result of each speech text.
[0049] The speech classification model may be a pre-trained model that can be applied to actual fields. The speech classification model may include, but is not limited to, a naive Bayes classifier, a decision tree, a support vector machine, a neural network, and the like.
[0050] Although pre-trained, the speech classification model usually cannot provide a "perfect" output result, that is, the speech classification model may not be able to correctly classify every speech text. In particular, as time goes by, more and more emerging words may appear in user speech, and the pre-trained speech classification model may not be able to accurately classify the latest user speech text. The purpose of optimizing the speech classification model of the present invention is to expect the optimized speech classification model to be as close to providing a "perfect" output result as possible.
[0051] After the initial sample set is input into the speech classification model, the speech classification model is used to output the model classification result of each speech text. The model classification result only represents the "classification answer" output by the speech classification model, that is, the actual classification result, which may be a correct classification result or an incorrect classification result.
[0052] For example, the server inputs a text of the speech "How much did you say seat insurance cost per person last time?" into the speech classification model, and the model classification result output by the speech classification model is "finance". Then "financial business" is the actual classification result of the speech text.
[0053] Step S230, screening out quality inspection texts with incorrect classification results based on the model classification results and the benchmark classification results.
[0054] After manual labeling, the benchmark classification results clearly define the correct classification results for each speech text. After the speech classification model, the model classification results clearly define the actual classification results for each speech text given by the speech classification model.
[0055] When the model classification result of a certain speech text is inconsistent with the benchmark classification result, it means that the model classification result is an incorrect classification result. In the initial sample set, all speech texts whose model classification results are inconsistent with the benchmark classification results are determined as quality inspection texts.
[0056] For example, the baseline classification result of the script "How much did you say seat insurance cost per person last time?" is "insurance business", and the actual classification result is "financial business", so this script text will be included in the quality inspection text.
[0057] Step S240: perform data enhancement on the quality inspection text to form a new sample set.
[0058] In the present invention, fixed-point data enhancement is performed on the quality inspection text to generate a generated speech text that is highly similar to the quality inspection text in style and content, forming a new sample set, thereby improving the quality and diversity of the training data. By introducing a new sample set to optimize the training speech classification model, the optimized speech classification model can better identify and understand the communication speech in different scenarios, thereby improving the accuracy of speech classification.
[0059] When performing data enhancement, you can, but are not limited to, using a method of adding, deleting, modifying, and checking sentences to generate new sentences, or using a similarity calculation to extract new sentences from a source database, or using a large model to generate new sentences.
[0060] For example, for a quality inspection text "How much did you say seat insurance cost per person last time?", through data enhancement, the following new samples can be generated:
[0061] How much is the seat insurance mentioned last time for one person?
[0062] Could you tell me how much the premium for one person is for the seat insurance you mentioned last time?
[0063] What is the coverage for one person for the seat insurance mentioned last time?
[0064] May I ask how much is the coverage amount for one person for the seat insurance mentioned last time?
[0065] What is the coverage of the seat insurance mentioned last time for one person?
[0066] Step S250, using the initial sample set and the newly added sample set to train the speech classification model to obtain an optimized speech classification model.
[0067] After the new sample set is obtained by data enhancement of the quality inspection text, the new sample set is merged with the initial sample set to form an updated sample set. Then, the updated sample set is used to iteratively train the speech classification model to obtain an optimized speech classification model.
[0068] The new sample set is combined with the initial sample set to iteratively train the speech classification model, forming a cycle process that can continuously optimize and improve the speech classification model, so that the speech classification model can continuously learn from the new sample set and continuously improve its performance. The optimized speech classification model can be closer to providing a "perfect" output result with higher accuracy.
[0069] pass Figure 2 The method shown, the present invention obtains an initial sample set consisting of multiple speech texts, determines the baseline classification result of each speech text in response to manual labeling of each speech text; uses the speech classification model to classify the initial sample set to obtain the model classification result of each speech text; according to the model classification result and the baseline classification result, screens the quality inspection texts with incorrect classification results; performs data enhancement on the quality inspection texts to form a new sample set; uses the initial sample set and the new sample set to train the speech classification model to obtain an optimized speech classification model. The speech classification model optimized based on the method provided in this application has more accurate output results and better model performance, can adapt to complex and changeable speech scenarios, can help users communicate more effectively, and improve the user experience.
[0070] In some optional embodiments, S210 obtains an initial sample set consisting of multiple speech texts, and determines a benchmark classification result of each speech text in response to manual labeling of each speech text, including: obtaining an initial sample set consisting of multiple speech texts, and displaying a benchmark classification labeling control corresponding to each speech text; receiving an input result for each benchmark classification labeling control, and determining a benchmark classification result for each speech text.
[0071] After the server obtains multiple speech texts, it can generate a corresponding benchmark classification labeling control for each speech text. When the trigger condition is met (such as obtaining a client receiving instruction, or entering a display cycle, etc.), the benchmark classification labeling control is displayed to the client.
[0072] In the benchmark classification marking control, preset marking options can be provided for the client's business personnel or developers to choose from; in the benchmark classification marking control, a marking content entry window can also be provided for the client's staff or developers to enter.
[0073] Business personnel or developers use the client to mark in the benchmark classification marking control and feed the marking results back to the server. After receiving the marking results, the server determines the benchmark classification results for each speech text.
[0074] In some optional embodiments, the S220 speech classification model includes: an input embedding layer, a position encoding layer, an encoder and a classifier; wherein the encoder is composed of a multi-head self-attention mechanism and a feedforward neural network; the initial sample set is classified using the speech classification model to obtain a model classification result for each speech text, including: inputting each speech text into the input embedding layer to obtain a first feature; obtaining the sequence information of each speech text through the position encoding layer; superimposing the first feature and the sequence information and inputting them into the encoder to obtain a second feature; inputting the second feature into the classifier to obtain a model classification result for each speech text.
[0075] The speech classification model proposed in the present invention can be divided into an input embedding layer, a position encoding layer, an encoder composed of a multi-head self-attention mechanism and a feedforward neural network, and a classifier according to the Transformer model structure.
[0076] The input embedding layer performs a series of convolution operations on the input speech text sequence features to capture the first feature of the speech text. The first feature may include, but is not limited to: character vector feature, word vector feature, part-of-speech tag, etc.
[0077] Since the Transformer model itself does not have the ability to process sequence position information, it is necessary to introduce position information into the model through the position encoding layer. The position encoding layer usually uses sine and cosine functions for calculation. The sequence information of each speech text is obtained through the position encoding layer.
[0078] Positional encoding:
[0079]
[0080] Among them, pos represents the position index, i represents the dimension index, and d model Represents the dimension of the Transformer model.
[0081] The self-attention mechanism consists of a multi-head self-attention and a feedforward neural network, with residual connections and normalization layers between them. The encoder composed of a multi-head self-attention mechanism and a feedforward neural network can well draw the global dependency between input and output.
[0082] The calculation process of the self-attention mechanism can be divided into the following steps:
[0083] Calculate the query matrix Q(Query), key matrix K(Key) and value matrix V(Value):
[0084] Q=XW Q ;
[0085] K=XW K ;
[0086] V=XW V ;
[0087] Among them, X represents the embedded representation of the input sequence, W Q , W K , W V They represent the learnable weight matrices respectively.
[0088] Calculate the attention weights:
[0089]
[0090] Among them, d k represents the dimension of the key matrix K.
[0091] Multi-head self-attention divides the input sequence into multiple "heads", each "head" uses a different weight matrix to calculate self-attention, and then concatenates the results and processes them through a linear layer.
[0092] Output of multi-head self-attention:
[0093] MultiHead(Q,K,V)=Concat(head1,...,head h )WO;
[0094] Among them, head i represents the output of the i-th “head”, and WO represents the learnable weight matrix.
[0095] Each multi-head self-attention layer is followed by a feed-forward neural network for further nonlinear transformation of the multi-head self-attention output.
[0096] Feedforward Neural Network:
[0097] FFN(x)=max(0,xW1+b1)W2+b2;
[0098] Among them, W1 and W2 represent learnable weight matrices, b1 and b2 represent bias vectors, and x represents the output of the previous layer.
[0099] After the first feature is superimposed with the sequential information, the first feature is input into the encoder as the embedding representation X of the input sequence, thereby obtaining the second feature, which may be a context vector.
[0100] The second feature is input into the classifier to output the model classification result of the speech text.
[0101] Let the initial sample set be X, and the model classification result be Y. The classification process of the speech classification model can be expressed by the following formula:
[0102] Y = f(X);
[0103] Among them, f represents the speech classification model.
[0104] In some optional implementations, S230 screens quality inspection texts with incorrect classification results based on the model classification results and the benchmark classification results, including: determining in turn whether the model classification results of each speech text are consistent with the benchmark classification results; and determining the speech texts whose model classification results are inconsistent with the benchmark classification results as quality inspection texts.
[0105] For each speech text in the initial sample set, determine whether its model classification result is consistent with the benchmark classification result. If the model classification result of a speech text is inconsistent with the benchmark classification result, it means that the output of the model is an incorrect result and data enhancement is required for the speech text. Therefore, the speech text is determined as one of the quality inspection texts. In the initial sample set, all speech texts whose model classification results are inconsistent with the benchmark classification results constitute the quality inspection text.
[0106] In some optional implementations, S240 performs data enhancement on the quality inspection text to form a new sample set, including: using a large model to perform data enhancement on the quality inspection text to obtain a generated speech text; and associating the generated speech text with a benchmark classification result of the corresponding quality inspection text.
[0107] Specifically, a big model is used to perform data enhancement on the quality inspection text to obtain a generated speech text, including: extracting feature fragments from the quality inspection text according to the benchmark classification results of the quality inspection text; inputting the feature fragments into the big model to transform the feature fragments through the big model; and generating multiple generated speech texts corresponding to the quality inspection text according to the transformed feature fragments.
[0108] For each line of text in the quality inspection text, a large model is used to generate text.
[0109] The quality inspection text is recorded as X arr , the benchmark classification result of each speech text in the quality inspection text is recorded as Y true , and the generated speech text is recorded as X aug , then the process of data enhancement can be expressed by the following formula:
[0110] X aug =g(X err , Y true );
[0111] Among them, g represents the large model.
[0112] In order to generate a generated speech text that is highly similar to the style and content of the quality inspection text, feature fragments can be extracted from the quality inspection text through the scene appeal label (i.e., the benchmark classification result of the quality inspection text). Due to the introduction of the benchmark classification result, the feature fragment has the tendency of the benchmark classification result. After the feature fragment is input into the big model, the feature fragment is transformed by the big model, and then the generated speech text corresponding to multiple quality inspection texts is generated based on the transformation results.
[0113] For example, for a quality inspection text "How much did you say seat insurance cost per person last time?", based on the baseline classification result of "insurance business", the feature segments "seat insurance" and "money" are extracted. After the feature segments are input into the big model, the big model can transform the feature segments into "seat insurance" and so on, and "money" can be transformed into "expenses", "premiums", "insurance amounts", "guaranteed amounts", "coverage ranges", etc. Therefore, the generated speech texts corresponding to multiple quality inspection texts are spliced and generated according to the transformation results, for example:
[0114] How much is the seat insurance mentioned last time for one person?
[0115] Could you tell me how much the premium for one person is for the seat insurance you mentioned last time?
[0116] What is the coverage for one person for the seat insurance mentioned last time?
[0117] May I ask how much is the coverage amount for one person for the seat insurance mentioned last time?
[0118] What is the coverage of the seat insurance mentioned last time for one person?
[0119] All generated speech texts constitute a new sample set. Each generated speech text in the new sample set is associated with the benchmark classification result of the corresponding quality inspection text, so as to determine the benchmark classification result of each generated speech text.
[0120] The present invention uses a large model to perform fixed-point data enhancement on quality inspection texts, and can generate generated speech texts that are highly similar to the quality inspection texts in style and content, thereby improving the quality and diversity of training data.
[0121] Compared with the traditional addition, deletion, modification and query method, the generated speech text formed by the method provided by the present invention is more accurate, richer in content, and closer to the real scene. The traditional method of extracting new sentences from the source database using similarity calculation usually requires setting a threshold to decide when to extract new sentences, and the selection of the threshold is highly subjective. The present invention reduces the dependence on the threshold and reduces the risk of inaccurate extraction of new sentences due to improper threshold selection by using a large model to form the generated speech text.
[0122] Introducing more data enhancement samples enables the speech classification model to learn a wider range of features, thereby enhancing the generalization ability of the speech classification model. The optimized speech classification model can better identify and understand communication speech in speech classification applications, improving classification accuracy.
[0123] In some optional implementations, S250 uses the initial sample set and the newly added sample set to train the speech classification model to obtain an optimized speech classification model, including: using the initial sample set and the newly added sample set as the updated sample set; dividing the updated sample set into a training set, a verification set and a test set; using the training set to iteratively train the speech classification model, using the verification set to verify the trained speech classification model, and using the test set to test the verified speech classification model to obtain the optimized speech classification model.
[0124] Combine the initial sample set and the newly added sample set into the updated sample set, and divide the updated sample set into the training set, test set, and validation set. Use the training set to train the speech classification model; evaluate the performance indicators of the speech classification model based on the validation set, understand the performance of the speech classification model on the validation set, and adjust the model parameters or fine-tune the model according to the performance of the speech classification model; test the performance and generalization ability of the speech classification model based on the test set. When the performance of the speech classification model meets the requirements, the optimized speech classification model is obtained.
[0125] The initial sample set X and the newly added sample set X aug Merge into updated sample set X new , the sample set X will be updated new Divide into training set D new , validation set Y new and test set C new . Then the iterative training process can be expressed by the following formula:
[0126] f new =Φ(D new );
[0127] Among them, Φ represents the iterative training process, f new Represents the trained speech classification model.
[0128] The present invention feeds back the generated speech text to the initial sample set to form an updated sample set for iterative training of the speech classification model, thereby forming a cyclic process that can be continuously optimized and improved. The speech classification model can continuously relearn from new training data, thereby continuously improving the performance of the speech classification model.
[0129] The present invention provides higher quality training data so that the optimized speech classification model can better adapt to complex and changeable communication scenarios, thereby improving the robustness of the speech classification model.
[0130] The speech classification model optimized by the above method improves classification accuracy, so that the optimized speech classification model can be applied to various fields for speech classification. By adopting the optimized speech classification model, users can communicate with customers more effectively, improve communication efficiency, and thus enhance communication experience and service satisfaction.
[0131] See also Figure 3 As shown, Figure 3 Another flowchart of the method for optimizing the speech classification model provided by an embodiment of the present invention includes the following steps:
[0132] Step S301, obtaining multiple speech texts after the optimization of the "last optimized speech classification model" as an initial sample set; then proceed to step S302 and step S310;
[0133] Step S302, displaying the benchmark classification labeling control corresponding to each speech text to the client; then proceeding to step S303;
[0134] Step S303, receiving the input result of each benchmark classification labeling control from the client, and determining the benchmark classification result of each speech text; then proceeding to step S304;
[0135] Step S304, input each speech text into the "speech classification model after the last optimization" to obtain the model classification result of each speech text; wherein the speech classification model includes an input embedding layer, a position encoding layer, an encoder composed of a multi-head self-attention mechanism and a feedforward neural network, and a classifier; go to step S305;
[0136] Step S305, determining in turn whether the model classification result of each speech text is consistent with the benchmark classification result, and determining the speech text whose model classification result is inconsistent with the benchmark classification result as the quality inspection text; then proceed to step S306;
[0137] Step S306, extracting characteristic segments from the quality inspection text according to the benchmark classification result of the quality inspection text; then proceeding to step S307;
[0138] Step S307, input the feature fragment into the large model to transform the feature fragment through the large model; then go to step S308;
[0139] Step S308, generating multiple generated speech texts corresponding to the quality inspection texts according to the transformed feature segments to form a newly added sample set; then proceed to step S309;
[0140] Step S309, associating the generated speech text with the benchmark classification result of the corresponding quality inspection text; then proceeding to step S310;
[0141] Step S310, taking the initial sample set and the newly added sample set as the updated sample set, dividing the updated sample set into a training set, a validation set and a test set; then go to step S311;
[0142] Step S311, iteratively train the "last optimized speech classification model" using the training set, verify the trained "last optimized speech classification model" using the verification set, and test the verified "last optimized speech classification model" using the test set to obtain the "current optimized speech classification model"; go to step S312;
[0143] Step S312, use the "speech classification model optimized this time" as the "speech classification model after the previous optimization" compared with the "speech classification model optimized next time"; go to step S304.
[0144] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.
[0145] In one embodiment, a device for optimizing a speech classification model is provided, and the device for optimizing a speech classification model corresponds one-to-one to the method for optimizing a speech classification model in the above embodiment. Figure 4 As shown, the optimization device of the speech classification model includes:
[0146] A benchmark classification result determination unit 401 is used to obtain an initial sample set consisting of a plurality of speech texts, and determine a benchmark classification result of each speech text in response to manual labeling of each speech text;
[0147] The model classification result determination unit 402 is used to classify the initial sample set using the speech classification model to obtain the model classification result of each speech text;
[0148] A quality inspection text screening unit 403 is used to screen quality inspection texts with incorrect classification results according to the model classification results and the benchmark classification results;
[0149] A new sample set forming unit 404 is used to perform data enhancement on the quality inspection text to form a new sample set;
[0150] The model optimization unit 405 is used to train the speech classification model using the initial sample set and the newly added sample set to obtain an optimized speech classification model.
[0151] In some optional implementations, the benchmark classification result determination unit 401 is specifically used to: obtain an initial sample set consisting of multiple speech texts, and display a benchmark classification labeling control corresponding to each speech text; receive an input result for each benchmark classification labeling control, and determine the benchmark classification result of each speech text.
[0152] In some optional embodiments, the speech classification model includes: an input embedding layer, a position encoding layer, an encoder and a classifier, wherein the encoder is composed of a multi-head self-attention mechanism and a feedforward neural network; a model classification result determination unit 402 is specifically used to: input each speech text into the input embedding layer to obtain a first feature; obtain the sequence information of each speech text through the position encoding layer; superimpose the first feature and the sequence information and input them into the encoder to obtain a second feature; input the second feature into the classifier to obtain the model classification result of each speech text.
[0153] In some optional implementations, the quality inspection text screening unit 403 is specifically used to: determine in turn whether the model classification result of each speech text is consistent with the benchmark classification result; and determine the speech text whose model classification result is inconsistent with the benchmark classification result as the quality inspection text.
[0154] In some optional implementations, a new sample set forming unit 404 is specifically used to: use a large model to perform data enhancement on the quality inspection text to obtain a generated speech text; and associate the generated speech text with a benchmark classification result of the corresponding quality inspection text.
[0155] In some optional implementations, the newly added sample set forming unit 404 is also specifically used to: extract feature fragments from the quality inspection text according to the benchmark classification results of the quality inspection text; input the feature fragments into the big model to transform the feature fragments through the big model; and generate multiple generated speech texts corresponding to the quality inspection text according to the transformed feature fragments.
[0156] In some optional implementations, the model optimization unit 405 is specifically used to: use the initial sample set and the newly added sample set as the updated sample set; divide the updated sample set into a training set, a verification set and a test set; use the training set to iteratively train the speech classification model, use the verification set to verify the trained speech classification model, and use the test set to test the verified speech classification model to obtain an optimized speech classification model.
[0157] For the specific definition of the optimization device of the speech classification model, please refer to the definition of the optimization method of the speech classification model in the above text, which will not be repeated here. Each module in the above-mentioned optimization device of the speech classification model can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.
[0158] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 5 As shown. The computer device includes a processor, a memory, a network interface and a database connected via a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile and / or volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external client through a network connection. When the computer program is executed by the processor, it implements the functions or steps on the server side of a method for optimizing a speech classification model.
[0159] In one embodiment, a computer device is provided. The computer device may be a client, and its internal structure diagram may be as follows: Figure 6 As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected via a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external server via a network connection. When the computer program is executed by the processor, it implements the functions or steps on the client side of a method for optimizing a speech classification model
[0160] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program:
[0161] Acquire an initial sample set consisting of a plurality of speech texts, and determine a baseline classification result of each speech text in response to manual labeling of each speech text;
[0162] Use the speech classification model to classify the initial sample set and obtain the model classification results of each speech text;
[0163] According to the model classification results and the benchmark classification results, filter out the quality inspection texts with incorrect classification results;
[0164] Perform data enhancement on the quality inspection text to form a new sample set;
[0165] The initial sample set and the newly added sample set are used to train the dialogue classification model to obtain an optimized dialogue classification model.
[0166] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:
[0167] Acquire an initial sample set consisting of a plurality of speech texts, and determine a baseline classification result of each speech text in response to manual labeling of each speech text;
[0168] Use the speech classification model to classify the initial sample set and obtain the model classification results of each speech text;
[0169] According to the model classification results and the benchmark classification results, filter out the quality inspection texts with incorrect classification results;
[0170] Perform data enhancement on the quality inspection text to form a new sample set;
[0171] The initial sample set and the newly added sample set are used to train the dialogue classification model to obtain an optimized dialogue classification model.
[0172] It should be noted that the above functions or steps that can be implemented by the computer-readable storage medium or computer device can refer to the relevant descriptions on the server side and the client side in the aforementioned method embodiment. To avoid repetition, they will not be described one by one here.
[0173] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
[0174] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.
[0175] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.
Claims
1. A method for optimizing a speech classification model, characterized in that: include: Acquire an initial sample set consisting of a plurality of speech texts, and determine a baseline classification result of each of the speech texts in response to manual labeling of each of the speech texts; Using the speech classification model to classify the initial sample set, and obtain the model classification result of each speech text; According to the model classification result and the benchmark classification result, filter out quality inspection texts with incorrect classification results; Performing data enhancement on the quality inspection text to form a new sample set; The initial sample set and the newly added sample set are used to train the speech classification model to obtain the optimized speech classification model.
2. The method for optimizing the speech classification model according to claim 1, characterized in that: The obtaining of an initial sample set consisting of a plurality of speech texts, and determining a benchmark classification result of each speech text in response to manual labeling of each speech text, includes: Obtain the initial sample set consisting of a plurality of the speech texts, and display a benchmark classification labeling control corresponding to each of the speech texts; Receive the input result of each of the benchmark classification marking controls, and determine the benchmark classification result of each of the speech texts.
3. The method for optimizing the speech classification model according to claim 1, characterized in that: The speech classification model includes: an input embedding layer, a position encoding layer, an encoder and a classifier; wherein the encoder is composed of a multi-head self-attention mechanism and a feedforward neural network; The initial sample set is classified by using the speech classification model to obtain the model classification result of each speech text, including: After inputting each of the speech texts into the input embedding layer, a first feature is obtained; Obtaining the sequence information of each of the speech texts through the position coding layer; The first feature and the sequence information are superimposed and input into the encoder to obtain a second feature; The second feature is input into the classifier to obtain a model classification result for each of the speech texts.
4. The method for optimizing the speech classification model according to claim 1, characterized in that: The step of screening quality inspection texts with incorrect classification results according to the model classification results and the benchmark classification results includes: Determine in sequence whether the model classification result of each of the speech texts is consistent with the benchmark classification result; The speech text whose model classification result is inconsistent with the benchmark classification result is determined as the quality inspection text.
5. The method for optimizing the speech classification model according to claim 1, characterized in that: The data enhancement of the quality inspection text to form a new sample set includes: A large model is used to perform data enhancement on the quality inspection text to obtain a generated speech text; The generated speech text is associated with the benchmark classification result of the corresponding quality inspection text.
6. The method for optimizing the speech classification model according to claim 5, characterized in that: The method of using a large model to perform data enhancement on the quality inspection text to generate a speech text includes: Extracting feature segments from the quality inspection text according to the benchmark classification result of the quality inspection text; Inputting the feature fragment into the large model to transform the feature fragment through the large model; Based on the transformed feature fragments, a plurality of generated speech texts corresponding to the quality inspection texts are generated.
7. The method for optimizing the speech classification model according to claim 1, characterized in that: The using the initial sample set and the newly added sample set to train the speech classification model to obtain the optimized speech classification model includes: Using the initial sample set and the newly added sample set as updated sample sets; Dividing the updated sample set into a training set, a validation set and a test set; The speech classification model is iteratively trained using the training set, the trained speech classification model is verified using the verification set, and the verified speech classification model is tested using the test set to obtain the optimized speech classification model.
8. An optimization device for a speech classification model, characterized in that: include: A benchmark classification result determination unit, used to obtain an initial sample set consisting of a plurality of speech texts, and determine a benchmark classification result of each of the speech texts in response to manual labeling of each of the speech texts; A model classification result determination unit, used to classify the initial sample set using a speech classification model to obtain a model classification result for each speech text; A quality inspection text screening unit, used for screening quality inspection texts with incorrect classification results according to the model classification results and the benchmark classification results; A new sample set forming unit is used to perform data enhancement on the quality inspection text to form a new sample set; The model optimization unit is used to train the speech classification model using the initial sample set and the newly added sample set to obtain the optimized speech classification model.
9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, it implements the steps of the method for optimizing the speech classification model as described in any one of claims 1 to 7.
10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method for optimizing the speech classification model as described in any one of claims 1 to 7 are implemented.