Text data augmentation for text classification using weakly supervised multi-reward reinforcement learning

Abstract

To provide a system and method that enable rapid and cost-effective human-in-the-loop synthesis of domain-specific textual training data for a deep learning model.SOLUTION: A data augmentation process incorporates a sentence generator, a sentence classifier, and weak-supervision by a domain expert that is "in the loop." Generally, both sentence generator and sentence classifier are implemented as machine learning models. The sentence generator generates new sentences based on manually labeled sentences and the sentence classifier generates labels for the newly generated sentences. The new sentences are corrected or verified by the domain expert and then used to retrain one or both of the sentence generator and the sentence classifier.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The systems and methods disclosed herein relate to text classification, and more particularly, to enhancing text training data for text classification.

Background Art

[0002] Background Unless otherwise indicated in this specification, the materials described in this section are not to be regarded as prior art merely by virtue of their inclusion in this section.

[0003] Information extraction (IE) is the process or task of obtaining knowledge by extracting and compiling structured information from unstructured text. Information extraction enables the efficient and effective utilization of information from unstructured text by downstream applications such as question answering applications. In some information extraction tasks, the most appropriate information unit for conveying a particular type of knowledge is a single sentence. For example, the sentence "The engine of this car is very noisy. (The engine of this car is very noisy.)" describes a problem related to a car. In this example, the entire sentence is the ideal unit for describing a problem related to a car because individual phrases (e.g., "the engine (engine)" or "very noisy (very noisy)"), which are parts of the sentence, can only convey the problem partially. Therefore, classifying the meaning of the entire sentence can be considered an information extraction task that can be used to form the basis of structured knowledge of sentences with a target meaning or classification (e.g., car problems).

[0004] Text classification has been widely studied in the field of natural language processing (NLP). Similar to other natural language processing tasks, prior art for text classification generally employs deep learning models. Deep learning models have been shown to achieve high performance in some benchmark datasets compared to conventional models such as rule-based algorithms or conventional feature-rich machine learning models. Benchmark datasets generally include a set of texts annotated with text-level labels, and some domain-specific benchmark datasets are created using domain-specific labels related to a specific domain. However, since these existing domain-specific benchmark datasets do not necessarily correspond to the domain of interest, deep learning models trained on these datasets cannot be directly applied to other specific domain-specific tasks of interest. Therefore, in many domain-specific tasks, it is necessary to generate a new domain-specific dataset to train a text classification model based on deep learning for the task. SUMMARY OF THE INVENTION PROBLEMS TO BE SOLVED BY THE INVENTION

[0005] Therefore, the problems of domain-specific tasks that require text classification are the same as those of any supervised deep learning model. Supervised deep learning models rely on a large amount of training data that is manually labeled for the teacher. This model tends to overfit with a small amount of training data and does not generalize well, so it is insufficient with a small amount of training data. However, since manual annotation of texts by domain experts is often very costly in practice, it is difficult to create a sufficiently large amount of training data through human intervention. Therefore, it is advantageous to provide a technique for more easily and cost-effectively creating a sufficiently large domain-specific dataset for a text classification model. MEANS FOR SOLVING THE PROBLEMS

[0006] Overview A method for generating labeled text training data is disclosed. The method includes a processor receiving a first unit of text data. The method further includes a processor generating a second unit of text data based on the first unit of text data using a first machine learning model. The method further includes a processor determining labels describing features of the second unit of text data using a second machine learning model. The method further includes an output device outputting the second unit of text data and the labels to a user. The method further includes receiving, via a user interface, either (i) a modification to the second unit of text data or a validation of the second unit of text data, or (ii) a modification to the labels or a validation of the labels. The method further includes a processor retraining a second machine learning model using either (i) the modified second unit of text data or the validated second unit of text data, or (ii) the modified labels or the validated labels.

[0007] A system for generating labeled text training data is disclosed. The system includes an output device. The system further includes a user interface. The system further includes a memory configured to store a plurality of units of text data, wherein each unit of text data in the plurality of units of text data has a label describing the features of each unit of text data. The system further includes a processor operably connected to the output device, the user interface and the memory. The processor is configured to read a first unit of text data from a plurality of units of text data stored in the memory. The processor is further configured to generate a second unit of text data based on the first unit of text data using a first machine learning model. The processor is further configured to determine a label describing the features of the second unit of text data using a second machine learning model. The processor is further configured to operate the output device to output the second unit of text data and the label to the user. The processor is further configured to operate the user interface to receive either (i) corrections to the second unit of text data and validation of the second unit of text data, and (ii) corrections to the label and validation of the label. The processor is further configured to retrain a second machine learning model using (i) a second unit of the corrected text data and a second unit of the validated text data, and (ii) a second unit of the corrected labels and a validated labels.

[0008] A non-temporary computer-readable medium for generating labeled text training data is disclosed. The computer-readable medium stores program instructions that, when executed by a processor, cause the processor to receive a first unit of text data. The computer-readable medium further stores program instructions that, when executed by a processor, cause the processor to generate a second unit of text data based on the first unit of text data using a first machine learning model. The computer-readable medium further stores program instructions that, when executed by a processor, cause the processor to determine a label describing the features of the second unit of text data using a second machine learning model. The computer-readable medium further stores program instructions that, when executed by a processor, cause the processor to operate an output device to output the second unit of text data and the label to the user. The computer-readable medium further stores program instructions that, when executed by a processor, cause the processor to operate a user interface to receive either (i) a modification to the second unit of text data or a verification of the second unit of text data, and (ii) a modification to the label or a verification of the label. This computer-readable medium also stores program instructions that, when executed by the processor, cause the processor to further retrain a second machine learning model using (i) a second unit of modified text data and a second unit of verified text data, and (ii) a second unit of modified labels and a verified labels. [Brief explanation of the drawing]

[0009] The above-described aspects and other features of this system and method will be explained below in reference to the attached drawings.

[0010] [Figure 1] This figure shows a system and method for synthesizing domain-specific training data for deep learning models. [Figure 2]This figure shows an exemplary embodiment of a text generator for generating new, uncertain, diverse, and representative high-quality texts. [Figure 3] This is a block diagram of an exemplary embodiment of a data augmentation system for synthesizing newly labeled text data. [Figure 4] This is a flowchart showing how to operate a data augmentation system to synthesize newly labeled text data. [Modes for carrying out the invention]

[0011] Detailed explanation Hereinafter, embodiments illustrated in the drawings and described in the following specification will be referenced for the purpose of facilitating an understanding of the principles of this disclosure. It should be understood that these embodiments are not intended to limit the scope of this disclosure. Furthermore, it should be understood that this disclosure includes any changes and modifications to the illustrated embodiments, and includes further applications of the principles of this disclosure that are common to those skilled in the art relating to this disclosure.

[0012] Overview of Systems and Methods for Data Augmentation Figure 1 illustrates a system and method that enables rapid and cost-effective human-in-the-loop synthesis of domain-specific training data for deep learning models. The system and method are described herein in relation to generating domain-specific training data for domain-specific text classification models. In particular, the exemplary domain-specific text classification model described herein is configured to classify entire texts as either (1) texts describing an automotive problem or (2) texts not describing an automotive problem. This text classification model could be a key component in an intelligent automotive aftermarket support service, for example, to help a user effectively and efficiently identify texts describing specific problems and solutions related to automotive repair.

[0013] However, it should be understood that the data augmentation systems and methods described herein are applicable not only to the synthesis of labeled text data in any domain, but also to the synthesis of non-domain-specific labeled text data. Similarly, it should be understood that the data augmentation systems and methods described herein are applicable not only to labeled text data for text classification models, but also to the synthesis of labeled text data for any deep learning model. Finally, although the systems and methods are exemplified as systems for augmenting English text data, it should be understood that they are, in principle, language-independent and applicable to any natural language.

[0014] Referring to Figure 1, the data augmentation process begins with a small set of training data 10 manually labeled by a domain expert 20. This small set of training data 10 is then used to synthesize a large amount of additional training data 30 using a human-participatory learning-augmentation framework. In this example, the small set of training data 10 includes automobile-related natural language sentences, which are manually labeled by the domain expert 20 with corresponding classification labels indicating whether the sentences (1) describe a problem or (2) do not describe a problem (for example, “This engine is very noisy” has the label “problem”). However, in further embodiments, additional semantic labels may be added to the small set of training data 10 beyond simple binary classification of the entire sentences.

[0015] The data augmentation process has three core functions: a text generator 40, a text classifier 50, and weak supervision by domain experts 20 "in the loop". Generally, both the text generator 40 and the text classifier 50 are implemented as machine learning models. As used herein, the term "machine learning model" refers to a system or program instruction and / or set of data configured to implement an algorithm, process or mathematical model (e.g., a neural network) that predicts or provides a desired output based on a given input. Generally, many or most parameters of a machine learning model are not explicitly programmed, and it will be understood that a machine learning model is not explicitly designed to follow specific rules in the conventional sense to provide a desired output for a given input. Instead, a machine learning model is provided with a corpus of training data from which it identifies or "learns" patterns and statistical relationships in the data, and these patterns and statistical relationships are generalized to make predictions or provide outputs for new data inputs. The results of the training process are embodied in the form of multiple learned parameters, kernel weights, and / or filter values, which are used in various components of the machine learning model to perform various operations or functions.

[0016] The text generator 40 generates new texts 30 based on manually labeled texts 10 (e.g., “This engine is very loud.”, “The noise level of this engine is very high.”, “This brake is very noisy.”, and “This horn is very noisy.”). Initially, the text generator 40 is trained with a small set of texts 10 using reinforcement learning with various reward functions that weakly supervise the generation of new texts 30 so as to provide useful teachers rather than redundant teachers for further training of the text classifier 50. In this way, the reward function maximizes the utility of human participation (i.e., domain experts 20) and minimizes the effort of wasted manual labeling.

[0017] The text classifier 50 generates labels for the newly generated texts 30. In particular, each of the newly generated texts 30 passes through the text classifier 50 to generate a proposed classification label or other proposed semantic label. The text classifier 50 is also initially trained using a small set of manually labeled texts 10 with a supervised learning process. Because the text classifier 50 is trained on a small number of examples, it generally performs poorly in the early iterations of the data augmentation process and does not generalize well.

[0018] The new text 30 with the proposed labels is displayed or provided for review by the domain expert 20, who interacts with the system to verify the new text 30 and correct any errors. In particular, the domain expert 20 can correct the proposed labels for the new text 30, or correct grammatical or similar issues in the text of the new text 30. Using the verified new text 30, one or both of the text generator 40 and the text classifier 50 are retrained. In this way, the performance of data augmentation is continuously improved, and the time required for the domain expert 20 to verify each newly generated text 30 is reduced.

[0019] The goal of the data augmentation process is to make the learning process for the text classifier 50 efficient with minimal supervision by domain experts. In particular, the data augmentation process learns various regularities for data augmentation in an iterative, human-participatory training process that is significantly more cost-effective than the conventional collection and manual labeling of training data. Below, we discuss the various advantages of the data augmentation process by highlighting its main differences from four related research areas: (1) data augmentation, (2) paraphrase generation, (3) text adversarial attacks, and (4) active learning.

[0020] Data augmentation (DA) is a family of techniques for extending existing labeled data into larger amounts of synthesized labeled data. The data augmentation processes described herein can be considered advanced data augmentation methods. In the case of image data, simple techniques such as rotation and inversion have proven effective because such operations can guarantee the class labels (semantic concepts) of the modified data. However, such simple techniques cannot be applied to text because text is discrete data, and even small changes (e.g., modification of a single word) can completely alter the meaning of a sentence, as shown by the fourth generated sentence in Figure 1 (i.e., "This horn is very noisy."). Many existing data augmentation techniques for text rely on heuristic methods, such as synonym substitution and backtranslation. These approaches require the availability of external resources such as synonym dictionaries and high-performance machine translation models. However, such resources are often unavailable in specific domains. Therefore, conventional resource-based data augmentation techniques cannot be scaled to the domain of interest. In contrast, the data augmentation process described herein learns directly from domain experts how to augment the data. Therefore, the data augmentation process described herein has two main advantages compared to conventional data augmentation techniques. First, the data augmentation process described herein eliminates the need for external resources. Second, the data augmentation process described herein can improve the performance of data augmentation along with the domain of interest by exploring regularities at various linguistic and knowledge levels through the learning process.

[0021] Paraphrase generation is the task of generating output text (e.g., sentences) that is semantically identical to the input text (e.g., sentences) but includes variations in vocabulary, syntax, or both. The data augmentation processes described herein are similar to paraphrase generation in that they involve variations in sentence modification. In particular, the data augmentation processes described herein can modify the input sentence not only at the word level but also at the phrase level or sentence level, as shown by the second generated sentence in Figure 1 (i.e., "The noise level of this engine is very high."). However, from a task perspective, the data augmentation processes described herein have different objectives than paraphrase generation. Paraphrase generation aims to paraphrase a given sentence in various ways, such as the first generated sentence in Figure 1 (i.e., "This engine is very loud."), while keeping the semantics of the generated sentences unchanged. In contrast, the data augmentation process described herein can significantly alter the meaning of input sentences to explore various meaningful teachers, as shown by the third and fourth generated sentences in Figure 1 (i.e., “This brake is very noisy.” and “This horn is very noisy.”). Furthermore, there are further differences from a methodological standpoint. Recent research on paraphrase generation also utilizes neural models, particularly neural encoder-decoder architectures. In contrast, the data augmentation process described herein significantly extends conventional encoder-decoder architectures because it also receives feedback from a sentence classification model as additional input. This feedback acts as an additional signal to optimize the generator's components for ideal data augmentation, thereby improving the generalization ability of the sentence classification model.

[0022] Adversarial attacks using text are used to assess the robustness of deep learning models by deceiving them with small perturbations to input examples. These text-based adversarial attacks have a different philosophy in that their primary focus is on deceiving the model with minimally modified examples while preserving the original labels of the input examples, in order to expose the model's fragility. In contrast, the data augmentation process described herein does not have such an adversarial intent. Instead, the data augmentation process described herein generates examples to improve the model's generalization ability. For the goal of improving generalization ability, the data augmentation process described herein is not limited to generating examples that retain the original labels of the input examples. Rather, this framework can intentionally generate examples that change the labels to have a completely different kind of teacher, as shown in the fourth generated sentence in Figure 1 (i.e., “This horn is very loud.” labeled as non-problem). The data augmentation process described herein is tolerant of the details of the target text classification model and merely employs the model's inputs and outputs; therefore, from a methodological standpoint, the data augmentation process described herein may appear similar to a black-box model for text adversarial attacks. However, the data augmentation process described herein differs from conventional black-box models for text adversarial attacks in that the generator components are optimized to produce texts that are not only uncertain but also diverse and representative, which is useful for improving the generalization of the model.

[0023] Finally, active learning (AL) refers to a learning process in which human-participatory training is used to support efficient learning. However, the data augmentation process described herein differs from conventional active learning in how human-assisted training is utilized in labeling the data. In the case of text classification, the conventional active learning process first trains a text classification model on an initial set of labeled texts (seed examples), then samples a small set of unlabeled texts from a large pool of unlabeled data to obtain texts labeled by a domain expert (human). The newly labeled texts are added to the initial training data, and the model is retrained based on this newly augmented training data. This process is repeated to progressively increase the amount of training data. In contrast, the data augmentation process described herein has at least one clear difference from conventional active learning. In particular, the data augmentation process described herein does not sample from unlabeled data, but rather uses a model trained on existing (labeled) texts to generate new texts and automatically proposes labels for these new texts. This synthesized data may contain errors (e.g., grammatically incorrect word order or incorrect labels) and may require correction by domain experts, especially in the early stages, but the quality of the synthesized data improves during the learning process. The data augmentation process described herein makes the entire learning process efficient by generating new data and proposing labels. Unlike active learning, the data augmentation process described herein can, in principle, iteratively expand the training data from seed examples without relying on arbitrary additional data (e.g., a large pool of unlabeled data). Moreover, the data augmentation process described herein is flexible in that it can also accept additional labeled sentences during the learning process.In that sense, the data augmentation processes described herein are separate from active learning and can be used to improve the performance of active learning.

[0024] Text generation model The ultimate goal of the data augmentation process described herein is to improve the generalization power of the text classifier 50 without expending the effort of manually labeling a very large set of domain-specific training texts. Thus, at the heart of the process is a text generator 40 capable of synthesizing new, high-quality texts from the original texts using only weakly supervised texts. While the text generator 40 is described primarily as generating texts, it should be understood that it may also be configured to generate any other units of text data, such as phrases or paragraphs. Therefore, the reference to “text” herein should be understood to include alternative units of text data that may or may not strictly contain texts.

[0025] In this context, "high-quality" new text (or other text data) refers to synthesized text that provides useful training for further training the text classifier 50 and improves the generalization performance of the text classifier 50. The text generator 40 is configured to generate new text that is uncertain, diverse, and representative. Each of the qualities of uncertainty, diversity, and representativeness is advantageous in providing useful training and improved generalization capabilities for the text classifier 50.

[0026] Where used herein in relation to the generation of new texts by the text generator 40, "uncertainty" refers to the confidence level of the text classifier 50 in outputting a proposed classification label for the new text. A new text that is "uncertain" is one that is close to the decision boundary of the text classifier 50 and for which the initially proposed classification label has a low confidence level. Correcting or verifying new texts with higher uncertainty by the domain expert 20 will provide more useful training for further training of the text classifier 50. Conversely, if the text classifier 50 already has a high confidence level regarding the correct classification label for the new text, verification by the domain expert 20 will not provide useful training.

[0027] In this specification, "diversity" as used in relation to the generation of new texts by the text generator 40 refers to the degree to which the new text differs in form or meaning from the original input text (e.g., a manually labeled text). A "diverse" new text is one that is significantly different from the original input text. Modifying or validating new texts with higher diversity by the domain expert 20 will provide more useful training for the text classifier 50. Conversely, if the new text is very similar to the original input text, validation by the domain expert 20 will not provide useful training.

[0028] Where used herein in relation to the generation of new texts by the text generator 40, "representativeness" means the extent to which a new text describes a frequent or common input to the text classifier 50 (e.g., a common and realistic car problem), or conversely, the extent to which a new text describes an outlier or rare input (e.g., an unrealistic or meaningless car problem). Modifying or verifying new texts that describe realistic events (e.g., "The engine needs an oil change.") by the domain expert 20 would provide more useful training for further training of the text classifier 50. Conversely, if a new text describes an unrealistic event (e.g., "The headlights need an oil change."), verification or modification by the domain expert 20 would not provide useful training.

[0029] Each of the desired qualities—uncertainty, diversity, and representativeness—is advantageous in providing useful training, but each quality also has its own weaknesses, which are complemented by one or both of the other qualities. For example, uncertainty generally favors outliers (i.e., sentences describing rare phenomena), but this is generally mitigated by representativeness. Similarly, representativeness generally favors similar sentences, resulting in reduced coverage, but this is generally mitigated by diversity. Therefore, rewarding all of these qualities together is crucial for synthesizing a balanced set of new, high-quality sentences for further training of the sentence classifier 50.

[0030] Figure 2 shows an exemplary embodiment of a text generator 40 configured to generate uncertain, diverse, and representative high-quality new texts. In the illustrated embodiment, the text generator 40 takes the form of an encoder-decoder model 42 having an encoder 44 and a decoder 46. The encoder-decoder model 42 formulates text generation as a sequence-to-sequence problem. In particular, the encoder-decoder model 42 is configured to map an input text x to an output text y, where x is a sequence of tokens or similar symbolic representations (i.e., individual words, characters, punctuation marks, etc.) of variable length m, x = x1, x2, ..., x m Similarly, y is a sequence of tokens or similar symbolic representations of variable length n, y=y1,y2,···,y n In this specification, "token" refers to a representation of an individual word, subword, character, or punctuation mark from text data. Therefore, "tokenization" of text data refers to the conversion of text data into a sequence of corresponding tokens.

[0031] The encoder 44 receives the input text x and converts the input text x into a series of hidden states h = h1, h2, ..., h m It is configured to encode the following. Given a sequence of hidden states h, the decoder 46 is configured to output a conditional probability distribution of possible output sentences y. In other words, the decoder 46 outputs the following probabilities:

number

[0032] In at least some embodiments, the text generator 40 is a transformer-based neural network encoder-decoder model 42, where both the encoder 44 and decoder 46 are stacks of residual attention layers. In one embodiment, the encoder-decoder model 42 is a pre-trainable transformer-based encoder-decoder, such as BART ("BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension" by Lewis et al., 2020), which is pre-trained on intra-domain data so as to capture domain-specific regularities. However, it will be understood that in some embodiments, a recurrent neural network (RNN)-based encoder-decoder model or any other sequence-to-sequence model may be used as well.

[0033] As described above, given a specific input sequence x, the decoder 46 outputs a conditional probability distribution of possible output sentences y. The final output of the decoder 46 is selected using an inference algorithm, such as a search algorithm, based on the conditional probability distribution of possible output sentences y. For example, in at least one embodiment, a beam search is used to identify a predetermined number of maximum likelihood output sentences y for a specific sequence x (e.g., k=4 possible output sentences as shown in the example in Figure 1). In at least one embodiment, the decoder 46 outputs a probability distribution for each token (e.g., each word). The beam width in the beam search is denoted as B≧k. In this case, for the first token in the output sequence y, the search algorithm selects B maximum likelihood outputs for that first token. Next, for the second token, the search algorithm selects B maximum likelihood outputs for this second token, provided that the possible outputs selected for the first token are a requirement. This process is repeated for all n tokens in the output sequence y until k maximum likelihood complete output sequences y (e.g., complete sentences) are identified. Once a set of k output sequences is identified, these k sets of output sequences can be supplied to the text classifier 50 to determine a proposed label, and then provided to the domain expert 20 for verification or correction as described above.

[0034] As described above, in at least some embodiments, the text generator 40 is trained using reinforcement learning based only on a small set of training data D (e.g., a small set of manually labeled texts 10), at least initially. The training data D consists of pairs of manually created text labels.

number

[0035] In at least some embodiments, the text generator 40 is trained using a policy-based reinforcement learning algorithm, and in this policy-based reinforcement learning algorithm, a policy p parameterized by model parameters θ θ is learned. In one embodiment, the text generator 40 is trained using the REINFORCE algorithm ("Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning" by Williams, 1992). The policy p θ is learned to reinforce the generation of output sequences that meet sequence-level measurement criteria of uncertainty, diversity, and representativeness.

[0036] The reinforcement learning process formulates each measurement criterion as a respective reward function r() calculated based on the generated text y and its original text x. The reinforcement learning process minimizes the expected negative reward as a loss function:

Equation

[0037] The reinforcement learning process is represented by the following equation: ∇ θ L(θ) ≈ -(r(x,y s )-b)∇ θ logp(y s |x) According to this, a single sample y s ~p θ The gradient expected by is approximated by b = r(x,y a ) is defined as, where r(x,y a ) is the best output y using the current model according to the inference algorithm at a given x during testing. a The reward obtained from (for example, y a (This is the maximum likelihood output obtained by beam search.)

[0038] In some embodiments, the reinforcement learning process is a multi-reward optimization approach, or in other words, multiple distinct reward functions r(x,y s ) is used. As mentioned above, it is important that the output sentence y simultaneously presents uncertainty, diversity, and representativeness. For this purpose, in at least some embodiments, the reinforcement learning process uses three reward functions r for uncertainty, diversity, and representativeness, respectively. U (x,y), r D (x,y) and r R We use (x,y). In particular, generating uncertain sentences, generating diverse sentences, or generating representative sentences can be considered as different and distinct tasks, and a single model sharing the same parameter θ can have different reward functions for each: ∇ θ L U (θ) ≈ -(r U (x,y s )-r U (x,y a ))∇ θ logp(y s |x), ∇ θ L D (θ) ≈ -(rD (x,y s )-r D (x,y a ))∇ θ logp(y s |x), ∇ θ L R (θ) ≈ -(r R (x,y s )-r R (x,y a ))∇ θ logp(y s |x) It is optimized by [this method].

[0039] The reward function of uncertainty r U (x,y) is designed to evaluate the confidence of the text classifier 50 in outputting a proposed classification label for a newly generated text y. In particular, S refers to the probabilistic text classification model of the text classifier 50. The probabilistic text classification model S is a pair of manually generated text labels.

number

[0040] The reward function of uncertainty r U The equation is as follows: r U (x,y)=H(y)=-Σ z∈Z P S (z|y)logP S (z|y) It can be evaluated according to the following, where H(y) is the well-known entropy used as an uncertainty measure widely used in previous research on active learning. The reward function of uncertainty r U This should be understood as rewarding each output text y generated by the text generator 40, where the entropy of the text classifier 50 is relatively high when assigning each classification label z.

[0041] The reward function r of diversity D (x,y) is designed to evaluate the difference between the newly generated sentence y and each input sentence x. In at least one embodiment, the reward function for diversity is given by the following equation: r D (x,y)=StrDiff(x,y) The difference is determined according to , where StrDiff() is a distance algorithm such as the Hamming distance or the Levenshtein edit distance. In at least one embodiment, the distance algorithm determines the difference at the token level (word level), but it is also possible to use a character-level distance algorithm or any other distance algorithm. Thus, the reward function r of the diversity D The system awards a reward for each output sentence y generated by the sentence generator 40, which is relatively different from each input sentence x.

[0042] Finally, the representativeness reward function r R (x,y) is designed to assess the extent to which a new sentence describes a frequent or common meaning, or conversely, the extent to which a new sentence describes an outlier or rare meaning. In one embodiment, the representativeness reward function leverages a neural language model LM. It will be understood by those skilled in the art that a language model is a statistical model that assigns probabilities to words and sentences. Generally, a language model assigns higher probabilities to sentences that contain real words and are syntactically correct. In at least one embodiment, the language model LM is trained using a large corpus of intradomain text data. In the case of the “automotive problems” domain described herein for illustrative purposes, the intradomain text data may include, for example, vehicle maintenance, repair, or troubleshooting manuals and other vehicle-related text documents. In this way, the language model LM assigns higher probabilities to real or common sentences in its respective domain of interest.

[0043] In at least one embodiment, the reward function for representativeness is given by the following equation, based on the perplexity of the language model LM for the newly generated text y: r R (x,y) = -Perplexity LM (y) It is calculated according to [the formula].

[0044] Therefore, the representativeness reward function r R The system rewards each output sentence y generated by the sentence generator 40, which has a relatively low perplexity when provided to the language model LM. Perplexity is a metric typically used to evaluate the intrinsic performance of the language model LM, and it will be understood by those skilled in the art that it has a low value when the language model LM assigns a high probability to the input, and conversely, a high value when the language model LM assigns a low probability to the input. Therefore, since the language model LM is good at predicting sentences, when the negative perplexity is high (i.e., when the original perplexity is low), the newly generated sentence y is representative. Perplexity function LM A specific formulation of (y) is the probability P that LM assigns to the newly generated sentence y. LM (y) can take various forms that are generally related to (y). In one exemplary embodiment, the perplexity function is Perplexity LM (y) = -log(P LM (y)) or similar is defined. In other exemplary embodiments, the perplexity function is:

number

[0045] Text classification model As described above, the ultimate goal of the data augmentation process described herein is to synthesize a larger set of new, high-quality, domain-specific training sentences that can be used to further train the text classifier 50. By training the text classifier 50 with the larger set of synthesized training sentences, the generalization power of the text classifier 50 can be significantly improved. However, the specific implementation of the text classifier 50 is not particularly important to the data augmentation process itself, and therefore the text classifier 50 can be implemented using a wide variety of probabilistic model architectures, and in particular, a wide variety of machine learning models. In many embodiments, the text classifier 50 is implemented using deep learning models (e.g., convolutional neural networks, recurrent neural networks, transformer-based neural networks, etc.).

[0046] As described above, the text classifier 50 implements a probabilistic text classification model S. The probabilistic text classification model S uses manually created pairs of text labels.

number

number

number

number

[0047] Data Enhancement System Figure 3 shows a block diagram of an exemplary embodiment of the data augmentation system 100. The data augmentation system 100 advantageously utilizes the methods described above to enable rapid and cost-effective human-participatory synthesis of domain-specific training data for deep learning models. In particular, in at least one embodiment, the system 100 is configured to synthesize a number of domain-specific sentences having domain-specific classification labels based on a small number of manually labeled sentences. This synthesized data is used to train a robust sentence classification model that can enable further downstream processing of the sentences for several applications, for example. These downstream applications may include, for example, an intelligent automotive aftermarket assistance service that helps a user effectively and efficiently identify sentences describing specific problems and solutions related to automotive repair.

[0048] In the illustrated exemplary embodiment, the data augmentation system 100 includes at least one processor 102, at least one memory 104, a communication module 106, a display screen 108, and a user interface 110. However, it will be understood that the components of the illustrated and described data augmentation system 100 are merely illustrative, and the data augmentation system 100 may include any alternative configuration. In particular, the data augmentation system 100 may include any computing device such as a desktop computer, laptop, smartphone, tablet, or other personal electronic device. Accordingly, the data augmentation system 100 may include any hardware components conventionally included in such computing devices.

[0049] Memory 104 is configured to store data and program instructions that enable the data augmentation system 100 to perform the various operations described herein when executed by at least one processor 102. Memory 104 may be any type of device accessible by at least one processor 102 and capable of storing information, such as a memory card, ROM, RAM, hard drive, disk, flash memory, or any other computer-readable medium that functions as a data storage device, as will be recognized by those skilled in the art. Furthermore, it will be recognized by those skilled in the art that “processor” includes any hardware system, hardware mechanism, or hardware component that processes data, signals, or other information. Accordingly, at least one processor 102 may include a central processing unit, a graphics processing unit, multiple processing units, dedicated circuits for implementing functions, programmable logic, or other processing systems. Furthermore, although the data augmentation system 100 is illustrated as a single system, it will be understood that it may include several separate systems that cooperate to implement the functions described herein.

[0050] The communication module 106 may include one or more transceivers, modems, processors, memory, oscillators, antennas, or other hardware that are conventionally included in a communication module to enable communication with various other devices. In at least some embodiments, the communication module 106 includes a Wi-Fi module configured to enable communication with a Wi-Fi network and / or a Wi-Fi router (not shown). In further embodiments, the communication module 46 may further include a Bluetooth® module, an Ethernet adapter, and a communication device configured to communicate with a wireless telegraph network.

[0051] The display screen 108 may include any various known types of displays, such as an LCD or OLED screen. In some embodiments, the display screen 108 may include a touchscreen configured to receive touch input from a user. The user interface 110 may appropriately include various devices configured to enable local operation of the data augmentation system 100 by a user, such as a mouse, trackpad or other pointing device, keyboard or other keypad, speaker and microphone, as will be recognized by those skilled in the art. Optionally, in some embodiments, a user may communicate with the data augmentation system 100 via a communication module 106 and remotely operate the data augmentation system 100 from another computing device having a similar user interface.

[0052] The program instructions stored in memory 104 include a data augmentation program 112, which includes a text data generation model 114 and a text data labeling model 116. In particular, the processor 102 executes the text data generation model 114 of the data augmentation program 112 to generate new text data based on a small set of manually labeled text data. Similarly, the processor 102 executes the text data labeling model 116 of the data augmentation program 112 to generate proposed labels for the newly generated text data. In the context of text classification, the text data generation model 114 and the text data labeling model 116 implement, for example, a text generator 40 and a text classifier 50, respectively.

[0053] How the data enhancement system operates Figure 4 shows a flowchart relating to method 200 for operating the data augmentation system. In the description of these methods, when it is stated that a task, calculation, or function is performed, it means that a processor (e.g., processor 102 of data augmentation system 100) executes programmed instructions (e.g., data augmentation program 112, text data generation model 114, or text data labeling model 116), which are stored in a non-temporary computer-readable storage medium (e.g., memory 104 of data augmentation system 100) operably connected to the processor for manipulating data so that a task or function is performed, or for operating one or more components of data augmentation system 100. Furthermore, the steps of the method can be performed in any feasible temporal order, regardless of the order shown in the drawings or the order in which the steps are described.

[0054] Method 200 begins with receiving units of text data (block 210). In particular, the processor 102 receives text data, which may be associated with corresponding labels that describe the features of the text data. The text data is, in particular, individual units x of text data, which may include, for example, sentences, but may also include any other units such as phrases or paragraphs of text data. In at least one embodiment, the processor 102 reads units x of text data from memory 104, which contains manually created pairs of sentence labels.

number

[0055] In at least one embodiment, multiple units x of text data (i)As mentioned above, this relates to a specific domain of information or knowledge (e.g., automotive issues). In at least one embodiment, each unit x of the text data (i) Corresponding label z for (i) This is the unit x of the text data. (i) A classification label that identifies the classification (for example, whether a text describes an automobile problem or not), and can also identify a wide variety of semantic concepts and attributes of the corresponding units of text data. In some embodiments, each unit x of the text data is (i) is two or more labels z (i) It may be considered to have the following characteristics.

[0056] Method 200 continues by using a text generation model to generate new units of text data based on units of received text data (block 230). In particular, the processor 102 is configured to execute program instructions of the text data generation model 114 to generate new units y of text data based on units x of received text data. In some embodiments, the processor 102 generates multiple new units y of text data based on each unit x of the received text data. As described above, in the context of text classification, the text data generation model 114 implements, for example, a text generator 40, and more specifically, an encoder-decoder model 42. However, it should be understood that the text data generation model 114 may implement different variations and types of machine learning models.

[0057] In some embodiments, the processor 102 converts (or "tokenizes") a unit x of received text data into an input sequence x of tokens corresponding to the text, and then encodes the input sequence x of tokens using an encoder of the text data generation model 114 (e.g., encoder 44 of encoder-decoder model 42). Next, the processor 102 uses a decoder of the text data generation model 114 (e.g., decoder 46 of encoder-decoder model 42) to determine a probability distribution for the output sequence y of tokens. Finally, the processor 102 uses an inference model of the text data generation model 114 to determine a new unit y of text data.

[0058] In at least some embodiments, the inference model of the text data generation model 114 is a search algorithm such as beam search that determines the "best" or most likely output sequence based on a probability distribution of the token output sequence y. Thus, if one new unit y of text data is to be determined, the processor 102 uses beam search to determine the new unit y of text data as the most likely output sequence of the token based on a probability distribution of the token output sequence. If multiple new units y of text data are to be determined, the processor 102 uses beam search to determine multiple new units y of text data as the k most likely output sequences of the token based on a probability distribution of the token output sequence.

[0059] As described above with respect to the encoder-decoder model 42 in Figure 2, the text data generation model 114 initially generates multiple units x of the text data of the training data D stored in memory 104. (i) Based on this, it is trained using a reinforcement learning process. In particular, processor 102 uses multiple reward functions r U (x,y), r D (x,y) and r RThe parameters of the text data generation model 114 are optimized using multi-reward optimization that enhances (x,y).

[0060] Method 200 continues by using a text labeling model to determine proposed labels for new units of text data (block 250). Specifically, processor 102 determines proposed labels from a predefined set of labels Z based on the new unit y of text data.

number

number

number

[0061] As described in detail above with respect to the text classifier 50, the text data labeling model 116 initially stores manually created pairs of text labels in memory 104.

number

[0062] Method 200 continues by receiving validation or modification of new units of text data or proposed labels from the user (block 270). In particular, new units y of text data and proposed labels

number

number

number

[0063] The processor 102 receives input via the user interface 110, which includes (i) a modified new unit y' of text data, which includes a modification to the new unit y of text data, (ii) Proposed label

number

number

number

number

number

number

[0064] Method 200 continues by retraining the text generation model and the text labeling model based on the validation or modification of a new unit of text data with the proposed label (block 290). In particular, the new unit y of text data and the proposed label

number

number

[0065] The text data generation model 114 and the text data labeling model 116 can be retrained to provide incremental performance improvements by adding new training examples to the training data D. In particular, the processor 102 retrains the text data generation model 114 using the reinforcement learning process described above with the updated set of training data D, which includes the new modified / validated units y' of the text data. Similarly, the processor 102 retrains the new modified / validated units y' of the text data and the modified / validated labels

number

[0066] In some embodiments, in addition to weakly supervised reinforcement learning, the text data generation model 114 can be further trained using pairs of input units x of text data and new units y of text data validated or modified by a domain expert 20. In this way, as more and more new units y of text data are generated and validated, the text data generation model 114 can be further trained using pairwise training use cases in a more idiomatic supervised training process.

[0067] Each embodiment within the scope of this disclosure may include a non-temporary computer-readable or machine-readable medium for carrying or holding computer-executable instructions (also referred to as program instructions) or data structures stored therein. Such a non-temporary computer-readable or machine-readable medium may be any available medium accessible by a general-purpose or dedicated computer. Such a non-temporary computer-readable or machine-readable medium may include, but are not limited to, RAM, ROM, EEPROM, CD-ROM, or other optical disk storage devices, magnetic disk storage devices, or other magnetic storage devices, or any other medium usable for carrying or storing desired program code means in the form of computer-executable instructions or data structures. Combinations of the above should also be included within the scope of a non-temporary computer-readable or machine-readable medium.

[0068] Computer executable instructions include, for example, instructions and data that cause a general-purpose computer, a dedicated computer, or a dedicated processing unit to perform a specific function or set of functions. Computer executable instructions also include program modules executed by computers in a standalone or networked environment. Generally, program modules include routines, programs, objects, components, and data structures that perform a specific task or implement a specific abstract data type. Computer executable instructions, associated data structures, and program modules represent examples of program code means for performing steps of the methods disclosed herein. A particular sequence of such executable instructions or associated data structures represents an example of the corresponding operation for performing the function described in such step.

[0069] Although the present disclosure has been illustrated and described in detail in the drawings and the aforementioned specification, these drawings and the aforementioned specification should be considered illustrative and not limiting in nature. They merely present preferred embodiments, and it should be understood that all changes, modifications, and further uses contained within the spirit of the present disclosure are protected.

Claims

1. A method for generating labeled text training data, The processor receives the first unit of text data, The processor generates a second unit of the text data based on the first unit of the text data using a first machine learning model, The processor uses a second machine learning model to determine labels that describe the features of the second unit of the text data, The output device outputs the second unit of the text data and the label to the user, The system receives, via the user interface, (i) a modification to the second unit of the text data and / or verification of the second unit of the text data, and (ii) a modification to the label and / or verification of the label. The processor retrains the second machine learning model using (i) the second unit of the modified text data and the second unit of the verified text data, and (ii) the second label and the verified label. Includes, Before generating the second unit of the text data, the processor trains the first machine learning model on multiple units of the text data using a reinforcement learning process. Methods that further include this.

2. The processor retrains the first machine learning model using either the second unit of the corrected text data or the second unit of the verified text data. The method according to claim 1, further comprising:

3. The first machine learning model is a sequence-to-sequence model. The method according to claim 1.

4. Generating the second unit of the text data is: The processor encodes an input sequence of tokens, which are tokenized units of the text data, using the encoder of the first machine learning model. The processor uses the decoder of the first machine learning model to determine the probability distribution of the token output sequence, The processor determines the second unit of the text data based on the probability distribution relating to the output sequence of the tokens, The method according to claim 3, further comprising:

5. Generating the second unit of the text data is: The processor determines the second unit of the text data as the most likely output sequence of the token based on the probability distribution relating to the output sequence of the token. The method according to claim 4, further comprising:

6. Generating the second unit of the text data is: The processor uses beam search to determine the most likely output sequence of the token based on the probability distribution relating to the output sequence of the token. The method according to claim 5, further comprising:

7. Generating the second unit of the text data is: The process further includes generating a plurality of second units of text data using the aforementioned processor, The plurality of second units of the text data are determined as a predetermined number of maximum likelihood output sequences of tokens based on the probability distribution relating to the output sequences of the tokens. The method according to claim 4.

8. At least one of the encoder and decoder of the first machine learning model has a transformer-based neural network architecture. The method according to claim 4.

9. Training the first machine learning model described above is The processor optimizes the parameters of the first machine learning model using multi-reward optimization, which enhances multiple reward functions. The method according to claim 1, further comprising:

10. The aforementioned plurality of reward functions include a first reward function, The first reward function rewards each output sequence of tokens generated by the first machine learning model when, given each input sequence of tokens to the first machine learning model, the second machine learning model has a relatively high level of (i) uncertainty and (ii) entropy in determining each label describing the features of each output sequence of tokens. The method according to claim 9.

11. The aforementioned multiple reward functions include a second reward function, The second reward function, when given each input sequence of tokens to the first machine learning model, rewards each output sequence of tokens generated by the first machine learning model that is relatively different from each input sequence of tokens. The method according to claim 9.

12. The aforementioned multiple reward functions include a third reward function, The third reward function rewards each output sequence of tokens generated by the first machine learning model, which has a relatively low perplexity when provided to the language model, when each input sequence of tokens is given to the first machine learning model. The method according to claim 9.

13. The second machine learning model described above is a deep neural network model. The method according to claim 1.

14. Prior to determining the labels, the processor trains the second machine learning model based on multiple units of text data. It further includes, Each unit of the text data in the plurality of units of the text data has a label that describes the characteristics of each unit of the text data. The method according to claim 1.

15. Outputting the second unit of the text data and the label is: A graphical user interface including the second unit of the text data and the label is displayed to the user on a display screen. The method according to claim 1, further comprising:

16. The label is a classification of the second unit of the text data. The method according to claim 1.

17. The first unit of the text data includes at least one natural language sentence, The second unit of the text data includes at least one natural language sentence. The method according to claim 1.

18. A system for generating labeled text training data, The system in question is Output device and User interface and A memory configured to store multiple units of text data, wherein each unit of text data in the multiple units of text data has a label that describes the characteristics of each unit of text data, A processor operably connected to the output device, the user interface, and the memory, Includes, The aforementioned processor, The first unit of text data is read from the plurality of units of text data stored in the memory. Using a first machine learning model, a second unit of text data is generated based on the first unit of text data. Using a second machine learning model, labels describing the features of the second unit of the text data are determined. The output device is operated to output the second unit of the text data and the label to the user. (i) to operate the user interface to receive either a modification to the second unit of the text data or a verification of the second unit of the text data, and (ii) to receive either a modification to the label or a verification of the label. (i) Retrain the second machine learning model using either the second unit of the modified text data or the second unit of the verified text data, and (ii) the second label or the verified label. It is configured in such a way, The aforementioned processor, Before generating the second unit of the text data, the first machine learning model is trained on multiple units of the text data using a reinforcement learning process. The system is further configured in this way.

19. A non-temporary computer-readable medium for generating labeled text training data, the computer-readable medium storing program instructions, The aforementioned program instruction is executed by the processor and, if it is not executed, the processor The first unit of text data is received, Using a first machine learning model, generate a second unit of text data based on the first unit of text data. A second machine learning model is used to determine labels that describe the features of the second unit of the text data. The output device is operated to output the second unit of the text data and the label to the user. (i) to operate the user interface to receive either a modification to the second unit of the text data or a verification of the second unit of the text data, and (ii) to receive either a modification to the label or a verification of the label. (i) using the second unit of the modified text data and the second unit of the verified text data, and (ii) using the second label and the verified label, the second machine learning model is retrained, Before generating the second unit of the text data, the first machine learning model is trained on multiple units of the text data using a reinforcement learning process. A non-temporary computer-readable medium.

Images