A label semantic enhanced weakly supervised text classification method and system

Abstract

This invention discloses a weakly supervised text classification method and system with enhanced label semantics, belonging to the field of machine learning. Based on the BERT weakly supervised text classification framework, in the category vocabulary construction stage, weighted category representation based on Zipf's law is used to denoise category words, and irrelevant words in the category vocabulary are removed by utilizing the decreasing semantic similarity property. In the sample labeling stage, word category labeling is based on the MASK mechanism, and then the classification model is optimized based on a self-training module. Using category indicator words in the samples as bridges, a cross-level semantic association of "sample sentence - indicator word - category label" is established. This invention introduces more algorithms to reduce label noise in the vocabulary construction and sample weak label generation stages to achieve the effect of enhanced label semantics, significantly improving text classification performance in different language environments.

Description

Technical Field

[0001] This invention belongs to the field of machine learning, and in particular relates to a weakly supervised text classification method and system with enhanced label semantics. Background Technology

[0002] In practical production and daily life, text classification is a crucial tool with significant positive impacts. For example, classifying news topics allows platforms to aggregate and push content in a targeted manner; classifying product reviews helps businesses understand consumer feedback; classifying pathologies aids doctors in rapid diagnosis and treatment; and performing sentiment analysis allows for the categorization of text sentiment as positive, negative, or neutral. Among text classification methods, weakly supervised text classification, which uses weakly supervised sources to label data samples and train the text model as accurately as possible without manually labeled samples, is considered a relatively ideal classification method. Common weakly supervised sources include user-provided category terms, expert semantic rules, or external knowledge bases. Because weakly supervised text classification does not use any manually labeled category information, it excels in mitigating reliance on precisely labeled data, cross-domain data fusion, and data completion.

[0003] A typical weakly supervised learning framework can be summarized into three stages:

[0004] 1) Vocabulary construction: A vocabulary generator is built to expand the vocabulary to construct a category vocabulary. Common expansion methods include topic word discovery methods and word vector methods.

[0005] 2) Sample Labeling: Based on the category vocabulary constructed in stage 1), labels are generated for the text. Explicit and implicit matching methods are typically used. Explicit matching usually directly counts the frequency of category words in the samples and selects the category with the highest frequency to generate a weak label. Implicit matching calculates the similarity between the category word vectors and the text semantic vectors, and assigns the most similar category to the sample.

[0006] 3) Sample Classification: Based on the samples obtained in stage 2) and their corresponding weak labels (data), a sample classifier is trained using supervised learning techniques. Currently, self-training techniques are commonly used. The previously trained model is used to predict high-confidence labels for unmatched corpora, and expanded samples are obtained. Then, the expanded samples are used to retrain the model, and the above steps are iteratively executed until the model performance is stable.

[0007] In current weakly supervised learning frameworks, significant noise is generated during vocabulary construction and sample labeling, impacting classification performance. Vocabulary construction introduces numerous irrelevant words, and inaccurate category vocabularies lead to incorrect sample labeling. Furthermore, since text semantics are more complex than word semantics, generating data labels based on individual words often results in labeling errors, i.e., label noise. Currently, no method has been found to effectively enhance label semantics and thus optimize weakly supervised text classification accuracy through efficient vocabulary expansion and sample labeling processes.

[0008] Furthermore, the published Chinese invention patent CN113971407A provides a semantic feature extraction method. This method selects candidate words based on the relevance between words and categories in text data, then constructs a category semantic lexicon based on the mutual information between the candidate words and pre-defined feature words. The text data is then masked using this lexicon, and a BERT model is trained using the masked text. This allows the BERT model to focus on category feature descriptions when encoding the semantic representation of the text, replacing manual annotation with the resulting semantic feature vectors, thus reducing the workload of manual annotation. While this method uses masks for the entire sentence's semantic learning, it lacks self-training, resulting in a lack of flexibility, an inability to effectively handle different scenario conditions, over-reliance on model architecture, datasets, and pre-training effects, and poor scalability. In existing self-training technologies, few incorporate mask mechanisms for text classification. Summary of the Invention

[0009] The purpose of this invention is to propose a weakly supervised text classification method and system with enhanced label semantics. Based on the BERT weakly supervised text classification framework, combined with Zipf's law, the MASK mechanism and self-training, it can significantly improve the accuracy of text classification, especially in the case of scarce labeled data, and has excellent generalization ability.

[0010] To achieve the above objectives, the present invention adopts the following technical solution:

[0011] A weakly supervised text classification method with enhanced label semantics includes the following steps:

[0012] For text data that needs to be classified, samples are extracted and generative methods are used to construct a candidate word list for the words in the samples;

[0013] For each word in the candidate word list, its contextual representation is determined based on the word's semantics in the sample, and the correlation between its contextual representation and its category representation is calculated.

[0014] Based on Zipf's law, the category representation of words is calculated according to the above relevance, and a category vocabulary is constructed based on the words with the highest relevance selected each time.

[0015] For each word in the sample sentence, the BERT model is used to determine the indicative category of the word based on the constructed category vocabulary, and the word is used as the corresponding category indicative word;

[0016] For each category indicator word, a masking mechanism is used to mask the category indicator word, and a classifier is trained using cross-entropy loss to predict the category label of the category indicator word based on the contextual representation of the category indicator word;

[0017] A self-training module consisting of a BERT model, a semantic similarity module, and a Softmax classifier is constructed. This self-training model generates sentence vector representations for sentences in the samples using the BERT model, predicts the label distribution based on the sentence vector representations using the Softmax classifier, and calculates the similarity between the sentence vector representations and the label representations using the semantic similarity module. This self-training module learns the label semantics of the category labels based on the similarity, calculates the true label distribution for each sentence, and uses the true label distribution for self-training.

[0018] The trained self-training module is used to classify the text data that needs to be classified.

[0019] Preferably, the method for determining the contextual representation of a word based on its semantics in the sample includes averaging all contextualized representations of the word appearing in the sample to obtain its contextual representation.

[0020] Preferably, cosine similarity is used to calculate the correlation between word context representation and category representation.

[0021] Preferably, when calculating the category representation of a word, the initial category representation is the contextual representation of the category name.

[0022] Preferably, when constructing the category vocabulary, if the correlation between the context representation and the category representation of all remaining words is lower than a threshold after multiple word selections, the selection process is stopped.

[0023] Preferably, the step of using the BERT model to determine the indicator category of a word based on the constructed category vocabulary includes: using the BERT model to find the k most relevant words, comparing the k words with the category vocabulary of each category, and selecting the category with the largest intersection exceeding a threshold as the indicator category of the word.

[0024] Preferably, word-level supervision is constructed for category-indicating words by using a set consisting of category-indicating words and their corresponding word categories.

[0025] Preferably, the self-training module obtains the category label representation through label embedding.

[0026] Preferably, during training, the self-training module uses cross-entropy loss and KL divergence as objective functions, where cross-entropy loss is the loss when predicting the label distribution, and KL divergence is used to determine the true label distribution.

[0027] A weakly supervised text classification system with enhanced label semantics, comprising:

[0028] The word generator is responsible for extracting samples from the text data that needs to be classified and constructing a candidate word list for the words in the samples using generative methods. For each word in the candidate word list, its context representation is determined based on the semantics of the word in the sample, and the correlation between its context representation and category representation is calculated. Based on Zipf's law, the category representation of the word is calculated according to the above correlation, and a category word list is constructed based on the word with the highest correlation selected each time.

[0029] The sample labeler is responsible for determining the category of each word in the sentence of the sample using the BERT model based on the constructed category vocabulary, and using that word as the corresponding category indicator word; for each category indicator word, the category indicator word is masked using the MASK mechanism, and the classifier is trained using cross-entropy loss to predict the category label of the category indicator word based on the context representation of the category indicator word;

[0030] The sample classifier is responsible for generating sentence vector representations for sentences in the samples. It includes a self-training module consisting of a BERT model, a semantic similarity module, and a Softmax classifier. This self-training module generates sentence vector representations for sentences in the samples using the BERT model, predicts label distributions based on the sentence vector representations using the Softmax classifier, and calculates the similarity between the sentence vector representations and label representations using the semantic similarity module. This self-training module learns the label semantics of the category labels based on the similarity, calculates the true label distribution for each sentence, and uses the true label distribution for self-training. The trained self-training module is then used to classify the text data that needs to be classified.

[0031] The technical effects achieved by this invention are as follows:

[0032] 1. This invention is based on the BERT weakly supervised text classification framework, which can improve the classification accuracy of the model; it can use unlabeled data for self-training, thereby improving the model's performance; it can be quickly trained for various text classification tasks and can handle large-scale data well; it can handle various types of text data, including news, comments, social media and other types of text data, and can adapt to different tasks, including sentiment analysis, text classification, named entity recognition and other text tasks.

[0033] 2. In the category vocabulary construction stage, this invention achieves category word denoising based on Zipf's law weighted category representation, utilizing the decreasing semantic similarity characteristic to remove irrelevant words from the category vocabulary. This approach offers several advantages: firstly, it reduces interference and improves classification accuracy; secondly, it reduces training and prediction time and computational resources; thirdly, it enhances the model's generalization ability, enabling it to better handle unknown text data; fourthly, it reduces the risk of overfitting, making the model more stable; and fifthly, through Zipf's law weighted category representation, it retains category-related information, allowing the model to better understand the semantic information in the text data, thereby improving classification accuracy.

[0034] 3. In the sample labeling stage, this invention optimizes the autoencoder pre-trained model using a word category prediction task based on the MASK mechanism. Using category indicator words in the samples as bridges, it establishes a cross-level semantic association between "sample sentence - indicator word - category label." Firstly, this automatically generates missing data, helping the model handle missing data, reducing data labeling costs, and enabling more efficient learning of robust semantic representations. This improves the model's generalization ability and allows it to better handle unknown text data. Secondly, it combats category vocabulary noise in traditional methods, helping the model learn more accurate word representations and contextual relationships, enhancing its semantic understanding, and better handling the complex semantics of natural language. This improves the model's classification performance, increases accuracy, and significantly enhances text classification performance in different language environments.

[0035] 4. This invention employs self-training, which has several advantages. First, it allows the model to learn richer linguistic knowledge and semantic representations in an unsupervised environment, thereby improving the model's robustness, expanding its application scope, and making it suitable for a wider range of natural language processing tasks. Second, it allows training with unlabeled data, reducing the cost of data labeling and expanding the training dataset to improve the model's generalization ability. Third, it allows the model to learn richer linguistic knowledge and semantic representations in an unsupervised environment, thereby improving the model's classification performance and accuracy. Fourth, it allows the model to learn more interpretable semantic representations, improving the model's interpretability and making the model's prediction results more reliable and trustworthy.

[0036] 5. This invention combines the masking mechanism with self-training. Self-training allows for unsupervised model training, while the masking mechanism optimizes the model under supervised conditions. This combination not only updates sentence semantics but also generates weak tags, resulting in a dual improvement in model robustness and generalization ability. It enhances classification performance and accuracy, expands the model's application scope, and makes it suitable for a wider range of natural language processing tasks. The masking mechanism improves model interpretability, making predictions more reliable and trustworthy, while self-training enables the model to learn more interpretable semantic representations. The combination further enhances interpretability. This invention also combines Zipf's law with the masking mechanism and self-training. Zipf's law can be used to remove noise and irrelevant words from the category vocabulary, which is more conducive to supervised optimization of the model by the masking mechanism and unsupervised model training by self-training. Attached Figure Description

[0037] Figure 1 This is a diagram of the weakly supervised text classification framework for enhanced label semantics of the present invention;

[0038] Figure 2 This is a diagram illustrating the process of constructing category words based on contextual representation;

[0039] Figure 3 This is a diagram of the self-training learning framework based on label semantics. Detailed Implementation

[0040] To make the above features and advantages of the present invention more apparent and understandable, a detailed description is provided below in conjunction with the accompanying drawings.

[0041] This invention proposes a weakly supervised text classification method with enhanced label semantics, and implements it based on a weakly supervised text classification system with enhanced label semantics. This system includes a vocabulary generator, a sample labeler, and a sample classifier, such as... Figure 1 As shown.

[0042] The specific processing procedure is explained below:

[0043] 1. The construction of category vocabularies based on contextual semantic representation is handled by the vocabulary generator, such as... Figure 2 As shown, Zipf's law is used to optimize category semantics in semantic representation learning. During vocabulary construction, the correlation between category words and category representations is compared to filter out high-noise semantics, thereby achieving denoising of the category vocabulary. Specifically, the following steps are included:

[0044] (1) Generating a Candidate Word List: For text data that needs to be classified, such as news data corpora, a candidate word list is constructed using generative methods. These candidate words may be category-related vocabulary. Generative methods are model-based natural language processing methods that learn language models to generate text that conforms to grammatical and semantic rules. This method is usually based on probabilistic models, such as n-gram models, recurrent neural network (RNN) models, variational autoencoder (VAE) models, etc. In generative methods, the model learns a probability distribution that describes the probability of the next word appearing given some context. Based on this probability distribution, sampling methods can be used to generate new word sequences, thereby constructing a candidate word list. These generated words are usually based on the context of the input text data and the language model knowledge that has been learned. In the process of constructing a candidate word list using generative methods, it is usually necessary to control the number of generated words and the grammatical and semantic correctness of the generated words. To this end, some techniques, such as beamsearch and rule-based filtering methods, can be used to screen the generated word sequences to ensure that the generated candidate word list meets certain quality standards.

[0045] (2) Calculate the context representation: For each word in the candidate word list, the average of all contextualized representations of the word in the input corpus is used as its context representation. These representations can be word vectors or words in the context window.

[0046] (3) Calculate the relevance: Calculate the relevance (also known as relevance or similarity) between each word and the category representation, for example, using cosine similarity to calculate the relevance.

[0047] (4) Calculate the category representation: Initially, the category representation is the context representation of the category name. Considering that the category label name and the category number have the highest relevance, and the relevance of other words gradually decreases, Zipf's law is introduced into the semantic calculation to optimize the category semantics. Zipf's law states that in most natural languages, the frequency of a word's occurrence is inversely proportional to the inverse of its rank. The category representation is calculated according to the following formula:

[0048]

[0049] Among them, h c The context representation of category c, i.e., the category representation, is obtained by averaging the context representations of each word in the vocabulary corresponding to that category, as shown by the formula; K c This indicates the size of the vocabulary corresponding to category c, which is the number of words contained in that category; Represents the context representation of the i-th word in the vocabulary. In this formula, the numerator is the weighted sum of the context representations of each word, where the weight is 1 / i, that is, decreasing according to Zipf's law; the denominator is the sum of the accumulated weights, that is, the sum of the reciprocals from 1 to K c The sum of the reciprocals is also decreasing according to Zipf's law.

[0050] (5) Construct a category vocabulary: Then, each time, select the word with the highest relevance as a candidate word and add it to the new category vocabulary, and recalculate the category representation and relevance. Repeat the above steps until the semantic relevance between all remaining words and the category representation is lower than the threshold σ.

[0051] 2. The category word prediction task based on the MASK mechanism is responsible for by the sample standardizer. In the process of generating weak annotations, this module compares the synonym list of the current word with the category vocabulary to generate word category annotations, and uses the MASK mechanism to utilize the context semantics to predict the category of the word, thereby realizing the training of model parameters under word-level matching supervision, and then realizing semantic enhancement. Specifically, it includes the following steps:

[0052] (1) Find category-indicative words: In the stage of finding category-indicative words, for each word in the sentence, use the BERT model to find the top-k most relevant words, such as "enterprise", "basketball", "business", "development", etc.; then compare these top-k words with the category vocabulary of each category, that is, perform vocabulary matching, and select the category (such as "sports") whose intersection exceeds m (m < k) and has the largest intersection as the indicative category of this word (such as "basketball"), and this word is the category-indicative word of this indicative category. In the entire corpus, through this step, the set S composed of category-indicative words w and their corresponding word categories c can be obtained w to construct word-level supervision for category-indicative words. ind

[0053] (2) Word category prediction: In the stage of word category prediction, for each category-indicative word w, use the MASK mechanism (that is, adopt the "[MASK]" term) to mask the category-indicative word; then train the classifier through cross-entropy loss, so that the model can rely on the context representation of word w to predict the category label of this word. The calculation formula is as follows:

[0054]

[0055] p(c w |h w ) = Softmax(Wh w + b);

[0056] where, L1 represents the loss function, p(cw |h w ) represents the context representation h of a given word w. w Below, the word 'w' belongs to category 'c'. w The probability, and These are the model parameters and biases of the linear layer, respectively, and K is the number of samples in the target distribution.

[0057] The advantage of using a masking mechanism is that it allows the model to predict the category of category indicator words based on contextual representations, rather than simply memorizing category keywords. In this way, BERT can learn better word embeddings, thus aiding in word category prediction.

[0058] 3. A self-training module based on label semantics, managed by the sample classifier, such as... Figure 3 As shown, the structure includes BERT, a semantic similarity module, and a Softmax classifier. The self-training module is enhanced by comparing the similarity between the sentence vector representation and the label representation, achieving the transformation from word-level category prediction to sentence-level category prediction, and thus combating label noise.

[0059] Specifically, the self-training module first generates sentence vector representations using BERT, for example... Figure 3 The sentence vector representation containing "basketball" is used, with the term "[CLS]" as the sentence vector representation. A Softmax classifier predicts the distribution based on the sentence vector representation. Then, label embedding is introduced to learn the label semantics based on the category label representation. Label embedding refers to using embedding techniques to map labels to a low-dimensional space, allowing the semantic relationships between labels to be well preserved in this space. Finally, the semantic similarity module combines the label embedding with the sentence vector representation generated by BERT to calculate the semantic similarity between the two, thus calculating the true label distribution Q. The calculation formula is as follows:

[0060]

[0061] Where V is the label embedding representation, W is the fully connected network parameter of the semantic similarity module, and v i Let be the sentence representation vector, b be the bias, q1 be the label distribution calculated by the self-training module, and y be the... one_hot Let be the one-hot vector of the label, α be the hyperparameter, and Q be the calculated true label distribution.

[0062] During self-training in the self-training module, metrics such as cross-loss entropy and KL divergence can be used as objective functions to train the model for text classification tasks on the dataset.

[0063] Calculate the cross-entropy loss during model training using the following formula:

[0064]

[0065] Where, p ic Let y be the probability that the i-th training sample belongs to the c-th class. ic Let N be the label of the i-th training sample corresponding to the c-th category, M be the total number of training samples, and L be the total number of training categories. i Let be the cross-entropy loss value of the i-th sample. As shown in the formula, when the predicted probability is very small, the loss value will be very high, and vice versa.

[0066] This invention uses KL divergence to determine the target distribution Q, as shown in the following formula:

[0067]

[0068] Where p(x) is the true distribution corresponding to the training sample x, and q(x) is the predicted distribution corresponding to the training sample. When the predicted distribution and the true label distribution are perfectly matched, the KL divergence value is 0.

[0069] As shown in the cross-entropy loss formula, due to the low prediction probability of incorrect labels in the initial training stage, traditional hard labeling not only loses the semantic distribution of correct labels but also leads to a larger loss calculation value, making the model more prone to overfitting incorrect labels. In contrast, the label distribution method proposed in this invention, compared to hard labeling, not only preserves the semantic distribution of correct labels but also reduces the loss calculation value, thereby mitigating error propagation and reducing overfitting of the model to label noise. This allows the model to have better stability and better generalization performance than hard labeling.

[0070] To make the objectives, solutions, and advantages of this invention clearer, the following experiments conducted on real datasets will be used as examples to further illustrate this invention in detail.

[0071] Taking the THUCNews dataset as an example, the training set and test set are divided into appropriate proportions. The title and description fields of the articles are retained, and the category labels are only used in the test. The specific implementation steps of the weakly supervised text classification method with enhanced label semantics proposed in this invention are described.

[0072] First, a context semantic representation method restricted by Zipf's law is adopted to construct a category vocabulary on the complete corpus. For each category name word that appears in the corpus, the MLM Head of BERT is used to predict its relevant words in the BERT vocabulary. In this embodiment, the vocabulary size is set to 30522, and the Chinese vocabulary size is 33586. Subsequently, the top-p words with the highest relevance are selected. In this embodiment, p = 50 in the English corpus and p = 30 in the Chinese corpus, and they are added to the category vocabulary as candidate words, and the occurrence frequency of each candidate word is counted. Then, they are sorted according to the occurrence frequency, and the top K words with the highest frequency are selected as the category vocabulary. In this embodiment, K = 100 in the English corpus and K = 60 in the Chinese corpus. Then, based on this category vocabulary, the context representation of words is obtained based on Zipf's law and combined with BERT to calculate the category representation and relevance. The above steps are repeated until the semantic similarity between the remaining words and the category representation is lower than the set threshold (the threshold is set to σ = 0.75 in this embodiment), and an optimized category vocabulary is obtained. Experiments show that after applying the category vocabulary denoising method using the context semantic representation method restricted by Zipf's law, the label prediction accuracy of the model is improved by 2.55% on the THUCNews dataset.

[0073] Secondly, the classification model M is trained using the word category prediction method based on the MASK mechanism introduced in this invention. The category indicative words in the sample are discovered using the category vocabulary. The method adopted in this embodiment is to traverse each word in the sample, and based on the semantic vector distance of the word in the BERT model, the top-k most relevant words are found. Then, these k words are compared with the category vocabulary of each category, and the category with an intersection exceeding m (m < k) and the largest intersection is selected as the category indicative word of this word. In the entire corpus, the above steps are repeated until all samples are completed, and finally the category indicative word w and the word categories c w constituting the set S ind . The MASK mechanism is used to predict the word categories of the category indicative words in the sample. During the initialization of the training process, is used as the objective function to obtain the category indicative word w and the word categories c w constituting the set S ind .

[0074] Finally, the total number of batches required for training is set, and a self-training framework is used to further optimize model M. Q is calculated in each batch using the self-training method mentioned in this invention, and model M is updated every 50 batches with a suitable loss function as the objective. In this implementation, cross-entropy loss is used as the loss function; other types of loss functions can also be used based on the framework of this invention. When training with cross-entropy loss, the cross-entropy loss is calculated, and the target distribution Q is calculated using KL-divergence, thus calculating the model loss value for each round. Experiments show that the word category self-training module based on the MASK mechanism can enhance label semantics in prediction, improving the model's label prediction accuracy by 2.76% on the THUCNews dataset.

[0075] This example achieves superior results compared to traditional methods in weakly supervised text classification tasks in languages ​​such as Chinese and English. The above description is merely one embodiment of the present invention and is not intended to limit the invention. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method of label semantic enhanced weakly supervised text classification, characterized in that, The method comprises the following steps: For text data to be classified, samples are extracted therefrom and a candidate word list is constructed for words in the samples using a generative method; For each word in the candidate word list, a context representation of the word is determined according to semantics of the word in the samples, and a correlation between the context representation and a category representation is calculated; The category representation of the word is calculated according to the correlation based on Zipf's law, and a category word table is constructed according to the word with the highest correlation selected each time; For each word in a sentence of the samples, an indicating category of the word is determined according to the constructed category word table using a BERT model, and the word is taken as a category indicating word; The step of determining the indicating category of the word according to the constructed category word table using the BERT model comprises: finding the most relevant k words using the BERT model, comparing the k words with a category word table of each category, and selecting a category with an intersection exceeding a threshold value and a largest intersection as the indicating category of the word; For each category indicating word, the category indicating word is masked using a MASK mechanism, and a classifier is trained using a cross-entropy loss to predict a category label of the category indicating word according to a context representation of the category indicating word; A self-training module composed of a BERT model, a semantic similarity module and a Softmax classifier is constructed, the self-training module generates a sentence vector representation for a sentence in the samples through the BERT model, predicts a label distribution according to the sentence vector representation through the Softmax classifier, and calculates a similarity between the sentence vector representation and a label representation through the semantic similarity module; the self-training module learns a label semantic of the category label according to the similarity, calculates a real label distribution of each sentence and performs self-training using the real label distribution; The text data to be classified is classified using the trained self-training module.

2. The method of claim 1, wherein, The method for determining the context representation of the word according to the semantics of the word in the samples comprises: averaging all contextualized representations of the word in the samples to obtain the context representation.

3. The method of claim 1, wherein, The correlation between the context representation of the word and the category representation is calculated using a cosine similarity.

4. The method of claim 1, wherein, In the calculation of the category representation of the word, the category representation at the initial time is a context representation of a category name.

5. The method of claim 1, wherein, In the construction of the category word table, if the correlations between the context representations of all remaining words and the category representations are lower than a threshold value after multiple word selections, the selection is stopped.

6. The method of claim 1, wherein, A set composed of the category indicating words and corresponding category words is constructed to perform word-level supervision on the category indicating words.

7. The method of claim 1, wherein, The self-training module obtains the category label representation through label embedding.

8. The method of claim 1, wherein, In the training of the self-training module, cross-entropy loss and KL divergence are taken as objective functions, the cross-entropy loss is a loss in the prediction of the label distribution, and the KL divergence is used to determine the real label distribution.

9. A weakly supervised text classification system with label semantic enhancement, characterized in that, The method comprises the following steps: A word table generator is responsible for extracting samples from text data to be classified and constructing a candidate word list for words in the samples using a generative method; For each word in the candidate word list, a context representation of the word is determined according to semantics of the word in the samples, and a correlation between the context representation and a category representation is calculated; The category representation of the word is calculated based on the Zipf law according to the correlation, and a category word table is constructed according to the word with the highest correlation selected each time; The sample marker is responsible for determining the indication category of each word in the sentence of the sample by using the BERT model according to the constructed category word table, and taking the word as the corresponding category indication word; For each category indication word, the category indication word is masked by using the MASK mechanism, and the classifier is trained by using the cross-entropy loss to predict the category label of the category indication word according to the context representation of the category indication word; The step of determining the indication category of the word by using the BERT model according to the constructed category word table includes: finding the most relevant k words by using the BERT model, comparing the k words with the category word table of each category, and selecting the category with the largest intersection exceeding a threshold as the indication category of the word; The sample classifier is responsible for generating a sentence vector representation for the sentence in the sample, and includes a self-training module composed of a BERT model, a semantic similarity module and a Softmax classifier. The self-training module generates a sentence vector representation for the sentence in the sample by the BERT model, predicts a label distribution according to the sentence vector representation by the Softmax classifier, and calculates the similarity between the sentence vector representation and the label representation by the semantic similarity module. The self-training module learns the label semantics of the category label according to the similarity, calculates the real label distribution of each sentence and performs self-training by using the real label distribution. The trained self-training module is used to classify the text data to be classified.

Images