Text classification method and device, electronic equipment and storage medium

Abstract

The present application relates to the field of artificial intelligence and digital medical technology, and provides a text classification method and device, electronic equipment and storage medium, based on pre-trained training of word level, the word vector representation of the extracted word can avoid the misplacement of word segmentation and lose the information of the word; In the fine-tuning stage, the word vector representation of the word is generated according to the word vector representation, based on the word vector representation, not only can effectively utilize the pre-training vector information, the word level vector is fused, also can effectively avoid the many problems caused by modifying pre-trained mode, and the word information of Chinese can be fused into the model, and the word segmentation of word level usually has a great effect in the scene of Chinese text classification, has a great influence on the final classification result of the model, improves the accuracy of semantic understanding of the text classification model, thereby improves the precision of the text classification model, and further improves the accuracy of text classification.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and digital medical technology, specifically to a text classification method, apparatus, electronic device, and storage medium. Background Technology

[0002] Text classification is of great significance in areas such as medical data, social media platforms, product reviews, email recognition, and sentiment analysis. With the rapid development of computer technology, using artificial intelligence for text classification has become quite common. With the rapid development of pre-trained models in the field of Natural Language Processing (NLP), the training of current NLP models mainly consists of two stages: pre-training and fine-tuning. In the pre-training stage, self-supervised methods are mainly used to pre-train on massive amounts of text corpora to obtain context token vectors. In the fine-tuning stage, these vectors are used to fine-tune the model on its own training corpus to obtain the final model.

[0003] In the process of developing this invention, the inventors discovered an inconsistency between pre-training and fine-tuning: in the pre-training stage, text is typically segmented into character-level tokens, while in the fine-tuning stage, it is further segmented into word-level tokens. This inconsistency between the pre- and fine-tuning inputs leads to a loss of accuracy in the text classification model. Existing technologies address this inconsistency by replacing character-level tokens with word-level tokens in the pre-training stage. However, due to the diverse combinations of Chinese words, using word-level tokens results in a significant increase in the vocabulary dimension during the pre-training stage, leading to poor vector quality and consequently, lower model accuracy. Summary of the Invention

[0004] In view of the above, it is necessary to propose a text classification method, device, electronic device and storage medium that can improve the accuracy of the text classification model, thereby improving the accuracy of text classification.

[0005] A first aspect of the present invention provides a text classification method, the method comprising:

[0006] Construct a training text set;

[0007] Obtain the character-level processing results and word-level processing results for each training text in the training text set;

[0008] Extract the character vector representation of each character from the character-level processing result;

[0009] Based on the multiple character vector representations corresponding to each word in the word-level processing results, a word vector representation for each word is generated;

[0010] A text classification model is obtained by training multiple word vector representations for each training text;

[0011] The text classification model described above is used to classify the text to be classified, and the classification result is obtained.

[0012] According to an optional embodiment of the present invention, the method further includes:

[0013] Obtain the sentence-level processing results for each training text in the training text set;

[0014] Based on the multiple word vector representations corresponding to each sentence in the sentence-level processing results, a sentence vector representation for each sentence is generated;

[0015] The text classification model is trained based on multiple sentence vector representations of each training text.

[0016] According to an optional embodiment of the present invention, the method further includes:

[0017] Obtain the sentence-level processing results for each training text in the training text set;

[0018] Based on the multiple word vector representations corresponding to each sentence in the sentence-level processing results, a sentence vector representation for each sentence is generated;

[0019] The text classification model is trained based on any two or all of the combinations of multiple character vector representations, multiple word vector representations, and multiple sentence vector representations of each training text.

[0020] According to an optional embodiment of the present invention, obtaining the word-level processing result of each training text in the training text set includes:

[0021] Multiple first keywords are extracted from the training text using a preset first extraction model;

[0022] Multiple second keywords are extracted from the training text using a preset second extraction model;

[0023] An initial keyword set is obtained based on the plurality of first keywords and the plurality of second keywords;

[0024] Identify the non-essential keywords in the initial keyword set;

[0025] The non-important keywords are removed from the initial keyword set to obtain the word-level processing result.

[0026] According to an optional embodiment of the present invention, classifying the text to be classified using the text classification model to obtain the classification result includes:

[0027] Obtain the character-level processing results and word-level processing results of the text to be classified;

[0028] Extract the unclassified character vector representation of each character in the unclassified character-level processing result;

[0029] Based on the multiple unclassified character vector representations corresponding to each word in the unclassified word level processing results, generate the unclassified word vector representation for each word;

[0030] The text classification model is used to classify the words based on multiple vector representations of the words to be classified, and the classification result is obtained.

[0031] According to an optional embodiment of the present invention, the method further includes:

[0032] Obtain the target training texts in the training text set that have the same category label as those in the classification results;

[0033] A word dictionary is constructed based on the word-level processing results corresponding to the target training text;

[0034] The matching degree of the category label is calculated based on the processing results of the word level to be classified and the word dictionary;

[0035] Determine whether the matching degree is greater than a preset matching degree threshold;

[0036] When the matching degree is greater than the preset matching degree threshold, the word dictionary is updated according to the processing result of the word to be classified, and the training text set is updated according to the text to be classified.

[0037] According to an optional embodiment of the present invention, constructing the training text set includes:

[0038] Get the original text;

[0039] The original text is cleaned to obtain cleaned text;

[0040] Label the cleaned text with category tags;

[0041] Construct text label pairs based on the cleaned text and the corresponding category labels;

[0042] The training text set is obtained based on multiple text label pairs.

[0043] A second aspect of the present invention provides a text classification apparatus, the apparatus comprising:

[0044] The construction module is used to construct the training text set;

[0045] The processing module is used to obtain the character-level processing results and word-level processing results for each training text in the training text set;

[0046] The extraction module is used to extract the character vector representation of each character in the character-level processing result;

[0047] The generation module is used to generate a word vector representation for each word based on multiple character vector representations corresponding to each word in the word-level processing results;

[0048] The training module is used to train a text classification model based on multiple word vector representations of each training text;

[0049] The classification module is used to classify the text to be classified using the text classification model and obtain the classification result.

[0050] A third aspect of the present invention provides an electronic device comprising a processor and a memory, the processor being configured to implement the text classification method when executing a computer program stored in the memory.

[0051] A fourth aspect of the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the text classification method.

[0052] The text classification method, apparatus, electronic device, and storage medium provided by this invention address the problem of a significant increase in vocabulary dimensionality caused by the use of word-level methods in the pre-training stage. This invention does not affect traditional character-level pre-training, and extracting character vector representations avoids the loss of word information due to misalignment in word segmentation. Compared to the use of character-level methods in the fine-tuning stage, this invention generates word vector representations based on character vector representations during the fine-tuning stage. Based on these word vector representations, it not only effectively utilizes pre-trained vector information and fuses word-level vectors, but also effectively avoids many problems caused by modifying the pre-training method. Furthermore, it integrates Chinese word information into the model. Word-level segmentation typically plays a significant role in Chinese text classification, greatly influencing the final classification results of the model, improving the accuracy of semantic understanding in text classification models, thereby enhancing the precision of text classification models and ultimately improving the accuracy of text classification. Attached Figure Description

[0053] Figure 1 This is a flowchart of the text classification model training method provided in the embodiments of the present invention.

[0054] Figure 2 This is a network architecture diagram of the text classification model provided in an embodiment of the present invention.

[0055] Figure 3 This is a flowchart of another text classification model training method provided in an embodiment of the present invention.

[0056] Figure 4 This is a flowchart of another text classification model training method provided in an embodiment of the present invention.

[0057] Figure 5 This is a flowchart of the text classification method provided in an embodiment of the present invention.

[0058] Figure 6 This is a structural diagram of the text classification model training device provided in an embodiment of the present invention.

[0059] Figure 7 This is a structural diagram of the text classification device provided in an embodiment of the present invention.

[0060] Figure 8 This is a schematic diagram of the structure of the electronic device provided in an embodiment of the present invention. Detailed Implementation

[0061] To better understand the above-mentioned objects, features, and advantages of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. Unless otherwise specified, the embodiments of the present invention and the features thereof can be combined with each other.

[0062] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing an embodiment in one alternative implementation and is not intended to be limiting of the invention.

[0063] The text classification method provided in this embodiment of the invention is executed by an electronic device, and correspondingly, the text classification device runs in the electronic device.

[0064] The embodiments of this invention can standardize data processing based on artificial intelligence technology. Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0065] Foundational technologies for artificial intelligence generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0066] Example 1

[0067] Figure 1 This is a flowchart of a text classification model training method provided in Embodiment 1 of the present invention. The text classification model training method specifically includes the following steps. Depending on different requirements, the order of the steps in this flowchart can be changed, and some steps can be omitted.

[0068] S11, construct the training text set.

[0069] Before acquiring a trained text classification model, the electronic device needs to construct a training text set. Based on the constructed training text set, a text classification model for classifying texts is trained. The training text set may include medical texts, which may be electronic healthcare records, electronic personal health records, including medical records, electrocardiograms, medical images, and other electronic records with archival value.

[0070] Electronic devices can acquire multiple raw texts through web crawlers or by using texts from publicly available corpus databases. These raw texts can then be combined to form a training text set, which can be stored locally. The raw texts can be a sentence, a paragraph, an article, or a medical text.

[0071] In an optional implementation, constructing the training text set includes:

[0072] Get the original text;

[0073] The original text is cleaned to obtain cleaned text;

[0074] Label the cleaned text with category tags;

[0075] Construct text label pairs based on the cleaned text and the corresponding category labels;

[0076] The training text set is obtained based on multiple text label pairs.

[0077] In this embodiment, the original text can be a user's Chinese review of a product, which can be positive or negative. The original text can also be a medical electronic record, such as an outpatient medical record or disease test results. Furthermore, the original text can be a dialogue between an online doctor and patient during a smart consultation on a consultation platform.

[0078] Original product review texts or doctor-patient dialogues on online consultation platforms often contain a large number of useless characters such as spaces, tags, and repeated punctuation. Therefore, it is necessary to clean the original text to obtain cleaned text. The specific cleaning process is existing technology and will not be described in detail here.

[0079] Each original text can be annotated by professionals, and the category labels annotated by the professionals can be determined as the category labels of the original text. The category labels can be positive or negative sentiment.

[0080] The training text set G can be represented as G = {(x1, y1), ..., ( i y i ), ..., ( n y n )}, where (x i y i ) represents the data pair consisting of the i-th product review text and its corresponding category label in the training text set g, n represents the total number of training texts in the training text set G, and y represents the total number of training texts in the training text set G. i ∈{0,1}, where 1 represents a positive category label and 0 represents a negative category label.

[0081] S12, obtain the character-level processing results and word-level processing results for each training text in the training text set.

[0082] The electronic device segments each training text into characters, obtaining character-level processing results. Simultaneously, the electronic device also segments each training text into words, obtaining word-level processing results.

[0083] Each training text corresponds to one character-level processing result and one word-level processing result. Each character-level processing result includes multiple characters, and each word-level processing result includes multiple words. A word is defined as containing one, two, three, or more characters.

[0084] The character-level processing result obtained by segmenting the i-th training text into characters can be expressed as x. i =[t i1 , t i2 , ..., t ij , ..., t ip ], where t ij Let be the j-th character in the i-th training text, and p represent the length of the i-th training text, which is the total number of characters in the i-th training text.

[0085] Segmenting the i-th training text into words, the resulting word-level processing result can be expressed as x. i =[w i1 w i2 , ..., w ij , ..., w iq ], where w ij Let q represent the j-th word in the i-th training text, and let q represent the word length of the i-th training text, which is the total number of words in the i-th training text.

[0086] It should be understood that the total number of characters in each training text is necessarily greater than the total number of words in that training text, i.e., p>q.

[0087] In an optional implementation, obtaining the word-level processing result for each training text in the training text set includes:

[0088] Multiple first keywords are extracted from the training text using a preset first extraction model;

[0089] Multiple second keywords are extracted from the training text using a preset second extraction model;

[0090] An initial keyword set is obtained based on the plurality of first keywords and the plurality of second keywords;

[0091] Identify the non-essential keywords in the initial keyword set;

[0092] The non-important keywords are removed from the initial keyword set to obtain the word-level processing result.

[0093] The preset first extraction model can be a Named Entity Recognition (NER) model. The NER model can include BERT (Bidirectional Encoder Representations from Transformers), BiLSTM (bi-directional long short-term memory), and CRF (Conditional Random Field). The training text is processed by the BERT model to obtain the word vector representation in the text. Then, the BiLSTM model learns the contextual feature information of each word based on the word vector representation in the text and performs named entity recognition. Finally, the CRF model is used to process the entity sequence output by BiLSTM to extract keywords and obtain multiple first keywords.

[0094] The preset second extraction model can be a Rapid Automatic Keyword Extraction (RAKE) model. Based on the RAKE model, words that do not need to be focused on in the training text are removed, words that reflect the main content of the training text are retained, and words in the title of the training text are extracted and retained. The words that reflect the main content of the text and the words in the title of the text are merged to obtain multiple second keywords.

[0095] In some implementations, obtaining an initial keyword set based on the plurality of first keywords and the plurality of second keywords may include: unifying the word format based on the plurality of first keywords and the plurality of second keywords to obtain the initial keyword set. Specifically, unifying the word format means labeling first keywords and second keywords extracted in different ways so that the initial keyword set can be filtered using these labels later. For example, if the first keyword and the second keyword are the same, then a "NER-RAKE" label is added after both the first keyword and the second keyword. If the first keyword and the second keyword are different, then a "NER" label is added after the first keyword and a "RAKE" label is added after the second keyword.

[0096] In some implementations, identifying unimportant keywords in the initial keyword set may include: calculating the term frequency-inverse document frequency (TF-IDF) index for each keyword in the initial keyword set; identifying keywords with a TF-IDF index greater than a preset threshold as important keywords; and identifying keywords with a TF-IDF index less than the preset threshold as unimportant keywords. The term frequency-inverse document frequency (TF-IDF) index is a commonly used weighting technique in information retrieval and data mining to assess the importance of a keyword to an initial text set. A smaller TF-IDF indicates lower keyword importance, while a larger TF-IDF indicates higher keyword importance. Electronic devices have preset TF-IDF thresholds. For keywords with a TF-IDF index less than the preset threshold, since their importance to the training text is relatively weak, their presence or absence can be considered to have little impact on the semantic understanding of the training text. Therefore, keywords with weak importance to the training text are deleted, and the remaining keywords are considered important keywords.

[0097] The above optional implementation method extracts multiple keywords from the same training text using two different extraction models, obtains an initial keyword set based on the extracted keywords, then identifies unimportant keywords in the initial keyword set, and deletes the unimportant keywords from the initial keyword set to obtain the word-level processing result, thereby improving the accuracy of keyword extraction.

[0098] S13, extract the character vector representation of each character in the character-level processing result.

[0099] Electronic devices can input the processing results at the character level into a pre-trained model to obtain the character vector representation of each character in the character-level processing results.

[0100] The pre-trained model can be either a BERT model or a RoBERTa model (Robustlyoptimized BERT Pretraining approach).

[0101] S14. Based on the multiple character vector representations corresponding to each word in the word-level processing results, generate the word vector representation of each word.

[0102] Based on the obtained word-level processing results, the electronic device performs vector averaging on the character vector representations corresponding to multiple characters in the same word to obtain the word vector representation of that word.

[0103] See Figure 2 As shown, assume a certain training text is: The appearance and packaging of this product are very good. Segment the training text by character, and the character-level processing result is as follows Figure 2 shown: this|com|mod|ity|'s|ex|ter|nal|ap|pear|ance|and|pac|kag|ing|are|very|good. Segment the training text by word, and the word-level processing result is as follows Figure 2 shown: this|product|'s|external|appearance|and|packaging|are|very|good.

[0104] Extract the word vector representation of each character in the character-level processing result through a pre-trained model as follows:

[0105] The word vector representation of "this" is e1;

[0106] The word vector representation of "com" is e2;

[0107] The word vector representation of "mod" is e3;

[0108] The word vector representation of "ity" is e4;

[0109] The word vector representation of "ex" is e5;

[0110] The word vector representation of "ter" is e6;

[0111] The word vector representation of "nal" is e7;

[0112] The word vector representation of "ap" is e8;

[0113] The word vector representation of "pear" is e9;

[0114] The word vector representation of "ance" is e10;

[0115] The word vector representation of "and" is e11;

[0116] The word vector representation of "pac" is e12;

[0117] The word vector representation of "kag" is e13.

[0118] Based on the multiple word vector representations corresponding to each word in the word-level processing result, the word vector representation of each word generated is as follows:

[0119] The word vector representation of "this" is e1;

[0120] The word vector representation of "product" is the average of e2 and e3;

[0121] The word vector representation of "ity" is e4;

[0122] The word vector representation of "external appearance" is the average of e5 and e6;

[0123] The word vector representation of "and" is e7;

[0124] The word vector representation of "packaging" is the mean of e8 and e9;

[0125] The word vector representation of "both" is e10;

[0126] The word vector representation of "very" is the mean of e11 and e12;

[0127] The word vector representation of "good" is e13.

[0128] In the prior art, the word vector representation directly obtained by the pre-trained model is greatly affected by the word frequency, resulting in a closer distance between the word vector representations encoded by high-frequency words and being more concentrated near the origin. This will cause that even if the semantics of a high-frequency word and a low-frequency word are equivalent, the difference in word frequency will also bring a large distance deviation, so that the distance between the word vector representations cannot well represent the semantic relevance, resulting in a poor processing effect on natural language.

[0129] In the embodiments of the present invention, instead of obtaining the word vector representation through the pre-trained model, the character vector representation is first obtained through the pre-trained model, and then the word vector representation is obtained based on the multiple character vector representations corresponding to each word, which can not only avoid being affected by the word frequency, but also avoid the reduction of the accuracy of the word vector representation caused by word segmentation errors, and improve the performance of semantic understanding.

[0130] S15, training a text classification model based on the multiple word vector representations of each training text.

[0131] The electronic device inputs the obtained multiple word vector representations into the CNN network for training to obtain a text classification model.

[0132] In the embodiments of the present invention, a logistic regression function can be selected as the loss function, and the classification result output by the CNN network is input into the logistic regression loss function to obtain the final positive and negative representations of the model. In the model training stage, by continuously optimizing the parameters of the model, the loss of the model is reduced, and then the training of the model is completed. During the training process, the mini-batch gradient descent method can be used, k is the size of each batch, and the Dropout strategy is applied to randomly remove some hidden layer units and their weights with a certain probability to prevent overfitting of the training data.

[0133] In existing technologies, misalignment in word segmentation can lead to slot failures, significantly reducing the model's accuracy. Conversely, if word segmentation is omitted and modeling is performed directly on individual characters, word information is lost. The model assumes each character represents a single meaning, implying that combinations of characters convey different meanings, thus hindering its ability to understand Chinese text.

[0134] Compared to existing technologies that use word-level methods in the pre-training stage, resulting in a significant increase in vocabulary dimensionality, this invention does not affect traditional character-level pre-training. Extracting character vector representations also avoids word segmentation misalignment and loss of word information. Furthermore, compared to existing technologies that use character-level methods in the fine-tuning stage, this invention generates word vector representations based on character vector representations during the fine-tuning stage. Based on these word vector representations, it not only effectively utilizes pre-trained vector information and fuses word-level vectors, but also effectively avoids many problems caused by modifying the pre-training method. Simultaneously, it integrates Chinese word information into the model. Word-level segmentation typically plays a crucial role in Chinese text classification, significantly impacting the model's final classification results, improving the accuracy of semantic understanding in text classification models, and thus enhancing the precision of text classification models and ultimately improving the accuracy of text classification.

[0135] Example 2

[0136] Figure 3 This is a flowchart of a text classification model training method provided in Embodiment 2 of the present invention. The text classification model training method specifically includes the following steps. Depending on different requirements, the order of the steps in this flowchart can be changed, and some steps can be omitted.

[0137] S31, Construct the training text set.

[0138] S32, obtain the character-level processing results, word-level processing results, and sentence-level processing results for each training text in the training text set.

[0139] The electronic device segments each training text by character and word, and also by sentence, to obtain sentence-level processing results.

[0140] Electronic devices can segment training text based on preset symbol rules to obtain sentence-level processing results.

[0141] Specifically, the preset symbol rules refer to: recognizing punctuation marks, segmenting the training text according to the position of the punctuation marks, thereby dividing the training text into several independent sentences.

[0142] Each training text corresponds to a sentence-level processing result, and each sentence-level processing result includes multiple sentences.

[0143] Segmenting the i-th training text into sentences, the resulting sentence-level processing result can be expressed as x. i =[s i1 s i2 , ..., s ij , ..., s is ], where s ij Let be the j-th word in the i-th training text, and s represent the sentence length of the i-th training text, which is the total number of sentences in the i-th training text.

[0144] It should be understood that the total number of characters in each training text is necessarily greater than the total number of words in that training text, i.e., p>q>s.

[0145] S33, Extract the character vector representation of each character in the character-level processing result.

[0146] S34. Based on the multiple character vector representations corresponding to each word in the word-level processing results, generate the word vector representation of each word.

[0147] S35, Based on the multiple word vector representations corresponding to each sentence in the sentence-level processing results, generate a sentence vector representation for each sentence.

[0148] Based on the obtained word-level processing results, the electronic device performs vector averaging on the vector representations of multiple words corresponding to the same sentence to obtain the sentence vector representation of the sentence.

[0149] For example, the sentence vector representation of the sentence "The appearance and packaging of the product are very good" can be the mean of e1, e2 and e3, the mean of e4, e5 and e6, the mean of e7, e8 and e9, the mean of e10, e11 and e12, and the mean of e13.

[0150] S36, The text classification model is obtained by training multiple sentence vector representations for each training text.

[0151] The electronic device inputs the obtained sentence vector representations into a CNN network for training to obtain a text classification model.

[0152] In this embodiment of the invention, a logistic regression function can be selected as the loss function. The classification results output by the CNN network are input into the logistic regression loss function to obtain the final positive and negative representations of the model. During the model training phase, the model parameters are continuously optimized to reduce the model loss, thereby completing the model training. During training, a mini-batch gradient descent method can be used, with k being the batch size. A dropout strategy can be applied to randomly remove some hidden layer units and their weights with a certain probability to prevent overfitting of the training data.

[0153] In this embodiment, since extracting character vector representations takes less time, word vector representations are obtained based on character vector representations, and sentence vector representations are obtained based on word vector representations. This results in less extraction time and higher extraction efficiency for sentence vector representations. Furthermore, sentences have contextual coherence, so sentence vector representations can obtain contextual information about sentences, preserving the most complete semantic and structural information without loss. Text classification models trained based on sentence vector representations have higher accuracy, leading to more accurate text classification results.

[0154] Example 3

[0155] Figure 4 This is a flowchart of a text classification model training method provided in Embodiment 3 of the present invention. The text classification model training method specifically includes the following steps. Depending on different requirements, the order of the steps in this flowchart can be changed, and some steps can be omitted.

[0156] S41, Construct a training text set.

[0157] S42, obtain the character-level processing results, word-level processing results, and sentence-level processing results for each training text in the training text set.

[0158] S43, Extract the character vector representation of each character in the character-level processing result.

[0159] S44, Based on the multiple character vector representations corresponding to each word in the word-level processing results, generate the word vector representation of each word.

[0160] S45, Based on the multiple word vector representations corresponding to each sentence in the sentence-level processing results, generate a sentence vector representation for each sentence.

[0161] S46, the text classification model is trained based on any two or all of the combinations of multiple character vector representations, multiple word vector representations and multiple sentence vector representations of each training text.

[0162] The combination or all of any two of the multiple character vector representations, multiple word vector representations and multiple sentence vector representations of the training text include: multiple character vector representations and multiple word vector representations; multiple character vector representations and multiple sentence vector representations; multiple word vector representations and multiple sentence vector representations; multiple character vector representations, multiple word vector representations and multiple sentence vector representations.

[0163] In the above embodiments, based on any two or all of the multiple character vector representations, multiple word vector representations and multiple sentence vector representations of each training text, more semantic information can be obtained from two or three dimensions. More semantic information helps to improve the accuracy of the text classification model, thereby improving the accuracy of the text classification model and thus improving the accuracy of the text classification results.

[0164] Example 4

[0165] Figure 5 This is a flowchart of a text classification method provided in Embodiment 4 of the present invention. The text classification method specifically includes the following steps. Depending on different needs, the order of the steps in the flowchart can be changed, and some steps can be omitted.

[0166] S51, Obtain the trained text classification model.

[0167] In this embodiment, the electronic device can train the text classification model using the text classification model training method described in Embodiment 1, the text classification model training method described in Embodiment 2, or the text classification model training method described in Embodiment 3.

[0168] In other embodiments, the electronic device may also simultaneously train a text classification model using the text classification model training methods described in Embodiments 1, 2, and 3, and then combine these three text classification models into a complete text classification model.

[0169] S52, use the text classification model to classify the text to be classified and obtain the classification result.

[0170] The text to be classified refers to the text that needs to be classified.

[0171] The electronic device responds to the instruction to classify the text to be classified by inputting the text to be classified into the trained text classification model, and outputs the classification result of the text to be classified through the text classification model.

[0172] The classification result includes the category label of the text to be classified and the category probability of that category label. The category label indicates the category to which the text to be classified belongs, for example, positive sentiment or negative sentiment. The category probability indicates the confidence level that the text to be classified belongs to that category.

[0173] In an optional implementation, if the electronic device trains the text classification model using the text classification model training method described in Embodiment 1, then the text classification model is used to classify the text to be classified, and the classification result includes:

[0174] Obtain the character-level processing results and word-level processing results of the text to be classified;

[0175] Extract the unclassified character vector representation of each character in the unclassified character-level processing result;

[0176] Based on the multiple unclassified character vector representations corresponding to each word in the unclassified word level processing results, generate the unclassified word vector representation for each word;

[0177] The text classification model is used to classify the words based on multiple vector representations of the words to be classified, and the classification result is obtained.

[0178] The electronic device segments the text to be classified into characters, obtaining character-level processing results. Simultaneously, the electronic device also segments the text into words, obtaining word-level processing results.

[0179] The vector average operation is performed on the multiple unclassified character vector representations corresponding to each word in the unclassified word level processing results to obtain the unclassified word vector representation of each word.

[0180] In an optional implementation, if the electronic device trains the text classification model using the text classification model training method described in Embodiment 2, then the text to be classified is classified using the text classification model, and the classification result includes:

[0181] Obtain the processing results at the character level, word level, and sentence level of the text to be classified.

[0182] Extract the unclassified character vector representation of each character in the unclassified character-level processing result;

[0183] Based on the multiple unclassified character vector representations corresponding to each word in the unclassified word level processing results, generate the unclassified word vector representation for each word;

[0184] Based on the multiple unclassified word vector representations corresponding to each sentence in the unclassified sentence-level processing results, generate the unclassified sentence vector representation for each sentence;

[0185] The text classification model is used to classify sentences based on multiple vector representations of the sentences to be classified, and the classification results are obtained.

[0186] In an optional implementation, if the electronic device trains the text classification model using the text classification model training method described in Embodiment 3, then the text to be classified is classified using the text classification model, and the classification result includes:

[0187] Obtain the processing results at the character level, word level, and sentence level of the text to be classified.

[0188] Extract the unclassified character vector representation of each character in the unclassified character-level processing result;

[0189] Based on the multiple unclassified character vector representations corresponding to each word in the unclassified word level processing results, generate the unclassified word vector representation for each word;

[0190] Based on the multiple unclassified word vector representations corresponding to each sentence in the unclassified sentence-level processing results, generate the unclassified sentence vector representation for each sentence;

[0191] The text classification model is used to classify texts based on any two or all of the following: multiple vector representations of characters to be classified, multiple vector representations of words to be classified, and multiple vector representations of sentences to be classified, to obtain the classification result.

[0192] The combination or all of any two of the multiple character vector representations to be classified, the multiple word vector representations to be classified, and the multiple sentence vector representations to be classified include: multiple character vector representations to be classified and multiple word vector representations to be classified; multiple character vector representations to be classified and multiple sentence vector representations to be classified; multiple word vector representations to be classified and multiple sentence vector representations to be classified.

[0193] It should be noted that, if Embodiment 3 is a text classification model trained based on multiple character vector representations and multiple word vector representations of the training text, then the text classification model is used to classify based on multiple character vector representations and multiple word vector representations to be classified, and a classification result is obtained. If Embodiment 3 is a text classification model trained based on multiple character vector representations and multiple sentence vector representations of the training text, then the text classification model is used to classify based on multiple character vector representations and multiple sentence vector representations to be classified, and a classification result is obtained. If Embodiment 3 is a text classification model trained based on multiple character vector representations, multiple word vector representations, and multiple sentence vector representations of the training text, then the text classification model is used to classify based on multiple character vector representations, multiple word vector representations, and multiple sentence vector representations to be classified, and a classification result is obtained.

[0194] In an optional implementation, the method further includes:

[0195] Obtain the target training texts in the training text set that have the same category label as those in the classification results;

[0196] A word dictionary is constructed based on the word-level processing results corresponding to the target training text;

[0197] The matching degree of the category label is calculated based on the processing results of the word level to be classified and the word dictionary;

[0198] Determine whether the matching degree is greater than a preset matching degree threshold;

[0199] When the matching degree is greater than the preset matching degree threshold, the word dictionary is updated according to the processing result of the word to be classified, and the training text set is updated according to the text to be classified.

[0200] In this embodiment, one category label corresponds to one word dictionary, and each word dictionary includes all the words (keywords) in the training text corresponding to the category label.

[0201] Each word in the word-level processing result to be classified is matched with the word dictionary to determine whether a word in the word-level processing result to be classified is a matching word. When a word identical to a word in the word dictionary is matched with a word in the word-level processing result to be classified, the word in the word-level processing result to be classified is determined to be a matching word; when no word identical to a word in the word dictionary is matched with a word in the word-level processing result to be classified, the word in the word-level processing result to be classified is determined to be a non-matching word.

[0202] Calculate the first total number of words in the word-level processing result of the text to be classified; calculate the second total number of matching words in the word-level processing result of the text to be classified; calculate the ratio of the second total number of words to the first total number of words to obtain the matching degree. A higher matching degree indicates a higher degree of agreement between the text to be classified and the training text with the same category label. A lower matching degree indicates a lower degree of agreement between the text to be classified and the training text with the same category label.

[0203] When the matching degree is greater than the preset matching degree threshold, the words in the word-level processing to be classified are written into the word dictionary and deduplicated, thereby updating the word dictionary. When the matching degree is greater than the preset matching degree threshold, the electronic device stores the text to be classified and adds it to the training text set to obtain an updated training text set, which is then used to update the text classification model. When the matching degree is not greater than the preset matching degree threshold, the text to be classified is deleted.

[0204] The above optional implementation calculates the matching degree of the category labels of the text to be classified, and updates the word dictionary based on the word-level processing results of the text to be classified when the matching degree is greater than a preset matching degree threshold. It also updates the training text set based on the text to be classified, thereby expanding the number of words in the word dictionary and the number of training texts. Expanding the word dictionary and training texts can further improve the training accuracy of the text classification model, thus improving the accuracy of text classification.

[0205] Example 5

[0206] Figure 6 This is a structural diagram of the text classification model training device provided in Embodiment 5 of the present invention.

[0207] In some embodiments, the text classification model training device 60 may include multiple functional modules composed of computer program segments. The computer programs of each program segment in the text classification model training device 60 may be stored in the memory of an electronic device and executed by at least one processor to perform the function of text classification model training.

[0208] In this embodiment, the text classification model training device 60 can be divided into multiple functional modules according to its functions. These functional modules may include: a construction module 601, an acquisition module 602, an extraction module 603, a generation module 604, and a training module 605. The module referred to in this invention is a series of computer program segments that can be executed by at least one processor and perform a fixed function, stored in memory. In this embodiment, the functions of each module will be detailed in subsequent embodiments.

[0209] The construction module 601 is used to construct a training text set.

[0210] The acquisition module 602 is used to acquire the word-level processing results of each training text in the training text set.

[0211] The acquisition module 602 is also used to acquire the word-level processing results of each training text in the training text set.

[0212] The acquisition module 602 is also used to acquire the sentence-level processing results of each training text in the training text set.

[0213] The extraction module 603 is used to extract the character vector representation of each character in the character-level processing result.

[0214] The generation module 604 is used to generate a word vector representation for each word based on multiple character vector representations corresponding to each word in the word-level processing results.

[0215] The generation module 604 is further configured to generate a sentence vector representation for each sentence based on multiple word vector representations corresponding to each sentence in the sentence-level processing results.

[0216] The training module 605 is used to train a text classification model based on multiple word vector representations of each training text.

[0217] The training module 605 is also used to train the text classification model based on multiple sentence vector representations of each training text.

[0218] The training module 605 is further configured to train the text classification model based on any two or all of the multiple character vector representations, multiple word vector representations and multiple sentence vector representations of each training text.

[0219] Specifically, the training module 605 trains the text classification model based on any two or all of the combinations of multiple character vector representations, multiple word vector representations, and multiple sentence vector representations of each training text, including any one of the following:

[0220] The text classification model is trained based on multiple character vector representations and multiple word vector representations of each training text;

[0221] The text classification model is trained based on multiple character vector representations and multiple sentence vector representations of each training text;

[0222] The text classification model is trained based on multiple word vector representations and multiple sentence vector representations of each training text;

[0223] The text classification model is trained based on multiple character vector representations, multiple word vector representations, and multiple sentence vector representations of each training text.

[0224] Example 6

[0225] Figure 7 This is a structural diagram of the text classification device provided in Embodiment Six of the present invention.

[0226] In some embodiments, the text classification device 70 may include a plurality of functional modules composed of computer program segments. The computer programs of each program segment in the text classification device 70 may be stored in the memory of an electronic device and executed by at least one processor to perform the function of text classification.

[0227] In this embodiment, the text classification device 70 can be divided into multiple functional modules according to its functions. The functional modules may include an acquisition module 701 and a classification module 702. The term "module" in this invention refers to a series of computer program segments that can be executed by at least one processor and perform a fixed function, and which are stored in memory. In this embodiment, the functions of each module will be described in detail in subsequent embodiments.

[0228] The acquisition module 701 is used to acquire the trained text classification model.

[0229] The classification module 702 is used to classify the text to be classified using the text classification model to obtain the classification result.

[0230] Example 7

[0231] This embodiment provides a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the steps described in the text classification model training method embodiment above, for example... Figure 1 S11-S15 as shown:

[0232] S11, Construct the training text set;

[0233] S12, obtain the character-level processing results and word-level processing results for each training text in the training text set;

[0234] S13, Extract the character vector representation of each character in the character-level processing result;

[0235] S14, Based on the multiple character vector representations corresponding to each word in the word-level processing results, generate the word vector representation of each word;

[0236] S15, a text classification model is trained based on the multiple word vector representations of each training text to obtain a text classification model.

[0237] When executed by a processor, the computer program also implements the steps in the above-described text classification model training method embodiment, for example... Figure 3 S31-S36 as shown:

[0238] S31, Construct the training text set;

[0239] S32, obtain the character-level processing results, word-level processing results, and sentence-level processing results for each training text in the training text set;

[0240] S33, Extract the character vector representation of each character in the character-level processing result;

[0241] S34, Based on the multiple character vector representations corresponding to each word in the word-level processing results, generate the word vector representation of each word;

[0242] S35, Based on the multiple word vector representations corresponding to each sentence in the sentence-level processing results, generate a sentence vector representation for each sentence;

[0243] S36, The text classification model is obtained by training multiple sentence vector representations for each training text.

[0244] When executed by a processor, the computer program also implements the steps in the above-described text classification model training method embodiment, for example... Figure 4 S41-S46 as shown:

[0245] S41, Construct the training text set;

[0246] S42, obtain the character-level processing results, word-level processing results, and sentence-level processing results for each training text in the training text set;

[0247] S43, Extract the character vector representation of each character in the character-level processing result;

[0248] S44, Based on the multiple character vector representations corresponding to each word in the word-level processing results, generate the word vector representation of each word;

[0249] S45, Based on the multiple word vector representations corresponding to each sentence in the sentence-level processing results, generate a sentence vector representation for each sentence;

[0250] S46, the text classification model is trained based on any two or all of the combinations of multiple character vector representations, multiple word vector representations and multiple sentence vector representations of each training text.

[0251] When executed by a processor, the computer program also implements the steps described in the text classification method embodiments above, for example... Figure 5 S51-S52 shown:

[0252] S51, Obtain the trained text classification model;

[0253] S52, use the text classification model to classify the text to be classified and obtain the classification result.

[0254] Alternatively, when the computer program is executed by the processor, it implements the functions of each module / unit in the above-described device embodiments, for example... Figure 6 Modules 601-605 in the middle:

[0255] The construction module 601 is used to construct a training text set.

[0256] The acquisition module 602 is used to acquire the word-level processing results of each training text in the training text set.

[0257] The acquisition module 602 is also used to acquire the word-level processing results of each training text in the training text set.

[0258] The acquisition module 602 is also used to acquire the sentence-level processing results of each training text in the training text set.

[0259] The extraction module 603 is used to extract the character vector representation of each character in the character-level processing result.

[0260] The generation module 604 is used to generate a word vector representation for each word based on multiple character vector representations corresponding to each word in the word-level processing results.

[0261] The generation module 604 is further configured to generate a sentence vector representation for each sentence based on multiple word vector representations corresponding to each sentence in the sentence-level processing results.

[0262] The training module 605 is used to train a text classification model based on multiple word vector representations of each training text.

[0263] The training module 605 is also used to train the text classification model based on multiple sentence vector representations of each training text.

[0264] The training module 605 is further configured to train the text classification model based on any two or all of the multiple character vector representations, multiple word vector representations and multiple sentence vector representations of each training text.

[0265] Alternatively, when the computer program is executed by the processor, it also implements the functions of each module / unit in the above-described device embodiments, for example... Figure 7 Modules 701-702 in the middle:

[0266] The acquisition module 701 is used to acquire the trained text classification model.

[0267] The classification module 702 is used to classify the text to be classified using the text classification model to obtain the classification result.

[0268] Example 8

[0269] See Figure 8 The diagram shown is a structural schematic of an electronic device provided in Embodiment 7 of the present invention. In a preferred embodiment of the present invention, the electronic device 8 includes a memory 81, at least one processor 82, at least one communication bus 83, and a transceiver 84.

[0270] Those skilled in the art should understand that Figure 8 The structure of the electronic device shown does not constitute a limitation of the embodiments of the present invention. It can be a bus structure or a star structure. The electronic device 8 may also include more or fewer other hardware or software than shown, or different component arrangements.

[0271] In some embodiments, the electronic device 8 is a device capable of automatically performing numerical calculations and / or information processing according to pre-set or stored instructions. Its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), programmable gate arrays (FPGAs), digital processors, and embedded devices. The electronic device 8 may also include client devices, including, but not limited to, any electronic product capable of human-computer interaction with a client via a keyboard, mouse, remote control, touchpad, or voice control device, such as personal computers, tablets, smartphones, and digital cameras.

[0272] The electronic device 8 described herein is merely an example. Other existing or future electronic products that are adaptable to this invention should also be included within the scope of protection of this invention and are incorporated herein by reference.

[0273] In some embodiments, the memory 81 stores a computer program that, when executed by the at least one processor 82, implements all or part of the steps in the text classification method described above. The memory 81 includes read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), one-time programmable read-only memory (OTPROM), electrically-erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, disk storage, magnetic tape storage, or any other computer-readable medium capable of carrying or storing data.

[0274] Furthermore, the computer-readable storage medium may primarily include a program storage area and a data storage area, wherein the program storage area may store the operating system, at least one application required for a function, etc.; and the data storage area may store data created based on the use of blockchain nodes, etc.

[0275] The blockchain referred to in this invention is a novel application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Essentially, a blockchain is a decentralized database, a chain of data blocks linked together using cryptographic methods. Each data block contains information about a batch of network transactions, used to verify the validity of the information (anti-counterfeiting) and generate the next block. A blockchain can include an underlying blockchain platform, a platform product service layer, and an application service layer.

[0276] In some embodiments, the at least one processor 82 is the control unit of the electronic device 8, connecting various components of the electronic device 8 via various interfaces and lines. It executes programs or modules stored in the memory 81 and calls data stored in the memory 81 to perform various functions and process data. For example, when the at least one processor 82 executes a computer program stored in the memory, it implements all or part of the steps of the text classification method described in this embodiment of the invention; or it implements all or part of the functions of the text classification device. The at least one processor 82 may be composed of integrated circuits, such as a single-packaged integrated circuit or multiple integrated circuits with the same or different functions, including combinations of one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips.

[0277] In some embodiments, the at least one communication bus 83 is configured to enable communication between the memory 81 and the at least one processor 82, etc.

[0278] Although not shown, the electronic device 8 may also include a power supply (such as a battery) to power the various components. Preferably, the power supply can be logically connected to the at least one processor 82 via a power management device, thereby enabling functions such as charging, discharging, and power consumption management. The power supply may also include one or more DC or AC power supplies, recharging devices, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components. The electronic device 8 may also include various sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be described in detail here.

[0279] The integrated unit implemented as a software functional module described above can be stored in a computer-readable storage medium. This software functional module, stored in a storage medium, includes several instructions to cause a computer device (which may be a personal computer, electronic device, or network device, etc.) or processor to execute portions of the methods described in the various embodiments of the present invention.

[0280] In the several embodiments provided by this invention, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and other division methods may be used in actual implementation.

[0281] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0282] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional modules.

[0283] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the invention. Therefore, the embodiments should be considered illustrative and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be embraced within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims. Furthermore, it is clear that the word "comprising" does not exclude other elements, and the singular does not exclude the plural. Multiple elements or devices recited in the specification may also be implemented by a single element or device in software or hardware. The terms "first," "second," etc., are used to indicate names and do not indicate any particular order.

[0284] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A method of text classification, characterized by, The method includes: Construct a training text set; Obtaining the character-level processing result and word-level processing result for each training text in the training text set includes: extracting multiple first keywords from the training text using a preset first extraction model, wherein the first extraction model is a named entity recognition model; extracting multiple second keywords from the training text using a preset second extraction model, wherein the second extraction model is a fast automatic keyword extraction model; obtaining an initial keyword set based on the multiple first keywords and the multiple second keywords; identifying non-important keywords in the initial keyword set; and deleting the non-important keywords from the initial keyword set to obtain the word-level processing result. The processing result at each character level is input into the pre-trained model, and the character vector representation of each character in the character-level processing result is extracted. In the fine-tuning stage, based on the multiple character vector representations corresponding to each word in the word-level processing results, a word vector representation for each word is generated, including: based on the word-level processing results, the character vector representations corresponding to multiple characters in the same word are averaged to obtain the word vector representation for each word; A text classification model is obtained by training multiple word vector representations for each training text; The text classification model described above is used to classify the text to be classified, and the classification result is obtained.

2. The text classification method of claim 1, wherein, The method further includes: Obtain the sentence-level processing results for each training text in the training text set; Based on the multiple word vector representations corresponding to each sentence in the sentence-level processing results, a sentence vector representation for each sentence is generated; The text classification model is trained based on multiple sentence vector representations of each training text.

3. The text classification method of claim 1, wherein, The method further includes: Obtain the sentence-level processing results for each training text in the training text set; Based on the multiple word vector representations corresponding to each sentence in the sentence-level processing results, a sentence vector representation for each sentence is generated; The text classification model is trained based on any two or all of the combinations of multiple character vector representations, multiple word vector representations, and multiple sentence vector representations of each training text.

4. The text classification method according to any one of claims 1 to 3, characterized by, The process of classifying the text to be classified using the text classification model to obtain the classification results includes: Obtain the character-level processing results and word-level processing results of the text to be classified; Extract the unclassified character vector representation of each character in the unclassified character-level processing result; Based on the multiple unclassified character vector representations corresponding to each word in the unclassified word level processing results, generate the unclassified word vector representation for each word; The text classification model is used to classify the words based on multiple vector representations of the words to be classified, and the classification result is obtained.

5. The text classification method of claim 4, wherein, The method further includes: Obtain the target training texts in the training text set that have the same category label as those in the classification results; A word dictionary is constructed based on the word-level processing results corresponding to the target training text; The matching degree of the category label is calculated based on the processing results of the word level to be classified and the word dictionary; Determine whether the matching degree is greater than a preset matching degree threshold; When the matching degree is greater than the preset matching degree threshold, the word dictionary is updated according to the processing result of the word to be classified, and the training text set is updated according to the text to be classified.

6. The text classification method of claim 4, wherein, The constructed training text set includes: Get the original text; The original text is cleaned to obtain cleaned text; Label the cleaned text with category tags; Construct text label pairs based on the cleaned text and the corresponding category labels; The training text set is obtained based on multiple text label pairs.

7. A text classification device, characterized in that, The apparatus is used to implement the text classification method as described in any one of claims 1 to 6, the apparatus comprising: The construction module is used to construct the training text set; The processing module is used to obtain the character-level processing results and word-level processing results for each training text in the training text set; The extraction module is used to extract the character vector representation of each character in the character-level processing result; The generation module is used to generate a word vector representation for each word based on multiple character vector representations corresponding to each word in the word-level processing results; The training module is used to train a text classification model based on multiple word vector representations of each training text; The classification module is used to classify the text to be classified using the text classification model and obtain the classification result.

8. An electronic device, characterized in that, The electronic device includes a processor and a memory, the processor being configured to implement the text classification method as described in any one of claims 1 to 6 when executing a computer program stored in the memory.

9. A computer-readable storage medium storing a computer program thereon, characterized in that, When the computer program is executed by a processor, it implements the text classification method as described in any one of claims 1 to 6.

Images