A short text classification method and device, electronic equipment and storage medium
By splitting short texts into text units and generating translated text vectors, and using a preset weight matrix and classification model, the problem of low classification accuracy of short texts is solved, achieving higher classification accuracy and efficiency.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-09-27
- Publication Date
- 2026-04-07
AI Technical Summary
Short text classification has low accuracy, is easily affected by noise, and lacks text features, resulting in low accuracy in downstream tasks.
The short text is split into text units, the translation text of each text unit is determined, and the text unit vector and the translation text vector are generated by a pre-trained model. These are then concatenated into an augmented vector, which is then classified using a preset weight matrix and a classification model.
It improves the representation quantity and classification accuracy of short texts, enhances classification efficiency, and quickly determines the classification results of short texts by constructing a pre-defined classification model.
Smart Images
Figure CN117312917B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of data processing technology, and in particular to a short text classification method. Background Technology
[0002] Short text is a collection of texts with fewer representations, including phrases, sentences, paragraphs, etc. Compared with long text, short text contains fewer representations and can extract fewer semantic features. The lack of features in short text representations can lead to low accuracy in downstream tasks such as classification and clustering, and make it more susceptible to noise interference. Summary of the Invention
[0003] This invention provides a method for classifying short texts to address the shortcomings of low accuracy in short text classification in existing technologies, thereby improving the accuracy of text classification when there are few text features.
[0004] In a first aspect, the present invention provides a short text classification method, comprising:
[0005] Based on the target short text, determine the text units and the corresponding translated texts for the text units;
[0006] Input the text unit and the translated text into the pre-trained model to obtain the text unit vector and the translated text vector output by the pre-trained model;
[0007] Input the target enhancement vector into a preset weight matrix to obtain the preset dimension vector of the target short text output by the preset weight matrix;
[0008] Input the preset dimension vector of the target short text into the preset classification model to obtain a preset number of classification results corresponding to the preset dimension;
[0009] Wherein, the target enhancement vector is obtained by concatenating the text unit vector and the translated text vector; the preset weight matrix is obtained by training based on the sample triples; the preset classification model is obtained by training based on the preset dimension vectors of all samples and the sample classification result to which each preset dimension vector belongs;
[0010] In one embodiment, before determining the preset dimension vector of the target short text based on the target enhancement vector of the target short text and the preset weight matrix, the method further includes:
[0011] Construct sample triples based on the sample short text;
[0012] Input the first enhancement vector and the second enhancement vector of the sample triple into the sample weight matrix to obtain the first preset dimension vector and the second preset dimension vector of the sample output by the sample weight matrix;
[0013] Input the first preset dimension vector of the sample, the second preset dimension vector of the sample, and the sample ternary vector into the first neural network to obtain the binary activation value output by the first neural network;
[0014] The preset weight matrix is obtained based on the binary activation value, the sample classification label in the sample triplet, and the first preset function;
[0015] The sample triplet is determined based on the first sample augmentation vector, the second sample augmentation vector, and the sample classification label between the first sample augmentation vector and the second sample augmentation vector; the sample triplet vector is determined based on the first sample augmentation vector, the second sample augmentation vector, and the absolute value of the difference between the first sample augmentation vector and the second sample augmentation vector.
[0016] In one embodiment, before determining the preset dimension vector of the target short text based on the target enhancement vector of the target short text and the preset weight matrix, the method further includes:
[0017] Based on the sample preset dimension vector, determine a preset number of sample classification results corresponding to the sample preset dimension;
[0018] Input the sample's preset dimension vector into the sample classification model to obtain a preset number of classification activation values;
[0019] Based on the preset number of classification activation values, the preset number of sample classification results, and the second preset function, the sample classification model is processed to obtain the preset classification model;
[0020] In one embodiment, constructing sample triples based on sample short text includes:
[0021] The sample text units are obtained by segmenting the sample short texts according to the corpus classification database;
[0022] Determine the sample classification label for each sample text unit group, wherein the sample text unit group is a combination of the first sample text unit and the second sample text unit;
[0023] For any sample text unit group, determine the translation of the first sample text unit and the translation of the second sample text unit in the sample text unit group;
[0024] Input the first sample text unit and its translation into the pre-trained model to obtain the vector of the first sample text unit and the translation vector of the first sample text unit output by the pre-trained model; input the second sample text unit and its translation into the pre-trained model to obtain the vector of the second sample text unit and the translation vector of the second sample text unit output by the pre-trained model.
[0025] The first augmented vector of the sample is obtained by concatenating the vector of the first text unit of the sample and the translation vector of the first text unit of the sample; the second augmented vector of the sample is obtained by concatenating the vector of the second text unit of the sample and the translation vector of the second text unit of the sample; the sample triple is constructed based on the first augmented vector of the sample, the second augmented vector of the sample, and the sample classification label.
[0026] In one embodiment, determining the sample classification label for each sample text unit group includes:
[0027] If it is determined that the first text unit of the sample and the second text unit of the sample are of the same type of text, then the group of sample text units is determined to be the same tag;
[0028] If it is determined that the first text unit of the sample and the second text unit of the sample are different types of text, then the sample text unit group is determined to be different tags;
[0029] In one embodiment, the translation vector is any of the following vectors:
[0030] The vector corresponding to any language;
[0031] Alternatively, a vector set composed of multiple languages.
[0032] Secondly, embodiments of the present invention provide a short text classification device, comprising:
[0033] A text determination unit is used to determine text units and corresponding translation texts based on the target short text.
[0034] A vector determination unit is used to determine a preset dimension vector of the target short text based on the target enhancement vector of the target short text and a preset weight matrix.
[0035] A weight vector determination unit is used to determine a preset dimension vector of the target short text based on the target enhancement vector of the target short text and a preset weight matrix.
[0036] A text classification unit is used to input a preset dimension vector of the target short text into a preset classification model to obtain a preset number of classification results corresponding to the preset dimension;
[0037] Wherein, the target enhancement vector is obtained by concatenating the text unit vector and the translated text vector; the preset weight matrix is obtained by training based on the sample triples; the preset classification model is obtained by training based on the preset dimension vectors of all samples and the sample classification result to which each preset dimension vector belongs.
[0038] Thirdly, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement any of the short text classification methods described above.
[0039] Fourthly, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the short text classification method as described above.
[0040] Fifthly, the present invention also provides a computer program product, including a computer program that, when executed by a processor, implements any of the short text classification methods described above.
[0041] The short text classification method, apparatus, electronic device, and storage medium provided by this invention can effectively increase the number of representations of short texts and improve the classification accuracy of short texts by splitting short texts into text units, determining the translated text of each text unit, merging the text representations in the text units and the translated text, and generating a preset dimension vector of enhanced text unit representations. Furthermore, by constructing a preset classification model, the classification result of short texts can be quickly determined through the preset dimension vector, thereby improving the classification efficiency of short texts. Attached Figure Description
[0042] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.
[0043] Figure 1 This is a schematic diagram of the short text classification method provided in an embodiment of the present invention;
[0044] Figure 2 This is a flowchart illustrating the process of determining a preset weight matrix provided in an embodiment of the present invention;
[0045] Figure 3 This is a schematic diagram of one of the model structures of the sample weight matrix provided in an embodiment of the present invention;
[0046] Figure 4 This is a schematic diagram of the process for constructing sample triples provided in an embodiment of the present invention;
[0047] Figure 5 This is a schematic diagram of the process for training a preset classification model provided in an embodiment of the present invention;
[0048] Figure 6 This is a schematic diagram of one of the model structures of the preset classification model provided in the embodiments of the present invention;
[0049] Figure 7 This is a schematic diagram of the short text classification device provided in an embodiment of the present invention;
[0050] Figure 8 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation
[0051] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.
[0052] Figure 1 This is a schematic diagram of the short text classification method provided in an embodiment of the present invention, such as... Figure 1 As shown, this short text classification method includes:
[0053] S110. Based on the target short text, determine the text units and the corresponding translation texts for each text unit;
[0054] S120: Input text units and translated text into the pre-trained model to obtain the text unit vector and translated text vector output by the pre-trained model;
[0055] S130. Input the target enhancement vector into the preset weight matrix to obtain the preset dimension vector of the target short text output by the preset weight matrix;
[0056] S140. Input the preset dimension vector of the target short text into the preset classification model to obtain a preset number of classification results corresponding to the preset dimension;
[0057] Among them, the target augmentation vector is obtained by concatenating the text unit vector and the translated text vector; the preset weight matrix is obtained by training based on the sample triples; and the preset classification model is obtained by training based on the preset dimension vectors of all samples and the sample classification result corresponding to the preset dimension vector of each sample.
[0058] It should be noted that the execution subject of the short text classification method provided in this embodiment of the invention can be a computer device, such as a mobile phone, tablet computer, laptop computer, handheld computer, in-vehicle electronic device, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. This invention does not limit the scope of the invention, and this invention uses a short text classification system as the execution subject for description.
[0059] In S110, the target short text refers to the short text that needs to be classified. The text unit can be a component of the target short text obtained by splitting the target short text according to a certain granularity. For example, if the target short text is a document, the target short text is split into sentences to obtain 10 sentences corresponding to the target short text, where each sentence in the 10 sentences represents a text unit.
[0060] It is understandable that the method of splitting the target short text into text units can be word-by-word, sentence-by-sentence, or paragraph-by-paragraph, and this invention does not limit this method.
[0061] It should be noted that determining the corresponding translation text for each text unit means that each text unit can correspond to one or more translation texts. For example, if a text unit is a Chinese sentence, its corresponding translation text can be an English sentence translated from the Chinese sentence in that text unit, or it can be a collection of English sentences, French sentences, and German sentences translated from the Chinese sentence in that text unit.
[0062] Understandably, machine translation can be used, which is more efficient and accurate than human translation, thus improving the accuracy of short text classification.
[0063] Understandably, supplementing the textual representation of short texts with translated texts can effectively increase the accuracy of short text classification. Furthermore, by increasing the variety of translated texts, the number of acquired textual representations can be further increased, thereby improving the accuracy of short text classification.
[0064] In S120, the text unit vector and the translated text vector are obtained by converting the text unit and the translated text into vector representations. This conversion method can be implemented using existing pre-trained models, such as semantic representation models (Embeddings from Language Models, ELMo), language representation models (Bidirectional Encoder Representation from Transformers, BERT), natural language processing models (Generative Pre-Trained Transformer, GPT), etc. This invention does not limit these methods.
[0065] In S130, the target augmentation vector refers to the concatenation of the text unit vector and the translated text vector obtained through the pre-trained model into a single vector. The concatenation method can be vertical concatenation, horizontal concatenation, etc., which are not limited in this invention.
[0066] It is understandable that by concatenating two vectors to obtain the target augmentation vector, the text unit vector and the text representation in the translated text vector can be combined together, which is equivalent to increasing the number of text representations, which is beneficial for subsequent text classification.
[0067] It should be noted that by using a preset weight matrix, the augmented vector can be converted into a vector of a preset dimension. For example, when the dimension of the text vector unit is m and the dimension of the translated text vector is n, the dimension of the augmented vector can be determined to be m+n. If the preset dimension is k, by setting a preset weight matrix of (m+n)*k, the dimension of the augmented vector can be converted into a vector of k dimensions.
[0068] It is understood that the present invention converts the augmented vector into a preset dimension vector by using a preset weight matrix, which is equivalent to integrating the text representation information in the augmented vector into the preset dimension vector. This is beneficial for subsequent processing of the preset dimension vector according to the preset model to obtain the classification result.
[0069] In S140, it can be understood that the number of classification results output is determined by the dimension of the preset dimension vector of the output of the preset weight matrix in S130. For example, if the dimension of the preset dimension vector of the output of the preset weight matrix in S130 is k, the number of classification results output in S140 is also k.
[0070] It should be noted that the preset classification model is trained based on a preset dimension vector and its corresponding classification result. For example, the classification result of a text unit can be determined through an existing classification corpus, and the sample pair composed of the augmentation vector obtained from the text unit and the classification result can be used as the sample to train the preset classification model.
[0071] It should be noted that the classification results can be represented by classification labels, such as labels for sentiment analysis, subject classification, question answering tasks, intent recognition, and natural language inference. Each text corresponds to at least one of these labels. For example, the classification result of a text could be labeled as positive, negative, or neutral for sentiment analysis; financial, sports, military, or social for subject classification; yes or no for question answering tasks; weather query, song search, or random chat for intent recognition; and derived, contradictory, or neutral for natural language inference.
[0072] This invention provides a short text classification method that, by splitting short texts into text units, determining the translated text of each text unit, merging the text representations of the text units and the translated texts, and generating a preset dimension vector of the enhanced representation of the text units, can effectively increase the number of representations of short texts and improve the classification accuracy of short texts. Furthermore, by constructing a preset classification model, the classification result of short texts can be quickly determined through the preset dimension vector, thereby improving the classification efficiency of short texts.
[0073] In one embodiment, the translation vector is any of the following vectors:
[0074] The vector corresponding to any language;
[0075] Alternatively, a vector set composed of multiple languages.
[0076] It should be noted that classification can also achieve effective classification by using the vector corresponding to any language as the translation vector. However, the more languages there are, the more representations the translation vector set carries. Therefore, the classification result determined by the translation vector set composed of multiple languages is more accurate than the classification result determined by using the vector corresponding to a single language as the translation vector.
[0077] It is understood that when classifying texts using the short text classification method provided in this embodiment of the invention, the number of translated languages can be determined by the computing power of the hardware device. For example, when the computing power of the hardware device is strong, a vector set composed of more languages can be used as the translated vector. When the computing power of the hardware device is weak, the translated vector corresponding to any language can be used for classification.
[0078] The short text classification method provided in this invention increases the amount of representational data carried by the text vector by determining the vector corresponding to any language as the translation vector, thereby increasing the accuracy of short text classification. Furthermore, by constructing a vector set composed of multiple languages, the amount of representational data carried by the translation vector can be greatly increased, thus enabling more accurate classification of short texts.
[0079] Figure 2 This is a flowchart illustrating the process of determining a preset weight matrix provided in an embodiment of the present invention, as shown below. Figure 2 As shown, before determining the preset dimension vector of the target short text based on the target augmentation vector and the preset weight matrix, the short text classification method also includes:
[0080] S210. Construct sample triples based on the sample short text;
[0081] S220. Input the first enhancement vector and the second enhancement vector of the sample triple into the sample weight matrix to obtain the first preset dimension vector and the second preset dimension vector of the sample output by the sample weight matrix.
[0082] S230. Input the first preset dimension vector of the sample, the second preset dimension vector of the sample, and the sample ternary vector to the first neural network to obtain the binary activation value output by the first neural network.
[0083] S240. Based on the binary activation value, the sample classification label in the sample triplet, and the first preset function, obtain the preset weight matrix;
[0084] The sample triplet is determined based on the first sample augmentation vector, the second sample augmentation vector, and the sample classification label corresponding to the first sample augmentation vector and the second sample augmentation vector; the triple vector is determined based on the first sample augmentation vector, the second sample augmentation vector, and the absolute value of the difference between the first sample augmentation vector and the second sample augmentation vector.
[0085] In S210, the sample short text is split according to the splitting method in the first embodiment to obtain sample text units, and the sample augmentation vectors corresponding to the sample text units are further obtained. A sample triplet is determined based on every two sample augmentation vectors and the sample classification labels corresponding to these two sample augmentation vectors.
[0086] For example, after splitting the sample short text word by word, four words are obtained. Based on these four words, four machine-translated translations are obtained. The vectors of these four words and the vectors of their respective four translations are concatenated to obtain four augmentation vectors. Two of these augmentation vectors and the sample classification labels corresponding to these two words are used to construct the first sample triplet. The remaining two augmentation vectors and the sample classification labels corresponding to the remaining two words are used to construct the second sample triplet.
[0087] It is understandable that constructing sample triples based on two sample augmentation vectors and their corresponding sample classification labels is beneficial for calculating the sample weight matrix based on the sample classification labels during subsequent training, thereby ensuring the accuracy of the preset weight matrix obtained during training.
[0088] The following example illustrates the process of S220 by training the sample weights using one of the sample triplet inputs. In S220, the first and second sample enhancement vectors of S220 are input into the sample weight matrix, which can be, for example, a (m+n)*k sample weight matrix. The result is the output of the sample weight matrix, which contains the first preset dimension vector (for example, X, where the dimension of X is k) and the second preset dimension vector (for example, Y, where the dimension of Y is k).
[0089] In S230, based on the vectors X and Y output in S220 and the absolute value of the difference between vectors X and Y (for example, it can be denoted as |XY|), X, Y and |XY| are concatenated to obtain a sample ternary vector (for example, it can be represented as (X, Y, |XY|)). (X, Y, |XY|) is then input into the first neural network to obtain binary activation values.
[0090] It should be noted that the first neural network can be a feedforward neural network.
[0091] Understandably, a sample triplet includes two distinct sample augmentation vectors: a first augmentation vector and a second augmentation vector. It also includes the absolute value of the difference between the first and second augmentation vectors. Constructing the sample triplet using the absolute value of the difference between the first and second augmentation vectors allows the difference between them to be substituted into the training of the sample weight matrix, thereby improving the fit of the pre-set weight matrix after training.
[0092] In S240, the difference between the binary activation values output in S230 and the sample classification labels in the sample triplet in S210 is calculated using the first preset function, and the parameters in the sample weight matrix are adjusted based on the calculation results. It can be understood that through training with a large number of samples and repeatedly adjusting the parameters in the sample weight matrix, the preset weight matrix is finally obtained.
[0093] It should be noted that the first preset function can be a loss function, a cost function, etc., such as the cross-entropy loss function, the perceptual loss function, etc., and this invention does not limit it.
[0094] Figure 3 This is a schematic diagram of one of the model structures of the sample weight matrix provided in an embodiment of the present invention, as shown below. Figure 3As shown, the model structure of the training sample weight matrix in this embodiment may include a first layer as an input layer, which may contain m+n neurons; a second layer as a hidden layer, which may contain k neurons; the output of the hidden layer may be connected to a feedforward neural network; the output of the feedforward neural network is a binary classification output, such as the aforementioned binary classification activation value; and finally, through the training of the above model structure, the preset weight matrix is determined.
[0095] The short text classification method provided in this invention can effectively improve the fitting degree of the sample weight matrix by constructing sample triples and sample triple vectors to train the sample weight matrix, thereby improving the efficiency of subsequent text classification.
[0096] Figure 4 This is a schematic diagram of the process for constructing sample triples provided in an embodiment of the present invention, as shown below. Figure 3 As shown, sample triples are constructed based on the sample short text, including:
[0097] S410. Segment the sample short texts according to the corpus classification database to obtain sample text units;
[0098] S420. Determine the sample classification label for each sample text unit group. The sample text unit group is a combination of the first sample text unit and the second sample text unit.
[0099] S430. For any sample text unit group, determine the translation of the first sample text unit and the translation of the second sample text unit in the sample text unit group.
[0100] S440. Input the first text unit of the sample and its translation to the pre-trained model, and obtain the vector of the first text unit of the sample and the translation vector of the first text unit of the sample output by the pre-trained model; input the second text unit of the sample and its translation to the pre-trained model, and obtain the vector of the second text unit of the sample and the translation vector of the second text unit of the sample output by the pre-trained model.
[0101] S450. The first augmented vector of the sample is obtained by concatenating the vector of the first text unit of the sample and the translation vector of the first text unit of the sample; the second augmented vector of the sample is obtained by concatenating the vector of the second text unit and the translation vector of the second text unit; and a sample triple is constructed based on the first augmented vector of the sample, the second augmented vector of the sample, and the sample classification label.
[0102] In S410, sample short texts in the existing corpus classification database are segmented according to a certain granularity (e.g., segmented word by word, sentence by sentence, or paragraph by paragraph) to obtain sample text units. For example, each word obtained by segmenting word by word is a sample unit, or each sentence obtained by segmenting sentence by sentence is a sample text unit.
[0103] In S420, the sample text group includes two different sample text units, namely the first text unit and the second text unit. Based on the classification corpus, the sample classification labels of the first text unit and the second text unit are determined. For example, if both the first text unit and the second text unit come from sample short texts with the classification label "finance", then the sample classification labels of the first text unit and the second text unit are the same type of text.
[0104] In S430, for the first text unit and the second text unit within each sample text group, machine translation is used to determine the translation of the first text unit corresponding to the first text unit and the translation of the second text unit corresponding to the second text unit.
[0105] In S440, the sample first text unit and sample second text unit determined in S420, and the sample first text unit translation and sample second text unit translation determined in S430 are input into the pre-trained model (e.g., BERT model) to obtain the sample first text unit vector (e.g., can be denoted as E_s1, its dimension can be m), the sample second text unit vector (e.g., can be denoted as E_s2, its dimension can be m), the sample first text unit translation vector (e.g., can be denoted as E_t1, its dimension can be n), and the sample second text unit translation vector (e.g., can be denoted as E_t2, its dimension can be n).
[0106] In S450, the sample first text unit vector E_s1 obtained in S440 is concatenated with the sample first text unit translation vector E_t1 to obtain the sample first enhancement vector, which can be denoted as E_s1t1. Similarly, the sample second text unit vector E_s2 is concatenated with the sample second text unit translation vector E_t2 to obtain the sample second enhancement vector, which can be denoted as E_s2t2. E_s1t1, E_s2t2, and the sample classification label determined in S420 are combined into a sample triplet.
[0107] The short text classification method provided in this invention determines sample classification labels through an existing classification corpus, constructs sample triples including sample translation vectors, and trains the sample matrix weights, thereby increasing the accuracy of subsequent classification based on the enhanced vectors.
[0108] In one embodiment, determining the sample classification label for each sample text unit group includes:
[0109] If the first text unit and the second text unit of the sample are determined to be of the same type, the sample text unit group is determined to have the same label.
[0110] If the first text unit and the second text unit of the sample are determined to be different types of text, then the sample text unit groups are determined to be different labels.
[0111] For example, if it is determined that the first text unit and the second text unit of the sample are of the same type of text, the sample text unit group is determined to have the same label, and the label can be represented by "1";
[0112] If the first text unit and the second text unit of the sample are determined to be different types of text, then the sample text unit groups are determined to be different labels, and the labels can be represented by "0".
[0113] It should be noted that the label is used to identify whether the first text unit of the sample and the second text unit of the sample are of the same type of text. It can be identified by "1" and "0", or by other label content such as "same" and "different".
[0114] The short text classification method provided in this invention uses "1" and "0" to simply identify the classification labels of the first and second text units of the sample, thereby improving the classification efficiency of subsequent text unit groups and sample triplet groups, simplifying the construction process of training samples, and ensuring the classification accuracy of training samples.
[0115] Figure 5 This is a schematic diagram of the process for training a preset classification model provided in an embodiment of the present invention. Figure 5 As shown, before determining the preset dimension vector of the target short text based on the target enhancement vector and the preset weight matrix, the process also includes:
[0116] S510. Determine the preset number of sample classification results corresponding to the preset dimension of the sample based on the preset dimension vector of the sample.
[0117] S520. Input the preset dimension vector of the sample into the sample classification model to obtain a preset number of classification activation values;
[0118] S530. Based on a preset number of classification activation values, a preset number of sample classification results, and a second preset function, process the sample classification model to obtain a preset classification model.
[0119] In S510, it should be noted that the number of dimensions in the preset dimension vector of the samples corresponds to the number of sample classification results. For example, if the preset dimension vector of the samples is k, then there are k classification results for the samples. When training the sample classification model, based on the preset dimension vector k, k sample classification results corresponding to the preset dimension vector are determined. Each set of k-dimensional preset dimension vectors and k sample classification results is used as a sample for training the sample classification model.
[0120] In S520, a preset dimension vector of the input sample is fed into the sample classification model to obtain a preset number of classification activation values. For example, inputting a preset dimension vector of k dimensions of the sample will result in k classification activation values input to the sample classification model.
[0121] In S530, the second preset classification function can be a loss function, cost function, etc., such as cross-entropy loss function, perceptual loss function, etc., which are not limited here. The second preset function is used to calculate the deviation between a preset number of classification activation values and a preset number of sample classification results. The sample classification model is adjusted according to the deviation values, and the preset classification model is obtained through training with a large number of samples.
[0122] Figure 6 This is a schematic diagram of one of the model structures of the preset classification model provided in the embodiments of the present invention. The following is in conjunction with... Figure 6 This describes the preset classification model in this embodiment.
[0123] Figure 6 The sample classification model is a fully connected feedforward neural network. During training, a preset number of classification activation values are obtained by inputting a preset dimension vector of the sample, and the model is adjusted by combining it with a second preset function. When used for short text classification, the preset dimension vector of the short text to be classified is input into the preset classification model, and k classification results are obtained from the output of the preset classification model to complete the classification.
[0124] The short text classification method provided in this invention can obtain a preset number of classification results corresponding to the preset dimension vector by inputting a preset dimension vector into a preset classification model. This can quickly determine the classification result of the short text through the preset dimension vector, thereby improving the classification efficiency of short text.
[0125] The short text classification device provided by the present invention is described below. The short text classification device described below can be referred to in correspondence with the short text classification method described above.
[0126] Figure 7 This is a schematic diagram of the short text classification device provided in an embodiment of the present invention. Figure 7 As shown, the short text classification device includes:
[0127] The text determination unit 710 is used to determine text units and corresponding translation texts based on the target short text.
[0128] Vector determination unit 720 is used to determine the preset dimension vector of the target short text based on the target enhancement vector of the target short text and the preset weight matrix.
[0129] The weight vector determination unit 730 is used to determine the preset dimension vector of the target short text based on the target enhancement vector of the target short text and the preset weight matrix.
[0130] The text classification unit 740 is used to input a preset dimension vector of the target short text into a preset classification model to obtain a preset number of classification results corresponding to the preset dimension.
[0131] The target augmentation vector is obtained by concatenating the text unit vector and the translated text vector; the preset weight matrix is obtained by training based on the sample triples; and the preset classification model is obtained by training based on the preset dimension vectors of all samples and the sample classification result corresponding to the preset dimension vector of each sample.
[0132] The short text classification device provided by this invention can effectively increase the number of representations of short texts and improve the classification accuracy of short texts by splitting short texts into text units, determining the translated text of each text unit, merging the text representations in the text units and the translated text, and generating a preset dimension vector of the enhanced representation of the text units. Furthermore, by constructing a preset classification model, the classification result of short texts can be quickly determined through the preset dimension vector, thereby improving the classification efficiency of short texts.
[0133] Figure 8 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 8 As shown, the electronic device may include: a processor 810, a communications interface 820, a memory 830, and a communication bus 840, wherein the processor 810, the communications interface 820, and the memory 830 communicate with each other via the communication bus 840. The processor 810 can call logical instructions in the memory 830 to execute a short text classification method, which includes:
[0134] Based on the target short text, determine the text units and the corresponding translation texts for each text unit;
[0135] Input text units and translated text into the pre-trained model to obtain the text unit vector and translated text vector output by the pre-trained model;
[0136] Input the target enhancement vector into the preset weight matrix to obtain the preset dimension vector of the target short text output by the preset weight matrix;
[0137] Input the preset dimension vector of the target short text into the preset classification model to obtain a preset number of classification results corresponding to the preset dimension;
[0138] Among them, the target augmentation vector is obtained by concatenating the text unit vector and the translated text vector; the preset weight matrix is obtained by training based on the sample triples; and the preset classification model is obtained by training based on the preset dimension vectors of all samples and the sample classification result corresponding to the preset dimension vector of each sample.
[0139] Furthermore, the logical instructions in the aforementioned memory 830 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0140] On the other hand, the present invention also provides a computer program product, the computer program product comprising a computer program that can be stored on a non-transitory computer-readable storage medium, wherein when the computer program is executed by a processor, the computer is able to execute the short text classification method provided by the above methods, the method comprising:
[0141] Based on the target short text, determine the text units and the corresponding translation texts for each text unit;
[0142] Input text units and translated text into the pre-trained model to obtain the text unit vector and translated text vector output by the pre-trained model;
[0143] Input the target enhancement vector into the preset weight matrix to obtain the preset dimension vector of the target short text output by the preset weight matrix;
[0144] Input the preset dimension vector of the target short text into the preset classification model to obtain a preset number of classification results corresponding to the preset dimension;
[0145] Among them, the target augmentation vector is obtained by concatenating the text unit vector and the translated text vector; the preset weight matrix is obtained by training based on the sample triples; and the preset classification model is obtained by training based on the preset dimension vectors of all samples and the sample classification result corresponding to the preset dimension vector of each sample.
[0146] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the short text classification method provided by the methods described above, the method comprising:
[0147] Based on the target short text, determine the text units and the corresponding translation texts for each text unit;
[0148] Input text units and translated text into the pre-trained model to obtain the text unit vector and translated text vector output by the pre-trained model;
[0149] Input the target enhancement vector into the preset weight matrix to obtain the preset dimension vector of the target short text output by the preset weight matrix;
[0150] Input the preset dimension vector of the target short text into the preset classification model to obtain a preset number of classification results corresponding to the preset dimension;
[0151] Among them, the target augmentation vector is obtained by concatenating the text unit vector and the translated text vector; the preset weight matrix is obtained by training based on the sample triples; and the preset classification model is obtained by training based on the preset dimension vectors of all samples and the sample classification result corresponding to the preset dimension vector of each sample.
[0152] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.
[0153] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.
[0154] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A short text classification method, characterized in that, include: Based on the target short text, determine the text units and the corresponding translated texts for the text units; Input the text unit and the translated text into the pre-trained model to obtain the text unit vector and the translated text vector output by the pre-trained model; Input the target enhancement vector into a preset weight matrix to obtain the preset dimension vector of the target short text output by the preset weight matrix; Input the preset dimension vector of the target short text into the preset classification model to obtain a preset number of classification results corresponding to the preset dimension; Wherein, the target enhancement vector is obtained by concatenating the text unit vector and the translated text vector; the preset weight matrix is obtained by training based on the sample triples; the preset classification model is obtained by training based on the preset dimension vectors of all samples and the sample classification result to which each preset dimension vector belongs; Before inputting the target enhancement vector into a preset weight matrix to obtain the preset dimension vector of the target short text output by the preset weight matrix, the method further includes: Construct sample triples based on the sample short text; Input the first enhancement vector and the second enhancement vector of the sample triple into the sample weight matrix to obtain the first preset dimension vector and the second preset dimension vector of the sample output by the sample weight matrix; Input the first preset dimension vector of the sample, the second preset dimension vector of the sample, and the sample ternary vector into the first neural network to obtain the binary activation value output by the first neural network; The preset weight matrix is obtained based on the binary activation value, the sample classification label in the sample triplet, and the first preset function; The sample triplet is determined based on the first sample augmentation vector, the second sample augmentation vector, and the sample classification label between the first sample augmentation vector and the second sample augmentation vector; the sample triplet vector is determined based on the first sample augmentation vector, the second sample augmentation vector, and the absolute value of the difference between the first sample augmentation vector and the second sample augmentation vector.
2. The short text classification method according to claim 1, characterized in that, Before determining the preset dimension vector of the target short text based on the target enhancement vector of the target short text and the preset weight matrix, the method further includes: Based on the sample preset dimension vector, determine a preset number of sample classification results corresponding to the sample preset dimension; Input the sample's preset dimension vector into the sample classification model to obtain a preset number of classification activation values; Based on the preset number of classification activation values, the preset number of sample classification results, and the second preset function, the sample classification model is processed to obtain the preset classification model.
3. The short text classification method according to claim 1, characterized in that, The construction of sample triples based on sample short texts includes: The sample text units are obtained by segmenting the sample short texts according to the corpus classification database; Determine the sample classification label for each sample text unit group, wherein the sample text unit group is a combination of the first sample text unit and the second sample text unit; For any sample text unit group, determine the translation of the first sample text unit and the translation of the second sample text unit in the sample text unit group; Input the first sample text unit and its translation into the pre-trained model to obtain the vector of the first sample text unit and the translation vector of the first sample text unit output by the pre-trained model; input the second sample text unit and its translation into the pre-trained model to obtain the vector of the second sample text unit and the translation vector of the second sample text unit output by the pre-trained model. The first enhanced vector of the sample is obtained by concatenating the vector of the first text unit of the sample and the translation vector of the first text unit of the sample; the second enhanced vector of the sample is obtained by concatenating the vector of the second text unit of the sample and the translation vector of the second text unit of the sample; the sample triple is constructed based on the first enhanced vector of the sample, the second enhanced vector of the sample, and the sample classification label.
4. The short text classification method according to claim 3, characterized in that, Determining the sample classification label for each sample text unit group includes: If it is determined that the first text unit of the sample and the second text unit of the sample are of the same type of text, then the group of sample text units is determined to be the same tag; If it is determined that the first text unit of the sample and the second text unit of the sample are different types of text, then the sample text unit group is determined to be a different label.
5. The short text classification method according to claim 3, characterized in that, The translated vector can be any of the following vectors: The vector corresponding to any language; Alternatively, a vector set composed of multiple languages.
6. A short text classification device, characterized in that, include: A text determination unit is used to determine text units and corresponding translation texts based on the target short text. A vector determination unit is used to input the text unit and the translated text into a pre-trained model to obtain the text unit vector and the translated text vector output by the pre-trained model. A weight vector determination unit is used to input a target enhancement vector into a preset weight matrix to obtain a preset dimension vector of the target short text output by the preset weight matrix. A text classification unit is used to input a preset dimension vector of the target short text into a preset classification model to obtain a preset number of classification results corresponding to the preset dimension; Wherein, the target enhancement vector is obtained by concatenating the text unit vector and the translated text vector; the preset weight matrix is obtained by training based on the sample triples; the preset classification model is obtained by training based on the preset dimension vectors of all samples and the sample classification result to which each preset dimension vector belongs; The device further includes: a training module, the training module being used for: Construct sample triples based on the sample short text; Input the first enhancement vector and the second enhancement vector of the sample triple into the sample weight matrix to obtain the first preset dimension vector and the second preset dimension vector of the sample output by the sample weight matrix; Input the first preset dimension vector of the sample, the second preset dimension vector of the sample, and the sample ternary vector into the first neural network to obtain the binary activation value output by the first neural network; The preset weight matrix is obtained based on the binary activation value, the sample classification label in the sample triplet, and the first preset function; The sample triplet is determined based on the first sample augmentation vector, the second sample augmentation vector, and the sample classification label between the first sample augmentation vector and the second sample augmentation vector; the sample triplet vector is determined based on the first sample augmentation vector, the second sample augmentation vector, and the absolute value of the difference between the first sample augmentation vector and the second sample augmentation vector.
7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the short text classification method as described in any one of claims 1 to 5.
8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the short text classification method as described in any one of claims 1 to 5.
9. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the short text classification method as described in any one of claims 1 to 5.
Citation Information
Patent Citations
Short text classification method based on semantic enhancement
CN108280206A
Long text classification method and system, electronic equipment and computer readable storage medium
CN113553848A