A method and system for Chinese short text classification based on word similarity

Abstract

The application discloses a Chinese short text classification method and system based on word similarity, which can be applied to judging text event type and classifying text. The method comprises the following steps: using a preset event type set to perform word segmentation on the text to obtain a vector form of an event trigger word and a belonging event type label. Text feature extraction is performed on the segmented text to obtain a text semantic vector. A natural language processing model is used on the obtained event trigger word and the preset event type set to vectorize to obtain respective word vectors, and the word vectors are used to calculate a similarity vector between the event trigger word and the event type set. The text semantic vector and the similarity vector are fused to obtain total features, which are input into a text classifier to classify the text, and a classification result is output and accuracy is evaluated. The application introduces the concept of text semantic feature and word similarity fusion, fully excavates the features of the text, and thus improves the accuracy and efficiency of the text classification.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of natural language processing, and more particularly relates to a Chinese short text classification method and system based on word similarity. BACKGROUND

[0002] At present, artificial intelligence has achieved great results in many fields, such as natural language processing, image processing, data mining, etc. Among them, text classification is an important research direction. Text classification is to realize information organization and extraction by dividing data into pre-defined categories.

[0003] Current text classification technology can quickly determine the category of an unknown document, making mass information processing more efficient, saving a lot of information processing cost, and quickly determining the text type to filter out the required information.

[0004] The prior art patent with publication number CN116127079A proposes a text classification method, which determines the text category to which the text segment to be classified belongs by calculating the semantic similarity between the text feature vector of the text segment to be classified and the support text feature vector of each text category. This scheme has the problem that the feature mining of the text is not sufficient. SUMMARY

[0005] The present application provides a Chinese short text classification method and system based on word similarity to overcome the problem of insufficient feature extraction in the prior art text classification method, which relies too much on the quality of data labeling.

[0006] The present application provides a Chinese short text classification method based on word similarity, comprising the following steps:

[0007] S1: using a preset event type set, performing word segmentation on the text to be classified, cleaning the data, extracting the label, obtaining the preset vector form of the event trigger word and the event type label to which the text belongs;

[0008] S2: performing text feature extraction on the word segmented text obtained in S1 to obtain a text semantic vector;

[0009] S3: using a pre-trained natural language processing model to vectorize the event trigger word obtained in S1 and the preset event type set respectively to obtain respective word vectors, and using the word vectors to calculate the similarity vector between the event trigger word and the event type set;

[0010] S4: fusing the text semantic vector obtained in S2 and the similarity vector obtained in S3 to obtain the total features for final classification;

[0011] S5: input the total features into a text classifier, classify the text to be classified, output a classification result, and evaluate the classification accuracy.

[0012] Further, the specific method of text feature extraction in step S2 is as follows:

[0013] S2.1: encode the segmented text obtained in step S1 using an encoder to obtain representation vectors of the text, i.e., word vectors, segment vectors, and position vectors;

[0014] S2.2: input the word vectors, segment vectors, and position vectors into a pre-trained model BERT for text semantic learning to obtain text features, including a text semantic vector of the entire sentence and a semantic vector corresponding to each word.

[0015] Further, in step S3, the natural language processing model adopts a Word2Vec model.

[0016] Further, in step S3, the vector similarity between the trigger word and the event type set is calculated using a cosine similarity algorithm.

[0017] Further, in step S4, the text semantic vector obtained in S2 and the similarity vector obtained in S3 are fused, specifically including:

[0018] S4.1: convert the text semantic vector and the similarity vector to the same dimension;

[0019] S4.2: calculate the total weight for feature fusion according to the vectors after dimension conversion;

[0020] S4.3: calculate the total features after fusion according to the vectors after dimension conversion and the total weight.

[0021] Further, the specific steps of step S5 are as follows:

[0022] S5.1: input the total features obtained in step S4 into a text classifier;

[0023] S5.2: classify the text to be classified using the text classifier to obtain a predicted label;

[0024] S5.3: encode the predicted label using a label encoder to obtain a preset vector of the predicted label;

[0025] S5.4: perform classification evaluation using the preset vector obtained from the event type label of the text and the preset vector of the predicted label to obtain the accuracy.

[0026] The second aspect of the application provides a Chinese short text classification system based on word similarity, which is used for realizing the Chinese short text classification method, and characterized by comprising:

[0027] A preprocessing module is configured to segment the text to be classified and extract labels, convert the labels into a preset vector form, and extract a text semantic vector.

[0028] A similarity module is configured to calculate a word similarity vector of trigger words and predefined event types.

[0029] A feature fusion module is configured to fuse the text semantic vector and the similarity vector to obtain total features.

[0030] A text classification and evaluation module is configured to input the total features into a text classifier to perform text classification and evaluate the classification accuracy.

[0031] Further, the preprocessing module comprises:

[0032] A text preprocessing submodule is configured to preprocess the text and segment the text into word strings.

[0033] A label extraction and conversion submodule is configured to extract event type labels and trigger words, and convert the event type labels into a preset vector.

[0034] A text semantic vector submodule is configured to obtain a semantic vector of the text to be classified.

[0035] Further, the similarity calculation module comprises:

[0036] A word vector submodule is configured to obtain word vectors of trigger words and event types.

[0037] A similarity calculation submodule is configured to calculate the similarity between words and construct a similarity vector.

[0038] Further, the feature fusion module comprises:

[0039] A dimension reduction submodule is configured to reduce the features to the same dimension.

[0040] A weight calculation submodule is configured to calculate total weights for feature fusion according to the text semantic vector and the similarity vector.

[0041] A fusion submodule is configured to fuse the features according to the total weights obtained by the weight calculation submodule to obtain total features.

[0042] Further, the text classification and evaluation module comprises:

[0043] A text classifier construction submodule is configured to input the fused total features into a text classifier.

[0044] The classification submodule classifies the text to be classified based on the text classifier constructed by the text classifier construction submodule;

[0045] The evaluation submodule is used for evaluating the classification accuracy.

[0046] Compared with the prior art, the technical scheme of the present application has the following beneficial effects:

[0047] The introduced word similarity feature is beneficial to compensate the probability value predicted by the pre-training model on the classification output for different event types, and for the event type with high similarity to the trigger word, the probability value predicted as the event type can be improved through the compensation of the similarity vector, thereby improving the accuracy of text classification. BRIEF DESCRIPTION OF DRAWINGS

[0048] In order to make the purpose, technical scheme of the present application more clear, the present application provides the following drawings and makes the following description:

[0049] Figure 1 The implementation flowchart of the text classification method of the present application;

[0050] Figure 2 The schematic diagram of the preprocessing module of the text classification system of the present application;

[0051] Figure 3 The schematic diagram of the similarity module of the text classification system of the present application;

[0052] Figure 4 The schematic diagram of the feature fusion module of the text classification system of the present application;

[0053] Figure 5 The schematic diagram of the text classification and evaluation module of the text classification system of the present application; DETAILED DESCRIPTION

[0054] The present application will be further described below in combination with the drawings and examples.

[0055] The present application provides a Chinese short text classification method and system based on word similarity, Figure 1 The implementation flowchart of the text classification method of the present application is shown, and the details are as follows:

[0056] S1: using a preset event type set, performing word segmentation on the text to be classified, cleaning data, extracting labels, obtaining one-hot form of event trigger words and event type labels to which the text belongs.

[0057] In one specific embodiment, the text to be classified is processed by Chinese word segmentation, and the tags are extracted and converted into one-hot vector tags. For example, the text to be classified is "Mobile phones have been reduced in price, 32-megapixel cameras only cost 1,000 yuan, and the cost performance is getting better and better". After word segmentation, we can get {mobile phone, already, reduced price, pixel, 1,000 yuan, cost performance, getting better and better}. The event type tag carried by this text is {finance / trading-reduction}, and the trigger word of this text is {reduction}. For example, if the predefined event types are {sale, financing, reduction, listing, franchising}, then the event type tags can be converted into one-hot vectors, such as finance / trading-reduction, which can be converted into [0, 0, 1, 0, 0].

[0058] S2: Extract text features from the segmented text obtained in S1. First, encode the text using an encoder to obtain a vector representation of the text. Then, input the vector representation into the pre-trained model BERT for text semantic learning to obtain text features ω∈R. bs×h Where b represents the number of samples, s represents the length of the sample sequence, and h represents the dimension of the hidden layer in the BERT model. The text feature takes the following form:

[0059] Where F i Let i represent the text features of the i-th sample.

[0060] F i The form is:

[0061] 768 represents the hidden layer dimension of the BERT model. hCLS represents the text semantic vector of the entire sentence, indicating sentence-level features, while hti represents the semantic vector corresponding to each word in the sentence, indicating word-level features.

[0062] S3: For the event trigger words and the preset event type set obtained in S1, use the pre-trained Word2Vec model to vectorize them respectively to obtain their respective word vectors. Calculate the similarity vector between the event trigger words and the event type set using these word vectors. The vector similarity calculation is based on the cosine similarity algorithm, where the formula for calculating cosine similarity is:

[0063]

[0064] Equation (5) can be used to obtain the similarity between the trigger word and the event type.

[0065] S4: Based on the text semantic vector and similarity vector obtained in steps S2 and S3, the two features are fused. First, the two feature vectors are transformed to the same dimension, and then the two feature vectors are fused to obtain the final total feature used for classification.

[0066] In a specific embodiment, the text semantic vector is a text feature F i The first row vector is a 768-length vector in the form of hCLS=[h1CLS, h2CLS, …, hiCLS, …, h768CLS], where hiCLS represents the output of the i-th dimension of the last hidden layer of the pre-trained model. The similarity vector is a 65-length vector in the form of:

[0067] s=[s1,s2,…,si,…,s65]

[0068] Where si represents the similarity between the trigger word and the event type ei. The text semantic vector needs to be reduced in dimension to the same dimension as the similarity vector to perform feature fusion.

[0069] The feature dimension reduction method used is as follows:

[0070]

[0071] The fusion used is based on the following formula:

[0072]

[0073] f G =Z G ·f′ text +(1-Z G )·f′ similar

[0074] Where sigmoid() represents the sigmoid activation function, W G , U G is the dimension reduction matrix, f′ text is the dimension-reduced text feature vector, is the learnable similarity vector weight, where f′ similari is the i-th similarity vector. Z G is the total weight calculated based on the contribution of the two features, which is a number between 0 and 1, f G ∈R b×n is the fused total feature.

[0075] S5: Input the total feature into the text classifier, classify the text to be classified, output the classification result, and evaluate the classification accuracy.

[0076] Figures 2-5 The module structure of the text classification system of the present application is shown, which includes:

[0077] The preprocessing module is configured to preprocess the training text and extract features, and the features are semantic vectors of the text. The preprocessing module comprises: a text preprocessing submodule configured to preprocess the text and divide the text into word strings; a label extraction and conversion submodule configured to extract event type labels and trigger words, and convert the event type labels into one-hot vectors; and a text semantic vector submodule configured to obtain semantic vectors of the text to be classified.

[0078] The similarity module is configured to obtain word vectors and calculate word similarity. The similarity calculation module comprises: a word vector submodule configured to obtain word vectors of the trigger words and the event types; and a similarity calculation submodule configured to calculate the similarity between the words and construct a similarity vector.

[0079] The feature fusion module is configured to fuse the text semantic vectors and the similarity vectors.

[0080] The feature fusion module comprises: a dimension reduction submodule configured to reduce the features to the same dimension; a weight calculation submodule configured to calculate total weights for feature fusion according to the text semantic vectors and the similarity features; and a fusion submodule configured to fuse the features according to the total weights obtained by the weight calculation submodule.

[0081] The text classification and evaluation module is configured to construct a text classifier and classify the text to be classified. The text classification and evaluation module comprises the following submodules: a text classifier construction submodule configured to construct a text classifier according to actual task requirements, and in a specific embodiment, the number of categories for classification is 65, and the text classifier is designed accordingly; a classification submodule configured to classify the text to be classified based on the text classifier constructed by the text classifier construction submodule; and an evaluation submodule configured to evaluate the classification accuracy of the network, and in a specific embodiment, the predicted results of the classifier are first converted into one-hot vectors, and then the converted predicted results are compared with the true labels of the samples, if the predicted labels are the same as the true labels, it is recorded as a correct prediction, otherwise it is recorded as a prediction error, and by predicting all test samples, the final accuracy is obtained. For example, the prediction result of the ith test sample is P i =[0, 0, 1, 0, 0], and the true label thereof is T i =[0, 1, 0, 0, 0], the prediction result is different from the true label, and it is recorded as a prediction error. Finally, the prediction accuracy of the classifier on the test set is wherein n represents the number of samples, (P i =T i ) represents whether the predicted result is the same as the true label, and if the predicted result is the same as the true label (P i =T i ), it is 1, otherwise it is 0.

[0082] Obviously, the above embodiments of the present application are merely exemplary but not intended to limit the embodiments of the present application. Based on the above description, any other variations or changes can be made by those skilled in the art without departing from the spirit and principles of the present application. It is not necessary to list all the embodiments here. Any modifications, equivalent replacements, and improvements made within the spirit and principles of the present application shall fall within the scope of the claims of the present application.

Claims

1. A Chinese short text classification method based on word similarity, characterized in that, Includes the following steps: S1: Using a preset set of event types, segment the text to be classified, clean the data, extract tags, and obtain preset vector forms of event trigger words and event type tags to which the text belongs; S2: Extract text features from the segmented text obtained in S1 to obtain the text semantic vector; S3: For the event trigger words and the preset event type set obtained in S1, use a pre-trained natural language processing model to vectorize them respectively, obtaining their respective word vectors. Calculate the similarity vector between the event trigger words and the event type set using these word vectors. The similarity vector is a vector of length 65, in the form of: s = [s1, s2, ..., si, ..., s65] Where si represents the similarity between the trigger word and the event type ei, calculated using the cosine similarity algorithm, as shown in the following expression: S4: Fuse the text semantic vector obtained in S2 with the similarity vector obtained in S3 to obtain the final total feature for classification. The text semantic vector needs to be reduced to the same dimension as the similarity vector before feature fusion can be performed. The feature dimensionality reduction method used is as follows: The fusion method used is based on the following formula: f G =Z G ·f′ text +(1-Z G )·f′ similar Where sigmoid() represents the sigmoid activation function, W G U G It is a dimension reduction matrix, f′ text It is the text feature vector after dimensionality reduction. These are learnable similarity vector weights, where f′ similari Z is the i-th similarity vector. G The total weight, f, is calculated based on the contribution of the two features and is a number between 0 and 1. G ∈R b×n It is the overall characteristic after fusion; S5: Input the total features into the text classifier, classify the text to be classified, output the classification results, and evaluate the classification accuracy.

2. The Chinese short text classification method based on word similarity according to claim 1, characterized in that, The specific method for text feature extraction in step S2 is as follows: S2.1: Encode the segmented text obtained in step S1 using an encoder to obtain the text representation vectors, which are word vectors, segment vectors, and position vectors. S2.2: Input word vectors, segment vectors, and position vectors into the pre-trained model BERT to perform text semantic learning and obtain text features. The text features include the text semantic vector of the entire sentence and the semantic vector corresponding to each word.

3. The Chinese short text classification method based on word similarity according to claim 1, characterized in that, In step S3, the natural language processing model adopts the Word2Vec model.

4. The Chinese short text classification method based on word similarity according to claim 1, characterized in that, Step S4 involves fusing the text semantic vector obtained in S2 and the similarity vector obtained in S3, specifically including: S4.1: Convert the text semantic vector and similarity vector to the same dimension; S4.2: Calculate the total weights used for feature fusion based on the vectors after dimensionality transformation; S4.3: Calculate the fused total features based on the vector after dimension transformation and the total weight.

5. The Chinese short text classification method based on word similarity according to claim 1, characterized in that, The specific steps of step S5 are as follows: S5.1: Input the total features obtained in step S4 into the text classifier; S5.2: Use the text classifier to classify the text to be classified and obtain the predicted label; S5.3: Obtain the preset vector of the predicted label by encoding the predicted label through the label encoder; S5.4: Use the preset vector obtained by the event type label of the text and the preset vector of the predicted label to perform classification evaluation and obtain the accuracy.

6. A Chinese short text classification system based on word similarity, used to implement the Chinese short text classification method based on word similarity as described in any one of claims 1-5, characterized in that, include: The preprocessing module is used to segment the text to be classified into words, extract the tags, convert the tags into a preset vector form, and extract the semantic vector of the text. The similarity module is used to calculate the word similarity vector between the trigger word and the predefined event type; The feature fusion module is used to fuse the text semantic vector and the similarity vector to obtain the total feature; The text classification and evaluation module is used to input the total features into the text classifier to classify the text and evaluate the classification accuracy.

7. A Chinese short text classification system based on word similarity according to claim 6, characterized in that, The preprocessing module includes: The text preprocessing submodule is used to preprocess text, segmenting it into word strings; The tag extraction and conversion submodule is used to extract event type tags and trigger words, and convert the event type tags into preset vectors; The text semantic vector submodule is used to obtain the semantic vector of the text to be classified.

8. A Chinese short text classification system based on word similarity according to claim 6, characterized in that, The similarity module includes: The word vector submodule is used to obtain word vectors for trigger words and event types; The similarity calculation submodule is used to calculate the similarity between words and construct a similarity vector.

9. A Chinese short text classification system based on word similarity according to claim 6, characterized in that, The feature fusion module includes: The dimensionality reduction submodule is used to reduce the dimensionality of features to the same dimension. The weight calculation submodule is used to calculate the total weights for feature fusion based on the text semantic vector and similarity vector; The fusion submodule performs feature fusion based on the total weights obtained from the weight calculation submodule to obtain the total features.

Images