A complex text classification method based on GRU-Glove algorithm

Abstract

The application discloses a text classification method based on a GRU-Glove algorithm, and is used for solving the problem of low text classification task execution efficiency caused by more new words and untimely training corpus updating. First, in the word segmentation stage, the GRU network is used for initial word segmentation of the text, and the Glove model is used for supplementary correction of the word segmentation result, so that the word segmentation result is obtained; second, in the classification stage, the GRU network is used as a classifier algorithm to classify the text. In the word segmentation stage, the GRU network is used for initial word segmentation of the text, and the Glove model is used for supplementary correction of the word segmentation result, so that the word segmentation result is obtained, the final word segmentation result can effectively identify new network words, the problem of inaccurate word segmentation existing in the prior art is solved, and therefore the accuracy of word segmentation of complex text with new words and the classification task accuracy can be improved.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and specifically to a complex text classification method based on the GRU-Glove algorithm. Background Technology

[0002] Text classification is one of the main application methods of Natural Language Processing (NLP). In recent years, with the development of big data technology and NLP technology, various text classification methods have been widely used in the field of network security and information technology. Text classification methods have achieved remarkable results in Internet intelligence mining, online public opinion analysis, and sensitive information retrieval, effectively improving the law enforcement efficiency of public security forces.

[0003] Conventional text classification methods comprise four tasks: text preprocessing, feature extraction, text representation, and category segmentation. Text preprocessing is a crucial task in text classification, primarily aimed at removing unnecessary stop words and performing Chinese word segmentation and part-of-speech tagging to provide accurate corpora for subsequent tasks. Chinese word segmentation is a key stage in text preprocessing, its main function being to divide a given Chinese string into linguistically meaningful units according to a specified algorithm. Conventional Chinese word segmentation methods mainly include dictionary-based classification methods, statistical segmentation methods, and word formation-based segmentation methods. These methods are widely used in Chinese word segmentation; however, they primarily rely on dictionary rules, statistical features, and local semantic features for segmentation, neglecting the semantic relationships between words in long text sequences. This results in mediocre performance on complex texts. Designing high-quality word segmentation methods remains a worthy area of ​​further research. Among the key issues requiring resolution is the low efficiency of text classification tasks caused by a large number of new words and untimely updates to the training corpus. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention aims to provide a complex text classification method based on the GRU-Glove algorithm.

[0005] To achieve the above objectives, the present invention adopts the following technical solution:

[0006] A complex text classification method based on the GRU-Glove algorithm, the specific process of which is as follows:

[0007] (1) Remove stop words from the original text data;

[0008] (2) Use word2vec technology to vectorize the characters of the text data processed in step (1) to obtain text vectors;

[0009] (3) Input the text vector obtained in step (2) into the pre-trained GRU word segmentation network for initial word segmentation;

[0010] (4) The initial word segmentation results output by the GRU network in step (3) are supplemented and corrected using the Glove model to obtain the final word segmentation results;

[0011] The specific process of supplementing and correcting the initial word segmentation results using the GloVe model is as follows:

[0012] For text X in the preliminary word segmentation results i and X j Establish the co-occurrence matrix Z = {Z 1,2 Z 1,3 ,…,Z i,j}, where Z i,j For text X i and X j The number of times text X appears in the same word sequence window i and X j co-occurrence function F(X) i ,X j )for:

[0013]

[0014] Where Ω t Let f(X) be the set of words in the text at time t. i,j Let ) be the weight function, defined as:

[0015]

[0016] Z max The maximum word frequency threshold is set to prevent excessively high word frequencies from affecting the frequency of normally occurring words. In this embodiment, the number of words in the current word segmentation result is taken as Z. max The value; finally, the criteria for determining the combination of new words are given:

[0017]

[0018] Where J(X) i ,X j () indicates the new word determination state, 1 represents X i and X j Combination X i X j This is a new word; 0 represents X. i and X j Combination X i X j For words that do not form a word, δ is the threshold value.

[0019] Repeat the above process until all J(X) are obtained. i ,X j If the value is 0, the final word segmentation result Ys is obtained;

[0020] (5) Vectorize the words in the final word segmentation result obtained in step (4) using word2vec to obtain word vectors;

[0021] (6) Input the word vectors obtained in step (5) into the pre-trained classification GRU network for classification to obtain the final text classification result.

[0022] Furthermore, the training process of the word segmentation GRU network is as follows:

[0023] Input the text vector sample into the word segmentation GRU network and calculate the reset gate r. t and update gate z t Value:

[0024] r t =σ(U r X t +W r S t-1 )

[0025] z t =σ(U z X t +W z S t-1 )

[0026] Where X t Let U be the text vector sample input at time t. r and U z W represents the weights input to the reset gate and update gate, respectively. z and W r These are the weights of the update gate and reset gate remembered from the previous time step, respectively, where σ represents the sigmoid function, and S... t-1 The hidden layer output is given at time t-1; subsequently, the candidate hidden layer state values ​​are calculated.

[0027]

[0028] Among them, U h W represents the weights input to the activation function tanh. h To integrate the information from the previous time step into the tanh weight values; the hidden layer output S at time t. t for:

[0029]

[0030] Thus, the output Y of the output layert for

[0031] Y t =σ(V·S t )

[0032] V represents the weight values ​​from the hidden layer input to the final output value;

[0033] Using the BP algorithm to analyze parameter W r W z U r U z W h U h Adjust V and repeat the above iterative process until training time I is reached. t1 .

[0034] Furthermore, the training process of the classification GRU network is as follows:

[0035] Input the word vector sample XE into the GRU and calculate the reset gate state r. t ':

[0036] r t '=σ'(U r 'XE t +W r 'S t-1 ')

[0037] Calculate and update the gate state:

[0038] z t '=σ(U z 'XE t +W z 'S t-1 ')

[0039] Calculate candidate hidden layer states:

[0040]

[0041] Output layer output values:

[0042] Y t '=σ'(V'·S t ')

[0043] Calculate the sample error:

[0044]

[0045] Where I t2 For training time; E t For single-sample error:

[0046]

[0047] whereXE t Let U be the word vector sample input at time t. r 'and U z 'W represents the weight values ​​input to the reset gate and update gate, respectively. z 'and W r ' represents the weight values ​​of the update gate and reset gate memorized from the previous time step, σ' represents the sigmoid function, S t-1 ' is the hidden layer output at time t-1; U h 'W represents the weights input to the activation function tanh. h 'The weight value for integrating information from the previous time step into tanh; S' t ' represents the hidden layer output at time t; V' represents the weights from the hidden layer input to the final output value; Y' represents the actual output, Y t ' is the network output at time t;

[0048] Using the BP algorithm to analyze parameter W r '、W z '、U r '、U z '、W h '、U h Correct ' and V'; repeat the above iterative process until training ends to obtain a trained classification GRU network.

[0049] Furthermore, the trained word segmentation GRU network, the GloVe model, and the trained classification GRU network are combined into a text classification model, GRU-GloVe-GRU, to classify the text X to be classified. D The input is fed into the text classification model GRU-Glove-GRU, and the classification result Y is automatically obtained. D And obtain the values ​​of each parameter in the model.

[0050] The present invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method.

[0051] The present invention also provides a computer device, including a processor and a memory, wherein the memory is used to store a computer program; and the processor is used to execute the computer program to implement the above-described method.

[0052] The beneficial effects of this invention are as follows: In the word segmentation stage, the method of this invention uses the GRU algorithm to perform initial word segmentation on the text, and uses the GloVe model to supplement and correct the word segmentation results, thereby obtaining the final word segmentation results. The final word segmentation results can effectively identify new words on the network, thereby solving the problem of inaccurate word segmentation in the prior art, and thus improving the word segmentation accuracy and classification accuracy of complex texts with new vocabulary. Attached Figure Description

[0053] Figure 1 This is a schematic diagram of the method flow according to an embodiment of the present invention. Detailed Implementation

[0054] The present invention will be further described below with reference to the accompanying drawings. It should be noted that this embodiment is based on the present technical solution and provides detailed implementation methods and specific operation processes, but the protection scope of the present invention is not limited to this embodiment.

[0055] This embodiment provides a text classification method based on the GRU-Glove algorithm. First, in the word segmentation stage, the GRU algorithm is used for initial word segmentation, and the Glove model is used to supplement and correct the segmentation results, thereby obtaining the final word segmentation result. Second, in the classification stage, the GRU algorithm is used as the classifier algorithm to classify the text, obtaining the final text classification method. Figure 1 As shown, the specific process of this method is as follows:

[0056] (1) Remove stop words from the text data to be classified;

[0057] (2) Use word2vec technology to vectorize the characters of the text data processed in step (1) to obtain text vectors;

[0058] (3) Input the text vector obtained in step (2) into the pre-trained GRU word segmentation network for initial word segmentation;

[0059] The training process of the word segmentation GRU network is as follows:

[0060] Input the text vector sample into the word segmentation GRU network and calculate the reset gate r. t and update gate z t Value:

[0061] r t =σ(U r X t +W r S t-1 )

[0062] z t =σ(U z Xt +W z S t-1 )

[0063] Where X t Let U be the text vector sample input at time t. r and U z W represents the weights input to the reset gate and update gate, respectively. z and W r These are the weights of the update gate and reset gate remembered from the previous time step, respectively, where σ represents the sigmoid function, and S... t-1 This represents the hidden layer output at time t-1. Next, the candidate hidden layer state values ​​are calculated.

[0064]

[0065] Among them, U h W represents the weights input to the activation function tanh. h The weights are calculated by integrating information from the previous time step into the tanh value. The hidden layer output S at time t... t for:

[0066]

[0067] Thus, the output Y of the output layer t for

[0068] Y t =σ(V·S t )

[0069] V represents the weights from the hidden layer input to the final output value.

[0070] Using the BP algorithm to analyze parameter W r W z U r U z W h U h Adjust V and repeat the above iterative process until training time I is reached. t1 In this embodiment, I t1 The value is 1000.

[0071] (4) The initial word segmentation results output by the GRU network in step (3) are supplemented and corrected using the Glove model to obtain the final word segmentation results;

[0072] Specifically, the process of supplementing and correcting the initial word segmentation results using the GloVe model is as follows:

[0073] For text X in the preliminary word segmentation results i and Xj Establish the co-occurrence matrix Z = {Z 1,2 Z 1,3 ,…,Z i,j}, where Z i,j For text X i and X j The number of times the same word appears in the same word sequence window; in this embodiment, the sequence window is set to 2, and the text X... i and X j co-occurrence function F(X) i ,X j )for:

[0074]

[0075] Where Ω t Let f(X) be the set of words in the text at time t. i,j Let ) be the weight function, defined as:

[0076]

[0077] Z max The maximum word frequency threshold is set to prevent excessively high word frequencies from affecting the frequency of normally occurring words. In this embodiment, the number of words in the current word segmentation result is taken as Z. max The value of . Finally, the criteria for determining the combination of new words are given:

[0078]

[0079] Where J(X) i ,X j () indicates the new word determination state, 1 represents X i and X j Combination X i X j This is a new word; 0 represents X. i and X j Combination X i X j For words that do not form a word, δ is the threshold value.

[0080] Repeat the above process until all J(X) are obtained. i ,X j If the value is 0, the final word segmentation result Ys is obtained;

[0081] (5) Vectorize the words in the final word segmentation result obtained in step (4) using word2vec to obtain word vectors;

[0082] (6) Input the word vectors obtained in step (5) into the pre-trained classification GRU network for classification to obtain the final text classification result.

[0083] The training process of the classification GRU network is as follows:

[0084] Input the word vector sample XE into the GRU and calculate the reset gate state r. t ':

[0085] r t '=σ'(U r 'XE t +W r 'S t-1 ')

[0086] Calculate and update the gate state:

[0087] z t '=σ(U z 'XE t +W z 'S t-1 ')

[0088] Calculate candidate hidden layer states:

[0089]

[0090] Output layer output values:

[0091] Y t '=σ'(V'·S t ')

[0092] Calculate the sample error:

[0093]

[0094] Where I t2 The training time is set to 500; E t For single-sample error:

[0095]

[0096] whereXE t Let U be the word vector sample input at time t. r 'and U z 'W represents the weight values ​​input to the reset gate and update gate, respectively. z 'and W r ' represents the weight values ​​of the update gate and reset gate memorized from the previous time step, σ' represents the sigmoid function, S t-1 ' is the hidden layer output at time t-1; U h 'W represents the weights input to the activation function tanh. h 'The weight value for integrating information from the previous time step into tanh; S' t' represents the hidden layer output at time t; V' represents the weights from the hidden layer input to the final output value; Y' represents the actual output, Y t ' is the network output at time t;

[0097] Using the BP algorithm to analyze parameter W r '、W z '、U r '、U z '、W h '、U h Correct ' and V'; repeat the above iterative process until training ends to obtain a trained classification GRU network.

[0098] It should be noted that the text classification model GRU-Glove-GRU is obtained by combining the trained word segmentation GRU network, the GloVe model, and the trained classification GRU network. This model classifies the text X to be classified. D The input is fed into the text classification model GRU-Glove-GRU, and the classification result Y is automatically obtained. D And obtain the values ​​of each parameter in the model.

[0099] For those skilled in the art, various corresponding changes and modifications can be made based on the above technical solutions and concepts, and all such changes and modifications should be included within the protection scope of the claims of this invention.

Claims

1. A complex text classification method based on the GRU-Glove algorithm, characterized in that, The specific process of the method is as follows: (1) Remove stop words from the original text data; (2) Use word2vec technology to vectorize the characters of the text data processed in step (1) to obtain text vectors; (3) Input the text vector obtained in step (2) into the pre-trained GRU word segmentation network for initial word segmentation; (4) The initial word segmentation results output by the GRU network in step (3) are supplemented and corrected using the Glove model to obtain the final word segmentation results; The specific process of supplementing and correcting the initial word segmentation results using the GloVe model is as follows: For text X in the preliminary word segmentation results i and X j Establish the co-occurrence matrix Z = {Z 1,2 Z 1,3 ,…,Z i,j }, where Z i,j For text X i and X j The number of times text X appears in the same word sequence window i and X j co-occurrence function F(X) i ,X j )for: Where Ω t Let f(Z) be the set of words in the text at time t. i,j Let ) be the weight function, defined as: Z max The maximum word frequency threshold is set to prevent excessively high word frequencies from affecting the frequency of normally occurring words. The number of words in the current word segmentation result is taken as Z. max The value; finally, the criteria for determining the combination of new words are given: Where J(X) i ,X j () indicates the new word determination state, 1 represents X i and X j Combination X i X j This is a new word; 0 represents X. i and X j Combination X i X j For words that do not form a word, δ is the threshold value. Repeat the above process until all J(X) are obtained. i ,X j If the value is 0, the final word segmentation result Ys is obtained; (5) Vectorize the words in the final word segmentation result obtained in step (4) using word2vec to obtain word vectors; (6) Input the word vectors obtained in step (5) into the pre-trained classification GRU network for classification to obtain the final text classification result.

2. The method according to claim 1, characterized in that, The training process of the word segmentation GRU network is as follows: Input the text vector sample into the word segmentation GRU network and calculate the reset gate r. t and update gate z t Value: r t =σ(U r X t +W r S t-1 ) z t =σ(U z X t +W z S t-1 ) Where X t Let U be the text vector sample input at time t. r and U z W represents the weights input to the reset gate and update gate, respectively. z and W r These are the weights of the update gate and reset gate remembered from the previous time step, respectively, where σ represents the sigmoid function, and S... t-1 The hidden layer output is given at time t-1; subsequently, the candidate hidden layer state values ​​are calculated. Among them, U h W represents the weights input to the activation function tanh. h To integrate the information from the previous time step into the tanh weight values; the hidden layer output S at time t. t for: Thus, the output Y of the output layer t for Y t =σ(V·S t V represents the weights from the hidden layer input to the final output value; Using the BP algorithm to analyze parameter W r W z U r U z W h U h Adjust V and repeat the above iterative process until training time I is reached. t1 .

3. The method according to claim 1, characterized in that, The training process of the classification GRU network is as follows: Input the word vector sample XE into the GRU and calculate the reset gate state r. t ': r t '=σ'(U r 'CAR t +W r 'S t-1 ') Calculate and update the gate state: mm t '=σ'(U z 'CAR t +W z 'S t-1 ') Calculate candidate hidden layer states: Output layer output values: Y t '=σ'(V'·S t ') Calculate the sample error: Where I t2 For training time; E t For single-sample error: whereXE t Let U be the word vector sample input at time t. r 'and U z 'W represents the weight values ​​input to the reset gate and update gate, respectively. z 'and W r ' represents the weight values ​​of the update gate and reset gate memorized from the previous time step, σ' represents the sigmoid function, S t-1 ' is the hidden layer output at time t-1; U h 'W represents the weights input to the activation function tanh. h 'The weight value for integrating information from the previous time step into tanh; S' t ' represents the hidden layer output at time t; V' represents the weights from the hidden layer input to the final output value; Y' represents the actual output, Y t ' is the network output at time t; Using the BP algorithm to analyze parameter W r '、W z '、U r '、U z '、W h '、U h Correct ' and V'; repeat the above iterative process until training ends to obtain a trained classification GRU network.

4. The method according to claim 1, characterized in that, The trained segmentation GRU network, the GloVe model, and the trained classification GRU network are combined into a text classification model, GRU-Glove-GRU, to classify the text X to be classified. D The input is fed into the text classification model GRU-Glove-GRU, and the classification result Y is automatically obtained. D And obtain the values ​​of each parameter in the model.

5. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method described in any one of claims 1-4.

6. A computer device, characterized in that, It includes a processor and a memory, the memory being used to store a computer program; the processor being used to execute the computer program to implement the method of any one of claims 1-4.

Images