Chinese text classification method and system based on structured multi-granularity feature fusion

Abstract

The invention discloses a Chinese text classification method and system based on structured multi-granularity feature fusion, and relates to the technical field of natural language process.The method comprises the steps that firstly, a pre-trained BERT encoder is used for generating character and word feature vectors according to a Chinese text, and feature vector representation of sentence granularity is obtained through maximum pooling operation; secondly, capturing structured context semantic features by constructing an initial fully-connected graph of granularity feature vectors of characters, words and sentences, and introducing a graph convolutional network to capture global and local correlation among nodes of the fully-connected graph to obtain structured multi-level semantic features; then, a structured multi-granularity interaction strategy is designed, weighting processing is carried out on different granularity features, and fusion of structured multi-granularity features is achieved; and finally, classifying the fused features by adopting a multi-layer full-connection neural network. According to the method, the understanding ability of the model for Chinese text complex semantics can be improved, and the Chinese text classification precision and efficiency are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and more specifically to a Chinese text classification method and system based on structured multi-granularity feature fusion. Background Art

[0002] Text classification is one of the important tasks in natural language processing (NLP), which aims to automatically assign text to predetermined categories based on its semantic content. It is widely used in many fields such as information retrieval, public opinion monitoring, sentiment analysis, news classification, spam filtering, etc. In the early research of text classification, common methods mainly rely on machine learning models based on statistical features, such as Naive Bayes, Support Vector Machine (SVM) and decision trees. These methods rely on manually extracted text features, such as TF-IDF and Bag of Words. Although they have achieved certain results in some simple tasks, these methods ignore the contextual information in the text and the semantic relationship between words, thus limiting the classification performance.

[0003] With the rapid development of deep learning, neural network models, especially text classification methods based on deep learning, have gradually become mainstream. Neural network architectures such as long short-term memory networks (LSTM) and convolutional neural networks (CNN) are widely used in text classification tasks and can capture contextual semantic features in text. In recent years, text classification methods based on pre-trained models, such as BERT and GPT, have also made significant progress. Through pre-training, such models can learn rich contextual semantic information from massive unlabeled texts and greatly improve the accuracy in classification tasks.

[0004] However, although these methods perform well in English text classification tasks, Chinese text classification still faces unique challenges. The characteristics of Chinese text, such as word segmentation without spaces, complex semantic structure, polysemous words, homophones, etc., bring difficulties to feature extraction. Limited by the training methods of pre-trained models, most models tend to rely on single-granularity features for text feature extraction, such as features based on word, phrase or sentence level. This single-dimensional feature extraction ignores the hierarchical structure and multi-level semantic expression in the text, and it is difficult to fully capture the complex semantic information of Chinese text. At the same time, the multi-granular semantic features in the text, such as characters, words, phrases and sentences, are often not effectively integrated, resulting in the model's inability to fully utilize multi-level structured information. This lack of feature integration limits the classification model's capture of contextual information and fine-grained semantics, making the classification results often perform poorly in a variety of practical scenarios. Especially in Chinese, due to the relatively complex syntactic structure and the abundance of synonyms and polysemous words, features of different granularities may carry key semantic information, and a single-granularity processing method cannot fully reflect the true meaning of the text. Moreover, in actual Chinese text classification tasks, existing methods often lack effective distinction and dynamic adjustment of the importance of different features, resulting in unstable classification effects when the model faces texts of different types or lengths.

[0005] Therefore, how to improve the accuracy of text classification through multi-granularity feature fusion is an urgent problem that technical personnel in this field need to solve. Summary of the invention

[0006] In view of this, the present invention provides a Chinese text classification method and system based on structured multi-granularity feature fusion. Through structured multi-granularity feature fusion technology and the introduction of graph convolutional networks, the model's ability to understand the complex semantics of Chinese text is improved, thereby achieving more efficient text classification.

[0007] In order to achieve the above object, the present invention adopts the following technical solution:

[0008] A Chinese text classification method based on structured multi-granularity feature fusion includes the following steps:

[0009] Step 1: Collect Chinese text and preprocess it to obtain the initial multi-granularity feature vector;

[0010] Step 2: Construct an initial multi-granularity fully connected graph based on the initial multi-granularity feature vector, and use the graph convolutional network to extract structured multi-level semantic features;

[0011] Step 3: Use a multi-granularity interaction mechanism to fuse structured multi-level semantic features to obtain multi-granularity fusion features;

[0012] Step 4: Input the multi-granularity fusion features into the multi-layer fully connected neural network to obtain the classification results.

[0013] Preferably, the initial multi-granularity feature vector includes feature vectors of three granularities: an initial character feature vector, an initial word feature vector, and an initial sentence feature vector.

[0014] Preferably, in step 1, a plurality of Chinese texts are collected to construct a Chinese text set and the Chinese text set is preprocessed, and the specific steps include:

[0015] Segment each Chinese text in the Chinese text collection into a number of word sequences of the same length and without overlapping;

[0016] Input each word sequence into the pre-trained BERT encoder to obtain the initial word feature vector and initial character feature vector of the Chinese text;

[0017] Divide each Chinese text into sentences and obtain sentence masks;

[0018] Perform the maximum pooling operation in the column direction according to the initial character feature vector and the sentence mask to obtain the reduced dimension feature vector, and then perform linear projection on the reduced dimension feature vector to obtain the initial sentence feature vector;

[0019] The initial character feature vector, the initial word feature vector and the initial sentence feature vector constitute the initial multi-granularity feature vector.

[0020] Preferably, the initial character feature vector, the initial word feature vector and the initial sentence feature vector are used to construct the initial character fully connected graph, the initial word fully connected graph and the initial sentence fully connected graph respectively, to obtain the initial multi-granularity fully connected graph; the graph convolutional network is used to extract the structured character semantic features, the structured word semantic features and the structured sentence semantic features from the initial character fully connected graph, the initial word fully connected graph and the initial sentence fully connected graph respectively, to form a structured multi-level semantic feature.

[0021] Preferably, the specific implementation process of step 3 is:

[0022] Step 31: Using a multi-granularity interaction mechanism to perform weighted fusion on the structured character semantic features, the structured word semantic features, and the structured sentence semantic features to obtain an initial fusion feature;

[0023] Step 32: Integrate the structured sentence semantic features into the corresponding structured character semantic features and structured word semantic features to obtain integrated features; transmit and interact the structured features based on the graph convolutional network, prioritize and emphasize the semantic information of different granularities in the Chinese text, so as to more effectively integrate these features and realize the feature interaction between the sentence granularity and the word and character granularity;

[0024] Step 33: Perform the column-wise maximum pooling operation on the integrated features to obtain multi-granularity fusion features, extract information from graph convolutional networks of different granularities, and further fuse these multi-granularity features to generate a global semantic representation.

[0025] Preferably, the output layer of the multi-layer fully connected neural network uses a Softmax function to predict the labels of the multi-granularity fusion features to obtain a classification result; and the multi-layer fully connected neural network is trained using a cross entropy loss function to obtain a classification model.

[0026] Preferably, the Chinese text is segmented into a number of equal-length and non-overlapping word sequences, and each word sequence is segmented into a number of characters; during segmentation, if the length of the last word sequence is smaller than that of the other word sequences, they are padded to the same length.

[0027] A Chinese text classification system based on structured multi-granularity feature fusion includes a preprocessing layer, a feature extraction layer, a feature fusion layer and a classification output layer; wherein,

[0028] Preprocessing layer, preprocessing the Chinese text to obtain the initial multi-granularity feature vector;

[0029] Feature extraction layer: constructs an initial multi-granularity fully connected graph based on the initial multi-granularity feature vector, and uses a graph convolutional network to extract structured multi-level semantic features;

[0030] The feature fusion layer uses a multi-granularity interaction mechanism to fuse structured multi-level semantic features to obtain multi-granularity fusion features;

[0031] The classification output layer inputs the multi-granularity fusion features into a multi-layer fully connected neural network to obtain the classification results.

[0032] Preferably, the preprocessing layer includes a pretrained BERT encoder and a linear projection layer. The pretrained BERT encoder encodes to obtain an initial character feature vector and an initial word feature vector. The linear projection layer obtains an initial sentence feature vector based on the initial character feature vector and the initial word feature vector to form an initial multi-granularity feature vector.

[0033] Preferably, the feature extraction layer includes a fully connected layer and a graph convolutional network layer; the fully connected layer constructs an initial character fully connected graph, an initial word fully connected graph, and an initial sentence fully connected graph according to the initial character feature vector, the initial word feature vector, and the initial sentence feature vector, respectively; the graph convolutional network layer includes several graph convolutional networks, which extract structured character semantic features, structured word semantic features, and structured sentence semantic features from the initial character fully connected graph, the initial word fully connected graph, and the initial sentence fully connected graph, respectively, to form structured multi-level semantic features.

[0034] Preferably, the feature fusion layer includes a transmission fusion module and a feature fusion module; the transmission fusion module integrates the structured sentence semantic features into the corresponding structured character semantic features and structured word semantic features to obtain integrated features; the feature fusion module performs a column-wise maximum pooling operation on the integrated features to obtain multi-granularity fusion features.

[0035] Preferably, the output layer of the multi-layer fully connected neural network uses the Softmax function to predict the labels of the multi-granularity fusion features to obtain the classification results; the classification model uses the cross entropy loss function for optimization training.

[0036] It can be seen from the above technical solutions that, compared with the prior art, the present invention discloses a Chinese text classification method and system based on structured multi-granularity feature fusion. First, starting from constructing the embedding vectors of characters, words and sentences, the pre-trained BERT encoder is used to generate character and word feature vectors, and the feature vector representation of sentence granularity is obtained by performing the maximum pooling operation on the word feature vectors in each sentence, so as to comprehensively obtain the semantic information in Chinese text from three different levels of characters, words and sentences, ensure the diversity and completeness of the features, and accurately capture the contextual semantic information in the text; secondly, by constructing the initial fully connected graph of the character, word and sentence granularity feature vectors, the semantic information in the character, word and sentence vectors is captured. The structured contextual semantic features contained in it, and by introducing a graph convolutional network, adaptively capture the global and local correlations between fully connected graph nodes, combine the shallow vector features and deep vector features of Chinese text, enhance the feature discrimination ability, fully capture the complex semantic information of Chinese text, and obtain structured multi-level semantic features; then, design a structured multi-granularity interaction strategy, weightedly process features of different granularities, give priority to important semantic information, enhance the information interaction of each graph convolutional network, and realize the fusion of structured multi-granularity features; finally, a multi-layer fully connected neural network is used to classify the fused features, thereby enhancing the model's ability to understand the complex semantics of Chinese text and improving classification accuracy and efficiency.

[0037] The present invention has the following specific beneficial effects:

[0038] (1) Improve classification accuracy: Through the extraction and fusion of structured multi-granular features, it is possible to fully capture the complex semantic information in Chinese text, especially the subtle semantic differences in the context, thereby significantly improving the accuracy of text classification, which is particularly suitable for a variety of practical application scenarios.

[0039] (2) Enhanced ability to understand complex texts: The introduction of graph convolutional networks enables the model to prioritize and emphasize important semantic information in the text, thereby effectively addressing the challenges of polysemous words, synonyms, and complex syntactic structures in Chinese texts and improving the model’s ability to understand complex semantics.

[0040] (3) Improved generalization performance: By combining multi-level structured features, the model can adapt to texts of different types and lengths, has strong generalization performance, and is suitable for a variety of tasks such as information retrieval, sentiment analysis, and news classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0042] Figure 1 A schematic diagram of the flow of the Chinese text classification method based on structured multi-granularity feature fusion provided by the present invention;

[0043] Figure 2 A schematic diagram of the structure of the Chinese text classification system based on structured multi-granularity feature fusion provided by the present invention. DETAILED DESCRIPTION

[0044] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0045] The embodiment of the present invention discloses a Chinese text classification method based on structured multi-granularity feature fusion, collecting a Chinese text set S, where S = {s0, s1, ..., s n} is the set of all texts to be classified, each text s i It contains features of multiple granularities, such as character-level, word-level, phrase-level, and sentence-level semantic information. i , its structured multi-granularity feature set F i It can be expressed as F i ={f i，0 , f i，1 …, f i，m}, where f i，j Represents text features of different granularities; the present invention performs feature fusion on these Chinese text features of different granularities to construct a unified feature representation function Where d<<|S|, so that each text s i Through its multi-granularity feature F iObtaining a unified vector representation in a low-dimensional space, this multi-granularity feature fusion method can capture the complex semantic information in Chinese text, thereby improving the accuracy of the classification model. Figure 1 As shown, the specific steps include:

[0046] S1: Collect Chinese text and preprocess it to obtain the initial multi-granularity feature vector;

[0047] S2: Construct an initial multi-granularity fully connected graph based on the initial multi-granularity feature vector, and use a graph convolutional network to extract structured multi-level semantic features;

[0048] S3: Use multi-granularity interaction mechanism to fuse structured multi-level semantic features to obtain multi-granularity fusion features;

[0049] S4: A multi-layer fully connected neural network is used to build a classification model, and multi-granularity fusion features are input into the classification model to obtain the classification results.

[0050] Furthermore, in S1, each Chinese text in the Chinese text set S is preprocessed. Since Chinese text can be decomposed into a series of words, which can be further decomposed into character sequences, and sentences are expressed through words, the initial character feature vector, the initial word feature vector and the initial sentence feature vector are obtained respectively, which together constitute the initial multi-granularity feature vector. In order to obtain a structured and hierarchical Chinese text representation, the preprocessing process is specifically as follows:

[0051] S11: When calculating the initial character feature vector and the initial word feature vector, the text s i Split into l non-overlapping word sequences, denoted as {w1, w2, ..., w l},w i ={c i0 , c i1 , …, c ir} is the i-th word sequence, containing r+1 characters, c ir represents the rth character in the i-th word sequence, where c i0 = [CLS] is a special starting vector marker for the BERT encoder; the length of each word sequence needs to be kept consistent during segmentation, so some word sequences are padded; each word sequence is input into the pre-trained BERT encoder g(·, ψ), and the output feature vector of the i-th word sequence can be calculated as follows:

[0052] [p i , x i1 …x ir ]=g(w i ,ψ),i=1,…,l

[0053] Where ψ represents the parameters of the pre-trained BERT encoder; the final hidden state p labeled by the [CLS] vector i As the initial word feature vector, the final hidden state x of other vector labels ij , j = 1, ..., r as the initial character feature vector, the initial word feature vector of the Chinese text can be obtained And the corresponding initial character feature vector represents a set of real numbers, and d represents the dimension of the feature;

[0054] S12: Divide the Chinese text into n sentences, each sentence contains l word sequences, then the sentence mask V of each sentence is mask It is expressed as:

[0055]

[0056] Among them, v1+v2+…+v n =lr, l represents the number of word sequences, r represents the number of characters in the word sequence; the same numbers in the sentence mask represent that the word sequences at these positions belong to the same sentence, and the size of the numbers represents the position information of the word sequence in the sentence;

[0057] According to the sentence mask, linear projection is performed on each sentence to obtain the initial sentence feature vector S of sentence granularity, which is expressed as:

[0058]

[0059] in, represents the reduced-dimensional feature vector of the k-th sentence, k = 1, 2, ..., n, maxpooling_c(·) represents the maximum pooling operation in the column direction; vk represents the k-th sentence, k = 1, 2, ..., n, and is the trainable parameter of the linear projection; 1 lr is a vector of size lr with all elements set to 1.

[0060] Furthermore, when extracting features in S2, a graph convolutional network (GCN) is used in combination with an initial multi-granularity feature vector to obtain the structured multi-level semantic features of Chinese text. By capturing information of different granularities of the text, such as characters, words, and sentences, the global and local structural features of the text are retained; specifically:

[0061] S21: construct an initial character fully connected graph, an initial word fully connected graph, and an initial sentence fully connected graph using the initial character feature vector, the initial word feature vector, and the initial sentence feature vector respectively; the fully connected graph G p It is expressed as:

[0062]

[0063] Where V represents the initial character feature vector, initial word feature vector or initial sentence feature vector that has been calculated, which can be P, X or S; E and A p Represent the fully connected graph G p The edge set and adjacency matrix of ; and They represent weight matrices respectively; d represents the dimension of the feature;

[0064] S22: For feature extraction at the character, word, and sentence granularity, multiple graph convolutional networks are used to simultaneously capture the correlations within and between different granularities of Chinese text, obtain structured character semantic features, structured word semantic features, and structured sentence semantic features, and form structured multi-level semantic features;

[0065] The feature extraction of each granularity is implemented using an N-layer graph convolutional network, and the output V of the graph convolutional network at the m-th layer of character, word, and sentence granularity is calculated. m+1 , obtain the m-th layer structured character semantic feature X m+1 , semantic features of structured words P m+1 and structured sentence semantic features S m+1 , forming structured multi-level semantic features, expressed as:

[0066]

[0067] in, Represents the adjacency matrix of the fully connected graph corresponding to the feature vector in the m-th layer of the graph convolutional network; is a diagonal node degree matrix whose elements The sum of the elements of all nodes b adjacent to node a in the adjacency matrix is ​​the degree of node a; V represents the trainable projection matrix of the m-th layer graph convolutional network; m represents the output of the graph convolutional network of a specific granularity at the m-1th layer; σ(·) is a nonlinear activation function; m=1,2,…,N.

[0068] Furthermore, feature fusion in S3 uses a multi-granularity interaction mechanism to weightedly fuse structured features from different granularities, emphasizing key semantic information and enhancing the interactivity between granularities. Since the important feature information contained in Chinese text is unevenly distributed, and different characters and words contribute differently to text classification, S3 prioritizes and emphasizes important semantic information in Chinese text based on feature transmission and interaction between graph convolutional networks, and effectively fuses and calculates semantic features of Chinese text at different granularities. Specifically:

[0069] S31: Use a multi-granularity interaction mechanism for transmission fusion to achieve feature interaction between macro sentence granularity and micro word and character granularity graph convolution, and integrate sentence features into corresponding word and character features;

[0070] If the character node in the initial character connectivity graph or the word node in the initial word connectivity graph belongs to a sentence, it is necessary to copy the structured sentence semantic features corresponding to the sentence and concatenate them with the structured character semantic features corresponding to the character node or the structured word semantic features corresponding to the word node. The calculation process is as follows:

[0071]

[0072] in, and Represents a known transfer mask generated based on the structure or characteristics of the input data. and represents the learning parameter of transmission fusion, concat(·) represents vector concatenation; Refers to the word-granular features used for feature transfer interaction, Refers to the character-level features used for feature transfer interaction; S m+1 represents the structured sentence semantic features output by the m-th layer graph convolutional network in S22; P m+1 and X m+1 They respectively refer to the structured word semantic features and structured character semantic features output by the m-th layer graph convolutional network after feature concatenation;

[0073] S32: In order to obtain more information from features of different granularities, the maximum pooling operation in the column direction is performed on the structured sentence semantic features corresponding to each layer of the graph convolutional network, as well as the structured word semantic features and structured character semantic features after feature splicing. The calculation process is as follows:

[0074]

[0075]

[0076]

[0077] Among them, m = 1, 2, ..., N, N is the number of layers of the graph convolution network of each granularity in the feature extraction process; multi-granularity features are obtained through the maximum pooling operation in the column direction, and sentence features Word features and character features

[0078] S33: The multi-granularity features obtained by fusion calculation are obtained to obtain the multi-granularity fusion feature u. The specific calculation process is as follows:

[0079]

[0080] Here, [·,·] represents the concatenation operation along the column dimension.

[0081] Furthermore, the classification in S4 is implemented through a multi-layer fully connected neural network. The multi-granular fusion features after fusing the feature vectors of characters, words and sentences calculated by the graph convolutional network are mapped to a specific category label space, and the softmax function is used to output the probability of each label to complete the text classification task.

[0082] Furthermore, the softmax function is used to output the probability of each label, and the cross entropy loss function L is used cls To train the model, adjust the parameters in the linear projection, graph convolutional neural network, multi-granularity interaction mechanism, and Softmax function. The calculation process is as follows:

[0083] y p =softmaxx(W p u+b p )

[0084]

[0085] Among them, y D is the Chinese text index set with labels, F is the dimension of the multi-granularity fusion feature output in S3, which is equal to the number of categories; W p represents the weight matrix; b p represents bias; y p represents probability; Y is the label indicator matrix, Y ft Represents the element at row f and column t in the label indicator matrix.

[0086] On the other hand, in a specific embodiment, a Chinese text classification system based on structured multi-granularity feature fusion, such as Figure 2 As shown, it includes a preprocessing layer, a feature extraction layer, a feature fusion layer and a classification output layer; wherein the preprocessing layer includes a pre-trained BERT encoder and a linear projection layer; the feature extraction layer includes a fully connected layer and a graph convolutional network layer, and the graph convolutional network layer includes several graph convolutional networks; the feature fusion layer includes a transmission fusion module and a feature fusion module; the classification output layer adopts a multi-layer fully connected neural network, and its output layer adopts a Softmax function to output the classification results.

[0087] As shown, the cross entropy loss function is used to optimize the training of the linear projection layer, graph convolution network, transmission fusion module and multi-layer warm link neural network. The present invention considers introducing the structured features of different granularities after fusion to realize the classification of Chinese texts, taking the Chinese text set to be classified as input and multiple category labels as output.

[0088] Furthermore, the preprocessing layer: preprocesses the Chinese text set to generate initial multi-granularity feature vectors of three granularities: characters, words, and sentences, which respectively represent text information of different granularities; the pre-trained BERT encoder encodes to obtain the initial character feature vector and the initial word feature vector, and the linear projection layer obtains the initial sentence feature vector based on the initial character feature vector and the initial word feature vector to form the initial multi-granularity feature vector.

[0089] Furthermore, the feature extraction layer: after constructing the initial fully connected graph using the initial multi-granularity feature vector, the graph convolutional network is applied to each granularity, and the correlation within and between these granularities is captured through the graph convolutional network (GCN), retaining the global and local structured features of the text, generating structured output features of a specific granularity, and further improving the ability to capture text semantics; the fully connected layer constructs the initial character fully connected graph, the initial word fully connected graph, and the initial sentence fully connected graph according to the initial character feature vector, the initial word feature vector, and the initial sentence feature vector respectively; the graph convolutional network layer includes several graph convolutional networks, which extract structured character semantic features, structured word semantic features, and structured sentence semantic features from the initial character fully connected graph, the initial word fully connected graph, and the initial sentence fully connected graph, respectively, to form structured multi-level semantic features.

[0090] Furthermore, the feature fusion layer: performs weighted fusion of structured features from character granularity, word granularity and sentence granularity, and interacts with structured features between graph convolutional networks, prioritizes and emphasizes semantic information of different granularities in Chinese text, so as to more effectively fuse these features. In order to realize the feature interaction between sentence granularity and word and character granularity, a transmission fusion module is introduced to integrate sentence features into corresponding word and character features. In addition, through the maximum pooling operation in the column direction, the multi-granularity features extracted from graph convolutional networks of different granularities in the feature extraction layer are further fused to generate a global semantic representation, which is used as part of the input of the classification output layer. Specifically, the transmission fusion module integrates the structured sentence semantic features into the corresponding structured character semantic features and structured word semantic features to obtain integrated features. The feature fusion module performs the maximum pooling operation in the column direction on the integrated features to obtain multi-granularity fusion features.

[0091] Furthermore, classification output layer: the fused multi-granularity feature vectors (characters, words and sentences) are input into the classification output layer and processed by a multi-layer fully connected neural network. The output layer uses the Softmax function to predict the probability of each label and determine the classification result. The parameters of the linear projection layer, graph convolutional network, transmission fusion module and multi-layer fully connected neural network are adjusted through the cross entropy loss function to optimize the system and ensure the accuracy and effectiveness of the text classification task.

[0092] On the other hand, by introducing dynamic routing or capsule networks to better capture the complex relationship between features of different granularities and achieve feature fusion, the accuracy and stability of classification can be further improved.

[0093] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0094] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A Chinese text classification method based on structured multi-granularity feature fusion, characterized in that: The following steps are involved: Step 1: Collect Chinese text and preprocess it to obtain the initial multi-granularity feature vector; Step 2: Construct an initial multi-granularity fully connected graph based on the initial multi-granularity feature vector, and use the graph convolutional network to extract structured multi-level semantic features; Step 3: Use a multi-granularity interaction mechanism to fuse structured multi-level semantic features to obtain multi-granularity fusion features; Step 4: Input the multi-granularity fusion features into the multi-layer fully connected neural network to obtain the classification results.

2. The Chinese text classification method based on structured multi-granularity feature fusion according to claim 1 is characterized in that: The initial multi-granularity feature vector includes an initial character feature vector, an initial word feature vector and an initial sentence feature vector.

3. The Chinese text classification method based on structured multi-granularity feature fusion according to claim 1 is characterized in that: In step 1, a number of Chinese texts are collected to construct a Chinese text set and the Chinese text set is preprocessed. The specific steps include: Each Chinese text in the Chinese text set is divided into several word sequences of the same length and non-overlapping, and each word sequence is input into the pre-trained BERT encoder to obtain the initial word feature vector and initial character feature vector of the Chinese text; Each Chinese text is divided into sentences to obtain sentence masks. The maximum pooling operation in the column direction is performed according to the initial character feature vector and the sentence mask to obtain a reduced dimension feature vector. Then, the reduced dimension feature vector is linearly projected to obtain the initial sentence feature vector. The initial character feature vector, the initial word feature vector and the initial sentence feature vector constitute the initial multi-granularity feature vector.

4. The Chinese text classification method based on structured multi-granularity feature fusion according to claim 2 is characterized in that: The initial character feature vector, the initial word feature vector and the initial sentence feature vector are used to construct the initial character fully connected graph, the initial word fully connected graph and the initial sentence fully connected graph respectively, and the initial multi-granularity fully connected graph is obtained; the graph convolutional network is used to extract the structured character semantic features, the structured word semantic features and the structured sentence semantic features from the initial character fully connected graph, the initial word fully connected graph and the initial sentence fully connected graph respectively to form a structured multi-level semantic feature.

5. The Chinese text classification method based on structured multi-granularity feature fusion according to claim 4 is characterized in that: The specific implementation process of step 3 is as follows: Step 31: Using a multi-granularity interaction mechanism to perform weighted fusion on the structured character semantic features, the structured word semantic features, and the structured sentence semantic features to obtain an initial fusion feature; Step 32: Integrate the structured sentence semantic features into the corresponding structured character semantic features and structured word semantic features to obtain integrated features; Step 33: Perform the maximum pooling operation in the column direction on the integrated features to obtain multi-granularity fusion features.

6. The Chinese text classification method based on structured multi-granularity feature fusion according to claim 1 is characterized in that: The output layer of the multi-layer fully connected neural network uses the Softmax function to predict the labels of the multi-granularity fusion features to obtain the classification results; the cross entropy loss function is used to adjust the preprocessing, graph convolutional network, multi-granularity interaction mechanism and multi-layer fully connected neural network according to the classification results.

7. The Chinese text classification method based on structured multi-granularity feature fusion according to claim 3 is characterized in that: The Chinese text is divided into several equal-length and non-overlapping word sequences, and each word sequence is divided into several characters; when dividing, if the length of the last word sequence is smaller than that of other word sequences, it is padded to the same length.

8. A Chinese text classification system based on structured multi-granularity feature fusion, characterized in that: The Chinese text classification method based on structured multi-granularity feature fusion according to any one of claims 1 to 7 comprises a preprocessing layer, a feature extraction layer, a feature fusion layer and a classification output layer; wherein: Preprocessing layer, preprocessing the Chinese text to obtain the initial multi-granularity feature vector; Feature extraction layer: constructs an initial multi-granularity fully connected graph based on the initial multi-granularity feature vector, and uses a graph convolutional network to extract structured multi-level semantic features; The feature fusion layer uses a multi-granularity interaction mechanism to fuse structured multi-level semantic features to obtain multi-granularity fusion features; The classification output layer inputs the multi-granularity fusion features into a multi-layer fully connected neural network to obtain the classification results.

Images