A cancer text classification method and system based on a multi-view graph convolutional network

Abstract

The application provides a cancer text classification method based on a multi-view graph convolutional network, comprising the following steps: S1, obtaining a text corpus file, a word segmentation dictionary and a stop word table, and generating a cleaned standard corpus; S2, performing word embedding construction and sentence embedding construction; S3, constructing a word-level view by taking a word as a node and constructing a sentence-level view by taking a sentence as a node, and establishing an edge between the nodes; S4, multi-view graph convolution feature learning, performing a graph convolution operation on the word-level view and the sentence-level view respectively, and extracting local and global features; S5, multi-view feature fusion; S6, inputting the fused features into a multi-layer perception (MLP) classification network, outputting a classification probability through a Softmax function, and outputting a classification result file corresponding to the text. The application realizes joint modeling of semantic information at the word level and the sentence level, and can simultaneously capture fine-grained local features and global semantic relationships.

Description

TECHNICAL FIELD

[0001] The application relates to a cancer text classification method based on a multi-view graph convolutional neural network, and belongs to the application of artificial intelligence and medical text analysis. BACKGROUND

[0002] With the rapid advancement of medical informatization, the number of cancer-related clinical texts, case records and research literature is growing exponentially. Such texts are not only large in scale, but also have strong language professional characteristics and complex structures. Medical texts usually contain a large number of field terms, abbreviations and implicit semantic relationships. Traditional text classification methods (such as TF-IDF, CNN, RNN, etc.) are difficult to fully capture the semantics and hierarchical structure in them, resulting in limited classification performance. In order to improve the automatic understanding and classification ability of medical texts, scholars further improve and optimize based on existing language models, and obtain pre-trained language models such as ClinicalBERT and BioBERT. They significantly improve the semantic modeling ability by retraining on medical corpus, and lay a foundation for medical natural language processing.

[0003] However, although the pre-trained model performs excellently in general language tasks, there are still many deficiencies in text classification in the medical field. On the one hand, the input length of the BERT model is limited, which is difficult to process long texts and may not be able to capture all relevant information, resulting in performance degradation. BERT uses a fixed-size vocabulary to construct word embeddings, which limits its ability to handle words outside the predefined vocabulary. This limitation may weaken its effectiveness in representing and understanding rare or specific domain terms. In recent years, graph neural networks (GNN) have been introduced into text classification tasks because they are good at processing data with inherent graph structure, such as TextGCN and ME-GCN. However, these methods still face major challenges in constructing appropriate graph representations for word and sentence-level text classification.

[0004] The state invention patent application 2025115551185, entitled "Chinese medicine text classification method and system based on hybrid graph diffusion model and heterogeneous graph convolutional neural network", constructs a single heterogeneous graph containing word-word, sentence-sentence and word-sentence three edge types, and combines a graph diffusion model for feature extraction. However, this method has significant defects when applied to cancer text classification, specifically:

[0005] 1. Embedding layer defects: GloVe and Word2Vec models are used to generate word embeddings, which have no character-level subword semantic capture ability and cannot effectively represent rare professional terms and domain abbreviations in cancer texts. The sentence vector is generated by averaging the word vectors, which easily causes the loss of semantic information of long sentences and complex sentences.

[0006] 2. Graph modeling defects: a single heterogeneous graph is constructed to fuse word, sentence nodes and cross-granularity edge types, independent feature learning of word-level and sentence-level semantics is not realized, features of different granularity nodes interfere with each other, resulting in fuzzy extraction of word-level local co-occurrence features and sentence-level global logical features in cancer text;

[0007] 3. Convolution layer defects: a unified convolution strategy is used for the heterogeneous graph, there is no independent weight matrix to differentially extract different granularity features, and it cannot adapt to the strong co-occurrence of professional terms and the complex logical association between sentences in cancer text;

[0008] 4. Fusion layer defects: the features extracted from the heterogeneous graph are fused without weight, and the importance of word-level and sentence-level features cannot be adaptively balanced according to the type of cancer text (such as literature, electronic medical record, pathological report);

[0009] 5. Domain adaptation defects: this method is designed for the characteristics of traditional Chinese medicine text, and does not consider the term distribution and text structure characteristics of cancer text, so the classification accuracy and robustness are greatly reduced when applied across domains.

[0010] Therefore, in view of the above defects of the prior art, a cancer text classification method and model based on a multi-view graph convolutional network are urgently needed, which can realize subword-level semantic capture, multi-granularity independent feature learning and adaptive feature fusion of cancer text, and solve many deficiencies of the prior art in cancer text classification. SUMMARY

[0011] The primary technical problem to be solved by the present application is to overcome the deficiencies of existing medical text classification technology in cancer text classification, such as insufficient embedding modeling, unreasonable graph structure design, no differentiated feature extraction, lack of adaptability in feature fusion, and poor domain adaptation, and to provide a cancer text classification method based on a multi-view graph convolutional neural network, which realizes multi-level and multi-granularity precise semantic modeling and classification of cancer text.

[0012] The purpose of the present application is achieved by the following technical solutions.

[0013] The present application provides a cancer text classification method based on a multi-view graph convolutional network, which is used to obtain accurate word-level semantic representation and sentence-level semantic representation in the same framework. The core difference between the present application and the existing heterogeneous graph convolution method is that the present application uses a double-view independent homogeneous graph architecture instead of a single heterogeneous graph, uses subword-aware word embedding + large model sentence embedding instead of GloVe / Word2Vec word embedding + word vector average sentence embedding, uses independent view convolution instead of unified heterogeneous graph convolution, and uses adaptive weighted fusion instead of weightless feature fusion. The core defects of the prior art are solved from the architecture, and the implementation steps include:

[0014] The FastText model is used to obtain the word embedding representation. FastText is an efficient word vector training algorithm, which is characterized by regarding a word as composed of several character n-gram subsequences, thereby capturing the word shape variation and subword-level semantic information in the vector space. In this model, each word w is represented as a set containing all its n-gram subsequences:

[0015]

[0016] wherein, represents the final embedding vector of the word , represents a single character-level n-gram subsequence belonging to the set ; represents the vector representation corresponding to the character n-gram .

[0017] The technical effect of this word embedding method is that it can capture the subword semantics of rare professional terms and abbreviations in cancer text. Even if the vocabulary does not appear in the training corpus, an effective vector representation can be generated through character-level n-grams, solving the problem of GloVe / Word2Vec model modeling failure for unregistered professional terms in the prior art.

[0018] The sentence-level embedding representation is obtained using the model. The model is based on XLM-RoBERTa, which can accurately capture sentence structure and contextual semantic information by expanding the maximum position length and multi-lingual and multi-granularity semantic modeling, thereby obtaining global semantic representation. For any given input sentence , the model processes it and maps it to a fixed-dimensional dense vector :

[0019]

[0020] The technical effect of this sentence embedding method is that it realizes long-distance semantic capture based on large-scale pre-training models, which is suitable for long texts such as lengthy scientific literature and electronic medical records in cancer text, solving the problem of semantic information loss and insufficient long sentence modeling capability caused by word vector average generation of sentence vectors in the prior art.

[0021] The word vector builds a word-level view. Whether there is an edge between nodes is determined by calculating the cosine similarity between any two word vectors. The formula for calculating the cosine similarity is:

[0022]

[0023] wherein , These represent the vector representations of the two words respectively. For vector dot product, Let L be the L2 norm of the corresponding vector. The formulas for calculating the vector dot product and the L2 norm are:

[0024]

[0025]

[0026] Set similarity threshold .when At that time, an undirected edge is established between word nodes X and Y. ;when In this case, no edges are established. A sentence-level view is constructed in the same way, and the existence of edges between sentences is determined based on the cosine similarity between sentence vectors.

[0027] The technical effect of constructing dual independent homogeneous graphs in this step is that word-level and sentence-level semantic modeling are completely decoupled, enabling independent feature learning. This avoids the feature extraction ambiguity problem caused by mutual interference between word-sentence cross-granularity nodes in existing heterogeneous graphs, allowing the model to accurately capture the co-occurrence association of professional terms between words and the logical reasoning relationship between sentences in cancer texts.

[0028] Perform graph convolution on the word-level view. Let the word-level graph be... Its characteristic matrix is Where m is the number of nodes and 100 is the word vector dimension. An adjacency matrix is ​​introduced. and its degree matrix The formula for calculating single-layer graph convolution is as follows:

[0029]

[0030]

[0031] in, This is the normalized adjacency matrix. For degree matrix, This is the original adjacency matrix. This represents the node feature matrix after the first layer of graph convolution. Represents a non-linear activation function. The input node feature matrix, is a trainable weight matrix.

[0032] The above graph convolution operation can be stacked multiple times to extract higher-level features, and its general formula is as follows:

[0033]

[0034] in, Indicates the first The node feature matrix of the layer, Indicates the first The weight matrix of the layer, This is the updated node feature matrix after graph convolution. The same convolution operation is performed on the sentence-level view.

[0035] The technical effect of performing convolution operations on the two views separately in this step is as follows: independent trainable weight matrices are assigned to word-level and sentence-level features to achieve differentiated feature extraction. Word-level convolution can specifically learn the co-occurrence rules of cancer professional terms, and sentence-level convolution can specifically learn the inter-sentence logical structure of cancer text. This solves the problem of inaccurate feature extraction caused by uniform weights in heterogeneous graph convolution in existing technologies.

[0036] Calculate the overall features. Let the word-level features be represented as... Sentence-level features are represented as The overall feature representation after fusion The calculation formula is as follows:

[0037]

[0038] in This is the weighted fusion coefficient, and its value range is... .

[0039] The effect of introducing the attention-based adaptive weighted fusion technique in this step is that the model does not use fixed fusion coefficients. Instead, we design a learnable attention network that uses word-level features. and sentence-level features As input, the most suitable fusion weights for the current text sample are dynamically calculated. Weighted fusion coefficients. The attention network automatically generates features based on the characteristics of the input text, such as text length, sentence complexity, and density of technical terms. This enables the model to adaptively balance the importance of local and global features according to different types of cancer texts, solving the problem of heterogeneous graph features lacking weighted fusion and being unable to adapt to different types of cancer texts in existing technologies.

[0040] Obtain fusion features Then, classification tasks can be performed based on the fused features. The fused features are input into a multilayer perceptron (MLP) classification network to complete the final text classification task.

[0041] The present invention also discloses a cancer text classification system based on a multi-view graph convolutional network. The system includes at least a processor and a memory connected to the processor, wherein: the memory is used to store computer programs or instructions; the processor is used to acquire and execute the computer programs or instructions in the memory so that the system performs the cancer text classification method disclosed in the present invention.

[0042] Beneficial effects:

[0043] This invention combines multi-view feature learning with graph convolutional neural networks to achieve joint modeling of semantic information in text at both the lexical and sentence levels, simultaneously capturing fine-grained local features and global semantic relationships. Compared with existing TCM text classification methods based on hybrid graph diffusion models and heterogeneous graph convolutional neural networks, this invention differs fundamentally in both model architecture and module design. The core distinguishing features and corresponding technical effects are as follows:

[0044] 1) Clearer semantic modeling and higher feature purity: By constructing two independent homogeneous graphs at the word and sentence levels, this invention avoids the semantic confusion that may result from directly mixing different types of nodes in heterogeneous graphs. Graph convolution operations within each view are performed in a semantically cohesive space, ensuring the purity and interpretability of the learned features and providing higher-quality basic features for subsequent accurate classification.

[0045] 2) Lower computational complexity and stronger model robustness: This invention uses standard multi-layer GCN stacking to capture high-order neighbor information, abandoning the computationally expensive graph diffusion mechanism. This not only significantly reduces the time cost of model training and inference, but also effectively avoids the problem of over-smoothing of node features caused by excessive diffusion, making the model exhibit stronger robustness and generalization ability when facing noisy data and imbalanced samples.

[0046] 3) A more intelligent and adaptive feature fusion mechanism: This invention introduces an attention mechanism into the fusion process of dual-view features for the first time, enabling dynamic learning of fusion weights. Unlike the fixed weighting coefficients in previous applications that require manual adjustment, the fusion strategy of this invention can adaptively adjust the contribution ratio of word-level and sentence-level features according to the semantic characteristics of the input text, thereby generating a more discriminative and flexible comprehensive text representation. Experiments show that this mechanism can consistently and stably improve classification accuracy and F1 score when processing diverse cancer texts. Attached Figure Description

[0047] Figure 1 This is a schematic diagram illustrating the overall process of the cancer text classification method based on multi-view graph convolutional networks proposed in this invention.

[0048] Figure 2This is a schematic diagram of the word-level and sentence-level embedding generation process proposed in this invention.

[0049] Figure 3 This is a schematic diagram of the construction process of the multi-view diagram structure proposed in this invention. Detailed Implementation

[0050] The present invention will be further described below with reference to embodiments, but the scope of protection of the present invention is not limited thereto:

[0051] This application provides a cancer text classification method based on a multi-view graph convolutional network. The technical content of the present invention will be described in detail below with reference to the accompanying drawings and specific examples.

[0052] To address the shortcomings of existing medical text classification methods, such as low vocabulary recognition accuracy, insufficient structural information capture, and poor model generalization ability, this invention proposes a text classification method (Cancer-MVGCN) specifically designed for the characteristics of cancer text. This method integrates multi-view learning and Graph Convolutional Networks (GCN). It constructs graph structures at both the word and sentence levels, extracts multi-layer feature representations using graph convolution operations, and then achieves accurate text classification through multi-view feature fusion. Figure 1 The specific implementation method is as follows:

[0053] First, combined Figure 2 The text is then embedded. This step includes word-level embedding and sentence-level embedding. The input text is cleaned and standardized, and word vectors are generated using the FastText model. FastText then embeds each word... Decompose it into its character-level n-gram set And represent the word vector as the sum of all its n-gram vectors, that is:

[0054]

[0055] in, For words The vector representation of , is the embedding vector of the character n-gram.

[0056] Secondly, in order to capture the overall semantic information of the sentence, the BGEM3 pre-trained model is used to vectorize the sentence, thus transforming the input sentence... Mapped to a dense vector of fixed dimensions :

[0057]

[0058] Through the above two-level embedding, this invention obtains two types of feature representations: word nodes and sentence nodes, providing a foundation for the construction of multi-view graph structures.

[0059] Combination Figure 3 A multi-view graph structure is constructed. For each text, two equally constructed graphs are created: a "word node view" and a "sentence node view." In the word node view, the cosine similarity between word vectors is used to determine whether to establish edge connections between nodes.

[0060]

[0061] When the similarity is greater than the threshold of 0.6, an edge connection is established. Otherwise, no edges are connected. The sentence node view is constructed in the same way, that is, an inter-sentence relationship graph is constructed based on the similarity between sentence vectors.

[0062] Through this dual-view structure, the model simultaneously preserves the local features of the text at the word level and the global semantic relationships at the sentence level.

[0063] Learning is performed based on multi-view features derived from graph convolution. After constructing the views, the word view graphs are then processed separately. With sentence view diagram Input the corresponding graph convolutional network module GCN With GCN Taking word view as an example, let the input feature matrix be... The adjacency matrix is Its normalized form is:

[0064]

[0065] Then the first The graph convolution output of the layer is:

[0066]

[0067] in, For activation function, This is a learnable weight matrix. By stacking multiple layers of GCN, the model gradually aggregates contextual dependencies and global structural information to obtain high-level semantic representations.

[0068] The feature extraction method for sentence-level views is the same, resulting in sentence-level graph convolutional features. .

[0069] Multi-view feature fusion is performed. To integrate information from different views, this invention designs a weighted fusion mechanism that sums the features of two views using weighted methods:

[0070]

[0071] in, The adjustable weighting parameters are used to balance the importance of word-level and sentence-level features. This weighted fusion strategy can effectively combine local word relationships and inter-sentence logical structure, improving the integrity and robustness of the overall representation.

[0072] Perform text classification and output. The fused text features... The input is fed into a fully connected multilayer perceptron (MLP) classifier, which outputs the probability distributions of each category after a nonlinear transformation and a softmax layer. The model is trained end-to-end using the cross-entropy loss function, and the optimization objective is to minimize the difference between the predicted results and the true labels.

[0073] Performance was validated and experimental results were presented. In this embodiment, the Cancer-MVGCN model of this invention was validated on two cancer text datasets (AIM and SPS) along with multiple baseline models. The experimental results are shown in Table 1.

[0074] Table 1. Performance comparison of each model on the cancer text dataset (%)

[0075]

[0076] As shown in the table above, the Cancer-MVGCN model of this invention significantly outperforms traditional methods and single-view GCN models on all datasets. In the AIM medical dataset, compared to the second-best performing model, accuracy is improved by 1.9%, and the F1 score is improved by 1.54%. In the SPS dataset, the accuracy and F1 score of this invention significantly surpass all comparative models. This result fully demonstrates that the proposed dual-view independent modeling and adaptive feature fusion mechanism can more accurately capture local and global semantic features when processing terminology-dense and structurally complex cancer text, thereby achieving superior classification performance.

[0077] Ablation and fault tolerance experiments were designed. To verify the importance of the multi-view structure and fusion strategy, this invention conducted ablation experiments. The experiments designed three variant models to be compared with the complete Cancer-MVGCN model:

[0078] CMVGCN_w / oMV: Removes the multi-view fusion module and directly concatenates the features of word-level and sentence-level views before inputting them into the classifier.

[0079] CMVGCN_w / oViewS: Removes sentence-level views and retains only word-level views for graph convolutional feature learning.

[0080] CMVGCN_w / oViewW: Removes word-level views and retains only sentence-level views for graph convolutional feature learning.

[0081] Table 2 Ablation Experiment Results (%)

[0082]

[0083] As shown in Table 2, removing any view module or multi-view fusion module significantly reduces model performance, indicating that each part is indispensable in the classification task. Furthermore, this system features parameter adaptation and anomaly handling capabilities, automatically adjusting the batch size and GCN layer number based on the input text length, number of nodes, and GPU memory capacity, thereby ensuring computational stability and result reliability.

[0084] The specific embodiments described herein are merely illustrative of the spirit of the invention. Those skilled in the art to which this invention pertains may make various modifications or additions to the described specific embodiments or use similar methods to substitute them, without departing from the spirit of the invention or exceeding the scope defined by the appended claims.

Claims

1. A cancer text classification method based on multi-view graph convolutional networks, characterized in that... It includes the following steps: S1. Obtain the text corpus file, word segmentation dictionary and stop word list, perform noise reduction, word segmentation and formatting on the original text, and generate the cleaned standard corpus; S2. Perform word embedding and sentence embedding construction; S3. Construct word-level views using words as nodes and sentence-level views using sentences as nodes; And establish edges between nodes; S4. Multi-view graph convolution feature learning: Graph convolution operations are performed on both word-level and sentence-level views to extract local and global features. The calculation method is as follows: in, This represents the normalized adjacency matrix. Indicates the first The node feature matrix of the layer, The node feature matrix updated after graph convolution. Indicates the first The weight matrix of the layer, Represents a non-linear activation function; S5, Multi-view Feature Fusion; S6. Text classification and model optimization: The fused features are input into a multilayer perceptron (MLP) classification network optimized for cancer text classification tasks, and the classification results corresponding to the text are output.

2. The method according to claim 1, characterized in that, The word embedding construction in S2 specifically includes: The FastText model is used to train the cleaned standard corpus to obtain lexical-level embedding representations. During model training, each word vector... It is obtained by summing all character-level n-gram vectors of the word, satisfying: In the formula, Indicator The final embedding vector, Indicates belonging to a set A single character-level n-gram subsequence; This represents the character n-gram. The vector representation of .

3. The method according to claim 1, characterized in that, S2 sentence embedding construction specifically includes: The pre-trained language model BGEM3 is used to encode each sentence in the text, generating dense sentence vectors of fixed dimensions. : ,in Represents a sentence in the text.

4. The method according to claim 1, characterized in that, In S3, edges are established between nodes by calculating the cosine similarity between nodes and comparing them using a similarity threshold. In the formula, This indicates calculating the cosine similarity. , These represent the vector representations of the two words respectively. For vector dot product, The L2 norm of the corresponding vector; when At the time, at the word node and Establish undirected edges between ;when At that time, no connection is established.

5. The method according to claim 4, characterized in that, Similarity threshold .

6. The method according to claim 1, characterized in that, In S4: in, Represents the original adjacency matrix. Representation degree matrix.

7. The method according to claim 1, characterized in that, S5 will include word-level features Sentence-level features By weighting factor By fusing the features, a comprehensive feature representation is obtained: .

8. The method according to claim 1, characterized in that, In S6, the MLP classification network contains three fully connected layers. The dimension of the first fully connected layer is twice the dimension of the fusion feature, the dimension of the second fully connected layer is the same as the dimension of the fusion feature, and the dimension of the third fully connected layer is the number of classification categories of the cancer text. The classification probability of each cancer text category is output through the Softmax function, and the cross-entropy loss function is used for training and optimization. Finally, the classification result file corresponding to the text is output.

9. A cancer text classification system based on a multi-view graph convolutional network, characterized in that, The system includes at least one processor and a memory connected to the processor, wherein: The memory is used to store computer programs or instructions; The processor is configured to acquire and execute computer programs or instructions in memory to cause the system to perform the method as described in any one of claims 1-8.

Images