A multi-channel feature collaborative text classification method and system

Abstract

The present invention belongs to but is not limited to the field of text classification technology, and in particular relates to a multi-channel feature collaborative text classification method and system, comprising: in the encoding stage, first inputting the target text information into the multi-channel feature collaborative module, and extracting feature vectors with different position information emphases through two independent encoders; these features are processed by the feature processing module to obtain output features; before classifying the text, the output features are also screened and judged by the threshold tuning strategy module. Before extracting features from the input text, attention is paid to the association between label information and text information, and a multi-channel is established to mine text information. At the same time, based on the threshold tuning strategy, semantically ambiguous texts that are difficult for the model to understand and semantically simple texts that are easy to understand are processed separately to further improve the generalization ability and classification performance of the model. Experimental results on four public data sets show that the FSTTS model has good generalization and effectiveness.

Description

Technical Field

[0001] The present invention belongs to but is not limited to the field of text classification technology, and in particular relates to a multi-channel feature collaborative text classification method and system. Background Art

[0002] Text classification is a core task in natural language processing, widely used in scenarios such as sentiment analysis, comment classification, and question-answering systems. However, the diversity and complexity of text data make improving text classification accuracy a pressing challenge. The structural characteristics of complex text make extracting effective information from it inefficient and time-consuming, and traditional methods have limitations in coping with text complexity.

[0003] Traditional deep learning methods for text classification primarily include convolutional neural networks (CNNs), recurrent neural networks (RNNs), attention mechanisms, and models improved upon these approaches. Kim et al. applied convolutional neural networks (CNNs) to text classification tasks, utilizing convolutional kernels of varying sizes to extract key information from text and better capture local correlations. Zhang et al. proposed Char-CNN, a character-level convolutional neural network. Experimental results show that when the training set is large enough, convolutional networks can achieve good results even without considering word meaning and grammatical structure. In the application of RNNs, Miklov et al. improved the RNN model, enhancing its ability to process sequential data. However, both traditional CNN and RNN models have limitations: RNNs may experience vanishing or exploding gradients when processing text, while CNNs are limited by long-term dependencies. To address this, Lai et al. proposed the RCNN model, a hybrid of RNN and CNN. This model uses the RNN to capture contextual information and employs max pooling to capture the underlying semantic information in the text. Its core concept is to combine the advantages of both while addressing their respective limitations. Xiao et al. proposed a neural network architecture that effectively encodes character input using convolutional and recurrent layers. This model achieves comparable performance to other models while requiring fewer parameters.

[0004] The introduction of the attention mechanism enables neural networks to pay different attention to different sentences when training text, achieving more reasonable natural language modeling. This method not only has stronger parallel capabilities, but also significantly improves training efficiency. Therefore, more and more neural networks have begun to integrate the attention mechanism. The Att-BLSTM model proposed by ZHOU et al. combines the attention mechanism with the bidirectional LSTM to effectively capture the semantic information in the sentence and convert it into useful feature representations. The BERT model is completely based on the Transformer architecture. It not only has excellent generalization but also amazing performance. Therefore, many excellent models have begun to be improved based on BERT. Lan et al. proposed a lightweight pre-training model ALBERT (ALite BERT). By adopting technologies such as factorized embedding parameters, cross-layer parameter sharing, and sentence order prediction loss, they successfully reduced the number of model parameters and improved performance, which will help apply natural language processing technology to more scenarios.

[0005] Chen et al. proposed a new semi-supervised text classification algorithm, MixText. The key lies in using a data augmentation method called TMix to generate new samples by interpolating in the latent space. This method performs well in few-shot scenarios, surpassing pre-trained models and other semi-supervised methods. Meng et al. proposed a method, LOTClass, to train text classification models using only labels without requiring any labeled documents as training data, achieving performance comparable to or even better than strong semi-supervised and supervised models. In response to the fact that directly training and deploying advanced methods on large-scale pre-trained models is not always satisfactory, Lin et al. proposed a method that combines linear classifiers with advanced methods and applies them to text features, and verified its effectiveness through experiments. Kim et al. proposed a simple framework, DRAFT, that can classify arbitrary topics. This framework uses a dense retriever to construct a customized dataset and fine-tune the classifier. With an extremely low number of parameters, DRAFT shows good performance in few-shot topic classification tasks.

[0006] Based on the above analysis, the current state of the art faces a pressing technical challenge: some existing text classification algorithms focus on fine-tuning the model or adjusting the classifier. However, not only do they inadequately explore the potential connections between text and labels, but they also fail to comprehensively consider the multifaceted nature of features. In terms of supervised training, much of the work still requires further refinement to enhance the model's ability to understand texts of varying complexity. Summary of the Invention

[0007] In view of the problems existing in the prior art, the present invention provides a multi-channel feature collaborative text classification method and system

[0008] The present invention is implemented as follows: a multi-channel feature collaborative text classification method comprising:

[0009] S1, in the encoding stage, the target text information is first input into the multi-channel feature collaboration module, and feature vectors with different position information emphases are extracted through two independent encoders;

[0010] S2, these features are processed by the Feature Processing Module (FPM) to obtain output features;

[0011] S3, before classifying the text, the output features need to be screened and judged by the threshold tuning strategy module.

[0012] Furthermore, in text classification tasks, text position information is crucial to model understanding and classification performance. The order of words and sentences in a text not only carries semantic meaning but also reflects dependencies between contexts. Accurately capturing and utilizing this position information can help models gain a deeper understanding of the structure and meaning of text, improving their ability to discern complex semantics.

[0013] Based on the importance of text position information, a multi-channel feature collaboration module is designed. It consists of two main components: a feature encoder module and a feature processing module. The feature encoder module integrates two independent encoders—the RoBERTa encoder and the RoFormer encoder. During the encoding phase, to obtain richer text features, these two encoders receive different input texts and extract feature representations with specific focuses.

[0014] Furthermore, using a label-aware data augmentation method, all labels are inserted before the sentence of the RoBERTa encoder to form a new sequence token_1. Using this sequence for feature extraction, the potential connection between the label and the sentence can be obtained, and the text feature fb based on label enhancement is obtained, as shown in Formula 5-6:

[0015] token_1=[[CLS],c1,c2,…,c k , x1,…,x n ] (5)

[0016] f b =RoBERTa(token_1) (6)

[0017] Furthermore, for the RoFormer encoder, its unique rotation position encoding method increases the learning of relative position information between sequence elements. Since the RoBEREa encoder has already incorporated label information, the RoFormer encoder is used only to extract features of the text sentence and obtain the feature f containing relative position information. r As shown in Formula 7-8:

[0018] token_2=[x1,x2,…,x n ] (7)

[0019] f r =RoFormer(token_2) (8)

[0020] Take the feature f r With f b Perform the splicing feature fusion method to form the output feature f c , as shown in Formula 9.

[0021] f c =cat(f r , f b ) (9)

[0022] Furthermore, considering the potential dimensional disaster caused by overly long features, the concatenated feature vectors undergo three dimensionality reduction processes through the feature processing module. This module includes a linear layer, a RELU activation function layer, a BN layer, and a Dropout layer, as shown in Formulas 10-14.

[0023] Splicing feature f c First, after the first linear layer, the dimension parameter is reduced from 3328 to 2024, and the output feature f is obtained. l1 After passing through the activation function RELU and BN layer, it enters the Dropout layer. After multiple experiments, the parameter is set to 0.45.

[0024] f l1 =BatchNormld(RELU(Linear(f c ))) (10)

[0025] f l1 =Dropout(f l1 , p=0.45) (11)

[0026] The dimension parameter of the second linear layer is reduced from 2024 to 1024, and the parameter of the Dropout layer is set to 0.3 after multiple experiments.

[0027] f l2 =BatchNormld(RELU(Linear(fl1 ))) (12)

[0028] fl2=Dropout(fl2,p=0.3)(13)

[0029] After the last feature processing, the dimension of feature fl2 is reduced to 768, and the output feature f suitable for subsequent text classification tasks can be obtained.

[0030] f=Linear(f l2 , output dim ) (14)

[0031] Furthermore, the threshold tuning strategy dynamically adjusts the distinction between semantically ambiguous and semantically simple text based on upper and lower thresholds. Through the threshold filtering function, different decisions are made based on the differences between these two types of text to improve the model's understanding capabilities. This processing approach is more consistent with human thinking habits, and its effectiveness has been verified through experiments.

[0032] The semantic fuzzy text screening strategy is shown in Formula 15, where λ represents the fuzzy category screening threshold, i.e., the upper threshold of the threshold tuning strategy module. Assuming that the kth text is a semantic fuzzy text, its loss function is represented as Lyk, and the average training loss is L avg For semantically ambiguous text that meets the formula, the model will use a re-learning method to deepen the model's understanding of this type of text.

[0033] L yk ≥λL avg (15)

[0034] Similarly, the semantically simple text category screening threshold μ is the lower threshold of the threshold tuning module. In order to avoid the model fitting being too biased towards semantically simple texts, the filtered semantically simple texts will be excluded from the current training. The semantically simple text screening strategy is shown in Equation 16:

[0035] L yk ≤μL avg (16)

[0036] When the upper threshold λ of the threshold screening function is 1.10 and the lower threshold μ is 0.93, the model achieves the best accuracy improvement on the SST-2 dataset.

[0037] Another object of the present invention is to provide a multi-channel feature collaborative text classification system for implementing the multi-channel feature collaborative text classification method, comprising:

[0038] The two core components of the model are the multi-channel feature collaboration module MCFSM (Multi-Channel Feature Synergy Module) and the threshold tuning policy module TTPM (Threshold Tuning Policy Module).

[0039] The multi-channel feature collaboration module consists of two main components: a feature encoder module and a feature processing module. The feature encoder module integrates two independent encoders - the RoBERTa encoder and the RoFormer encoder. During the encoding stage, in order to obtain richer text features, the two encoders receive different input texts and extract feature representations with specific focuses.

[0040] The threshold tuning strategy module makes judgments based on upper and lower thresholds and dynamically adjusts the definition of semantically ambiguous text and semantically simple text.

[0041] Another object of the present invention is to provide a computer device, which includes a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the multi-channel feature collaborative text classification method.

[0042] Another object of the present invention is to provide a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of the multi-channel feature collaborative text classification method.

[0043] Another object of the present invention is to provide an information data processing terminal, which includes the multi-channel feature collaborative text classification system.

[0044] In combination with the above technical solutions and the technical problems solved, the advantages and positive effects of the technical solutions to be protected by the present invention are as follows:

[0045] First, in response to the limitations of text information utilization in traditional text classification tasks and the lack of consideration of texts with different semantic complexities during model training, the present invention proposes a multi-channel feature collaborative text classification model FSTTS based on a threshold tuning strategy. Before extracting features from the input text, the model focuses on the relationship between label information and text information, and establishes a multi-channel approach to mining text information. At the same time, based on the threshold tuning strategy, semantically ambiguous texts that are difficult for the model to understand and semantically simple texts that are easy to understand are processed separately, further improving the model's generalization ability and classification performance. Experimental results on four public datasets have shown that the FSTTS model has good generalization and effectiveness.

[0046] Second, the patent of this invention proposes an efficient and accurate real-time monitoring method for online public opinion by making full use of the massive real-time text data in social platforms. This method is based on text mining and natural language processing technology. By analyzing multi-dimensional information such as keywords, emotional tendencies, and user behavior patterns in social platforms, it realizes the dynamic capture and trend analysis of online public opinion. Unlike traditional public opinion monitoring systems, the present invention does not require additional hardware support and can run efficiently based on existing computing resources, greatly reducing installation and maintenance costs. This lightweight design is not only suitable for the public opinion management needs of governments and enterprises, but also convenient for families and individuals to use in daily life.

[0047] This invention specifically addresses the current social context in which online public opinion has permeated everyday life, designing a real-time public opinion monitoring solution applicable to a variety of life scenarios. Through efficient data acquisition and processing modules, users can quickly access the latest developments in online public opinion, including the spread of hot topics, the scope of negative sentiment, and early warning information for potential public opinion crises. Furthermore, the invention also incorporates sentiment analysis algorithms to accurately identify emotional fluctuations within public opinion content, providing users with a more intuitive display of public opinion status and response recommendations.

[0048] At the technical level, this invention ensures the system's stability and real-time performance in highly concurrent data environments through a large-scale data processing optimization algorithm and an adaptive model adjustment mechanism. Furthermore, the system supports flexible customization, allowing users to set public opinion keywords, monitoring ranges, and warning thresholds based on their needs, thereby enabling precise monitoring of specific topics or events. This highly intelligent approach to public opinion analysis significantly reduces the user's reliance on technical background knowledge, making it easy for non-professional users to master and operate the system.

[0049] The practical application value of the present invention is reflected in its purification of the network environment and the maintenance of a healthy public opinion ecology. By grasping the trends of online public opinion in real time, users can quickly respond to public opinion crises, convey positive voices in a timely manner, and prevent the spread of false information. Especially in the era of social media where information dissemination is rapid and widespread, the present invention provides strong technical support for building a harmonious network environment, helps to improve the efficiency of public opinion management, and promotes the healthy development of cyberspace. In summary, the present invention meets the needs of the current era and has important social significance and economic value. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 Schematic diagram of the RoBERTa model structure provided by an embodiment of the present invention;

[0051] Figure 2 Schematic diagram of the implementation of rotational position encoding provided by an embodiment of the present invention;

[0052] Figure 3 Schematic diagram of the FSTTS model structure provided by an embodiment of the present invention;

[0053] Figure 4 This is a flow chart of FSTTS training provided by an embodiment of the present invention;

[0054] Figure 5 This is a schematic diagram of an example of binary classification of semantically ambiguous text provided by an embodiment of the present invention;

[0055] Figure 6 This is a flow chart of a multi-channel feature collaborative text classification method provided by an embodiment of the present invention;

[0056] Figure 7 This is a structural diagram of a multi-channel feature collaborative text classification system provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0057] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0058] During the encoding phase, this method extracts features from the input target text using a multi-channel feature collaboration module. This module consists of two independent encoders, each focusing on generating feature vectors based on different location information. For example, one encoder might focus on the global semantics of the text, while the other prioritizes fine-grained features of the local context. Through collaborative encoding, the two encoders perform in-depth analysis of the text content from different dimensions, ensuring comprehensive and rich feature extraction, laying the foundation for subsequent feature processing and classification.

[0059] After multi-channel feature extraction, these features are passed to the feature processing module for further processing. This module's main function is to fuse and optimize the feature vectors from the two channels. Through techniques such as feature alignment, weighting, or dimensionality reduction, it unifies the dimensionality and semantic information of different features. This fusion process eliminates feature redundancy and retains the key information of the text, thereby generating high-quality output features and providing precise semantic support for subsequent classification tasks.

[0060] Before text classification, output features are filtered and judged by the Threshold Tuning Strategy module. This module further optimizes features based on preset threshold conditions, selecting features that significantly contribute to the classification task while filtering out noise or redundant information. This process dynamically adjusts feature activation weights, enhancing adaptability to classification targets and improving the model's classification accuracy and stability.

[0061] Ultimately, the features extracted, processed, and threshold-tuned through multi-channel feature extraction are fed into the classifier for classification. Thanks to the collaborative optimization of the previous modules, this method comprehensively considers the semantic information of the text from both global and local perspectives, significantly improving the accuracy and robustness of the classification results. This multi-channel collaborative approach is particularly well-suited for complex text classification tasks and can effectively address the semantic diversity and uncertainty in multi-category text classification scenarios.

[0062] The BERT model combines the Masked Language Model (MLM) task with a bidirectional Transformer encoder architecture, trained on a large unlabeled corpus to obtain a deep feature representation of the input text. While leveraging an attention mechanism to globally understand contextual information, the model employs a fixed random masking approach to prevent overfitting. Furthermore, the Next Sentence Prediction (NSP) task helps improve the model's adaptability to diverse downstream tasks, providing rich semantic representations for these tasks.

[0063] The RoBERTa model is an improved pre-training model based on the BERT model. Compared to the BERT model, RoBERTa introduces a dynamic masking mode during the data preprocessing stage, generating a dynamic mask for each input sequence. This improvement enables the model to gradually adapt to different masking strategies and learn different language representations with the continuous input of large amounts of data. During the pre-training stage, RoBERTa uses the BPE (Byte Pair Encoding) method, expands the vocabulary size, and does not pre-process the input text. Based on these improvements, RoBERTa has greater robustness and accuracy in capturing deep language details.

[0064] The RoFormer model is a language model based on the Transformer architecture, which uses the Rotary Position Embedding (RoPE) method. This method uses a rotation matrix to encode the absolute position of word embeddings and embeds the relative position information into the self-attention mechanism. The core idea is to encode the relative position by multiplying the word embedding representation (q and k in Transformer) with the rotation matrix, realizing the fusion of relative position encoding and linear self-attention, providing flexibility in sequence length, and making the dependency between characters decrease as the relative position increases. The implementation of RoPE is as follows: Figure 2 shown.

[0065] In order to achieve relative position encoding, RoPE adds absolute position information to q and k through formula (1).

[0066]

[0067] where q m is the mth element of q, k n Similarly, in order to include relative position information, the present invention uses a function g, which represents the m With k n The inner product between the generated vectors represents that this function only takes the word embeddings q and k and their relative positions mn as input variables, as shown in formula (2).

[0068] q m T k n = <f q (q, m), f k (k, n)>=g(q, k, mn) (2)

[0069] The function f(x) containing absolute position information is converted into a matrix form using a complex formula, as shown in formula (3). Since matrix multiplication usually represents a spatial transformation operation, it can be seen that formula (3) represents a rotation operation on q, so it is called a rotation position encoding method.

[0070]

[0071] The expression extended to higher dimensions can be simplified as formula (4).

[0072]

[0073] Theoretical analysis of the RoPE method employed by RoFormer demonstrates that relative position can be naturally represented through vector products within the self-attention mechanism. This method represents the relative position relationship between features through the rotation angle between vectors, achieving the effect of encoding relative positions using absolute position encoding, thereby enhancing the model's ability to understand semantically ambiguous text. This method offers advantages such as flexibility, attenuation, and scalability.

[0074] This paper proposes a text classification model FSTTS that combines threshold tuning strategy and multi-channel feature collaboration. Its overall framework is as follows Figure 3 The two core components of the model are the Multi-Channel Feature Synergy Module (MCFSM) and the Threshold Tuning Policy Module (TTPM).

[0075] During the encoding phase, the target text is first fed into the Multi-Channel Feature Collaboration Module. Two independent encoders then extract feature vectors emphasizing different positional information. These features are then processed by the Feature Processing Module (FPM) to generate output features. Before the text is classified, the output features are further filtered and judged by the Threshold Tuning Strategy Module.

[0076] In text classification tasks, text position information is crucial to model understanding and classification performance. The order of words and sentences in a text not only carries semantic meaning but also reflects dependencies between contexts. Accurately capturing and utilizing this position information can help models gain a deeper understanding of the structure and meaning of text, improving their ability to discern complex semantics.

[0077] Based on the importance of text position information, the present invention designs a multi-channel feature collaboration module. The multi-channel feature collaboration module consists of two main components: a feature encoder module and a feature processing module. Figure 4 As shown in Figure 2, the feature encoder module integrates two independent encoders—the RoBERTa encoder and the RoFormer encoder. During the encoding phase, to obtain richer text features, these two encoders receive different input texts and extract feature representations with specific focuses.

[0078] Labels in datasets often have an implicit association with the text. For example, in the sentence "George purchased a small interest in which baseball team?" from the TREC dataset, the subject "George" is a person's name, so the text should be assigned the label "human." Associating label information with textual information can effectively enhance the model's understanding capabilities. Inspired by this, the present invention introduces a label-aware data augmentation method for the RoBERTa encoder.

[0079] The input to the RoBERTa encoder typically consists of a label and a sentence for classification, but this input clearly ignores the potential impact of labels on classification. To address this lack of label information, this paper uses a label-aware data augmentation method to insert all labels before the sentence in the RoBERTa encoder to form a new sequence token_1. Feature extraction using this sequence can capture the potential connection between labels and sentences, resulting in a text feature fb based on label augmentation, as shown in Formula 5-6:

[0080] token_1=[[CLS],c1,c2,…,c k , x1,…,x n] (5)

[0081] f b =RoBERTa(token_1) (6)

[0082] For the RoFormer encoder, its unique rotation position encoding method increases the learning of relative position information between sequence elements. Since the RoBEREa encoder has already incorporated label information, the RoFormer encoder is used only to extract features of text sentences to obtain features containing relative position information. r As shown in Formula 7-8:

[0083] token_2=[x1,x2,…,x n ] (7)

[0084] f r =RoFormer(token_2) (8)

[0085] Zhang Haifeng et al. proposed to extract domain common features and characteristic features based on the fusion of dual BERT model and Fpnet, and to purify them with feature projection to improve the classification effect of the model. However, this method is sensitive to the choice of hyperparameters and has poor interpretability. Therefore, in order to ensure that the final output features retain as much information as possible, the present invention adopts the method of converting feature f r With f b Perform the splicing feature fusion method to form the output feature f c , as shown in Formula 9.

[0086] f c =cat(f r , f b ) (9)

[0087] Considering the dimensionality disaster that may occur due to overly long features, the present invention performs three-dimensionality reduction processing on the concatenated feature vectors through a feature processing module. This module includes a linear layer, a RELU activation function layer, a BN layer, and a Dropout layer, as shown in Formulas 10-14.

[0088] Splicing feature f c First, after the first linear layer, the dimension parameter is reduced from 3328 to 2024, and the output feature f is obtained. l1 After passing through the activation function RELU and BN layer, it enters the Dropout layer. After multiple experiments, the parameter is set to 0.45.

[0089] f l1 =BatchNormld(RELU(Linear(f c ))) (10)

[0090] f l1 =Dropout(f l1 , p=0.45) (11)

[0091] The dimension parameter of the second linear layer is reduced from 2024 to 1024, and the Dropout layer parameter is set to 0.3 after multiple experiments.

[0092] f l2 =BatchNormld(RELU(Linear(f l1 ))) (12)

[0093] f l2 =Dropout(f l2 , p=0.3) (13)

[0094] After the last feature processing, the feature f l2 The dimension is reduced to 768, and the output feature f suitable for subsequent text classification tasks can be obtained.

[0095] f=Linear(f l2 , output dim ) (14)

[0096] In the field of text classification, the ultimate performance of a model depends on the effectiveness of the training process. To achieve good classification results, many existing models continuously iterate and optimize parameters until a pre-set performance threshold is reached. However, this traditional training mechanism fails to fully consider the impact of text that differs significantly from standard text complexity during training on the model's understanding ability. In fact, distinguishing between semantically ambiguous and semantically simple text can effectively improve the model's overall understanding ability.

[0097] like Figure 5 As shown in the figure, for a text binary classification task, the traditional training iteration method may cause the model to lack depth in understanding ambiguous semantic text with unclear meaning or prone to ambiguity, resulting in the model being insensitive to the differences between features.

[0098] From the perspective of loss function, semantically ambiguous text suffers a greater loss during training, while semantically simple text suffers a smaller loss. Based on this difference, the present invention designs and implements a threshold tuning strategy. This strategy makes judgments based on upper and lower thresholds and dynamically adjusts the definition of semantically ambiguous text and semantically simple text. Figure 5 As shown in the figure, through the threshold screening function, different decisions are taken according to the differences between the two types of text to improve the model's understanding ability. This processing method is more in line with human thinking habits in form, and the effectiveness of this method has been verified through experiments.

[0099] The semantic fuzzy text screening strategy is shown in Formula 15, where λ represents the fuzzy category screening threshold, i.e., the upper threshold of the threshold tuning strategy module. Assuming that the kth text is a semantic fuzzy text, its loss function is expressed as L yk For semantically ambiguous texts that satisfy the formula, the model will adopt a re-learning method to deepen the model's understanding of this type of text.

[0100] L yk ≥λL avg (15)

[0101] Similarly, the semantically simple text category screening threshold μ is the lower threshold of the threshold tuning module. In order to avoid the model fitting being too biased towards semantically simple texts, the filtered semantically simple texts will be excluded from this training. The semantically simple text screening strategy is shown in Equation 16:

[0102] L yk ≤μL avg (16)

[0103] During the actual training process, in order to screen out the most appropriate upper threshold λ and lower threshold μ, the present invention conducted multiple experiments on the SST-2 dataset. After multiple rounds of experiments, it was proved that when the upper threshold λ of the threshold screening function is 1.10 and the lower threshold μ is 0.93, the model has the best accuracy improvement effect on the SST-2 dataset. The experimental results are shown in Section 3.4 of the present invention.

[0104] In order to verify the effectiveness of the model of the present invention, the present invention selects 4 benchmark text classification datasets for experiments.

[0105] The SST-2 dataset is a binary classification dataset for sentiment analysis, labeled as positive or negative sentiment.

[0106] The SUBJ dataset is a review dataset with sentences labeled as subjective or objective.

[0107] TREC dataset is a question-answering dataset divided into 6 categories for question type classification

[0108] The CR dataset is a sentiment analysis dataset consisting of customer reviews, which is used to judge the positive or negative sentiment of product reviews.

[0109] Table 1 summarizes the statistics of these datasets.

[0110] Table 1 Dataset statistics

[0111]

[0112] The environments of the present invention and the ablation experiment are both based on Python-3.10 and PyTorch2.1.0, the CPU is AMD EPYC7T83, the graphics card is NVIDIA GeForceRTX4090-24G, and the CUDA version is 12.2, see Table 2.

[0113] Table 2 Experimental environment

[0114]

[0115]

[0116] The present invention uses accuracy (Acc) to evaluate the text classification results of the present invention. The specific calculation is shown in Formula 17:

[0117]

[0118] Among them, TP (True Positive) means that the positive sample in text classification is predicted to be positive, FP (False Positive) means that the negative sample in text classification is predicted to be positive, TN (True Negative) means that the negative sample in text classification is predicted to be negative, and FN (False Negative) means that the negative sample in text classification is predicted to be positive.

[0119] 3.2 Experimental Results and Analysis

[0120] In order to verify the superiority of the model of the present invention in the text classification task, the present invention conducted multiple comparative experiments with BERT, RoBERTa, EFL, LM-CPPF models, and the experimental results are shown in Table 3. Among them, the best results in each data set are marked in bold. As can be seen from Table 3, the model of the present invention after optimization of semantics and features is superior to the baseline model DualCL (using RoBERTa encoder) in all aspects, and achieved accuracies of 94.29%, 97.50%, 98.20% and 94.58% on the SST-2, SUBJ, TREC and CR data sets respectively. Among them, the improvement is the largest on the SST-2 data set, with the accuracy increased by 1.54%.

[0121] Furthermore, the overall experimental results across the four datasets demonstrate that the FSTTS model achieves the best overall performance. While its performance on the SST-2 dataset is slightly lower than that of the EFL model, on the remaining three datasets, the FSTTS model surpasses the EFL model in accuracy, with improvements of 0.40% and 2.08%, respectively. Analysis reveals that the EFL model utilizes a larger pre-trained encoder, RoBERTa-large, and that using a larger pre-trained model significantly improves performance in text classification tasks.

[0122] Table 3 Experimental results of each model on different datasets

[0123]

[0124] Summarizing the performance of the FSTTS model on the SST-2, SUBJ, TREC, and CR datasets, we can infer that the labels in the SST-2 dataset are more closely linked to the textual information, and the distribution of its samples in the feature space is closer to the decision boundary compared to other datasets. Therefore, after optimizing the FSTTS model for features and semantics, the accuracy on this dataset has improved significantly. This is also confirmed by the results of ablation experiments. The excellent performance on the SUBJ, TREC, and CR datasets may be due to the threshold tuning strategy that effectively distinguishes semantically ambiguous from simple text, enhancing the model's generalization ability.

[0125] To verify the effectiveness of the multi-channel feature collaboration module and threshold tuning strategy, we conducted ablation experiments. As shown in Table 4, the ablation experiments were conducted in three parts on four public datasets, with the best results for each dataset shown in bold.

[0126] Table 4 Ablation experiment

[0127]

[0128]

[0129] The first part of the ablation experiment design uses only the multi-channel feature collaboration module, aiming to explore whether there are differences in text understanding between two independent encoders that capture information at different positions in the text. The experimental results show that the FSTTS w / o TTP model, with the addition of the multi-channel feature collaboration module, outperforms the baseline model DualCL on all four datasets. This not only demonstrates the close connection between labels and text, and that fusing labels can improve the model's overall understanding, but also shows that fusing positional information plays a significant role in improving the model's classification performance. Therefore, the addition of this module significantly enhances the model's understanding of text.

[0130] The second part of the experiment adjusted the model training process, increasing the model's processing of semantically ambiguous and semantically simple text, which is more consistent with human thinking from a training perspective. The experimental results show that the classification performance of the FSTTS w / o MCFS model using the threshold tuning strategy has improved significantly on all four datasets. The magnitude of the improvement suggests that the sentences in the SST-2 dataset are mostly distributed near the classification decision line in the feature space, confirming the effectiveness of this strategy.

[0131] The experiments in the third part prove that the simultaneous use of the multi-channel feature collaboration module and the threshold tuning module can further improve the classification ability of the model. The experimental results show that the two modules can not only cooperate with each other in improving the model classification effect, but also have a certain generalization ability.

[0132] The threshold tuning strategy module uses a specific threshold parameter design. The value of this parameter determines the model's tolerance for text variations. Due to differences in text labels, text quality, and text length across datasets, the choice of threshold parameter may also vary.

[0133] Table 5 Accuracy of different upper and lower thresholds on the SST-2 dataset

[0134]

[0135] To explore the impact of different threshold parameter selections within the threshold tuning strategy module on the model's performance on text classification tasks, a threshold selection analysis experiment was designed on the SST-2 dataset. The experimental results are shown in Table 5. As can be seen, the FSTTS model's accuracy on the SST-2 dataset varies significantly when different threshold ranges are adopted. When the upper threshold is selected as 1.10, its mean accuracy is numerically superior to the other upper thresholds. When the lower threshold is selected as 0.93, its mean accuracy also outperforms the other lower thresholds. Comprehensive comparative analysis reveals that on the SST-2 dataset, the FSTTS model achieves optimal performance when the upper and lower thresholds are 1.10 and 0.93, respectively.

[0136] like Figure 6 As shown in FIG, the multi-channel feature collaborative text classification method includes:

[0137] S1, in the encoding stage, the target text information is first input into the multi-channel feature collaboration module, and two independent encoders are used to extract feature vectors with different position information emphases;

[0138] S2, these features are processed by the feature processing module to obtain output features;

[0139] S3, before classifying the text, the output features need to be screened and judged by the threshold tuning strategy module.

[0140] like Figure 7 As shown in the figure, the multi-channel feature collaborative text classification system includes:

[0141] The two core components of the model are the multi-channel feature collaboration module MCFSM and the threshold tuning strategy module TTPM;

[0142] The multi-channel feature collaboration module consists of two main components: a feature encoder module and a feature processing module. The feature encoder module integrates two independent encoders—the RoBERTa encoder and the RoFormer encoder. During the encoding phase, to obtain richer text features, the two encoders receive different input texts and extract feature representations with specific focuses.

[0143] The threshold tuning strategy module makes judgments based on upper and lower thresholds and dynamically adjusts the definition of semantically ambiguous text and semantically simple text.

[0144] The Multi-Channel Feature Collaborative Text Classification System optimizes text classification tasks by combining a Multi-Channel Feature Collaborative Module (MCFSM) with a Threshold Tuning Strategy Module (TTPM). The system's core leverages the representational power of multi-channel features and a dynamic threshold tuning strategy to improve text classification accuracy and generalization, demonstrating enhanced robustness when processing semantically complex and ambiguous text.

[0145] The feature encoder module in MCFSM contains two independent encoders - RoBERTa encoder and RoFormer encoder.

[0146] The RoBERTa encoder aims to efficiently capture the deep semantic features of text and focuses on modeling global semantic information.

[0147] The RoFormer encoder introduces rotational position encoding, focusing on capturing local features and positional relationships.

[0148] During the encoding stage, the two encoders receive different input texts and extract feature representations with specific focuses, thereby forming rich multi-channel text features.

[0149] The feature processing module fuses and optimizes the feature representations output by the RoBERTa and RoFormer encoders.

[0150] -Through the feature alignment mechanism, the scale inconsistency problem between multi-channel features is solved.

[0151] -Use the attention mechanism to give more important features higher weights to ensure the capture of key semantic information.

[0152] The goal of the feature processing module is to generate multi-channel collaborative feature representations to provide high-quality input for subsequent classification.

[0153] TTPM uses dynamic upper and lower thresholds to evaluate the semantic complexity of text.

[0154] Semantically ambiguous text: When the feature complexity is higher than the upper threshold, an enhanced classification strategy is adopted to improve classification accuracy by increasing the weight distribution calculation of the feature processing module.

[0155] Semantically simple text: When the feature complexity is below the lower threshold, the feature processing flow is simplified to improve computational efficiency.

[0156] This dynamic tuning strategy enables the system to work efficiently in different semantic environments.

[0157] When processing text, the system first extracts diverse features using a multi-channel feature collaboration module, then optimizes these features using a threshold tuning strategy module. Finally, the optimized features are input into a classifier to accurately categorize the text. This process is adaptable to multiple text types, significantly improving classification performance in scenarios with semantic uncertainty.

[0158] The system can be deployed on different types of computing devices, including:

[0159] A computer device that collaboratively executes the stored multi-channel feature collaborative text classification method through a memory and a processor;

[0160] Computer-readable storage media for loading and execution on different computing platforms;

[0161] Information data processing terminal, suitable for smart terminal and server environments, supporting large-scale text classification tasks.

[0162] Through the collaborative design of hardware and algorithms, the system has good scalability and a wide range of practical application scenarios. The present invention has been extensively experimentally verified on a variety of text classification datasets, and the two-category datasets SST-2, SUBJ, CR and the six-category dataset TERC were selected as typical test datasets. These datasets cover different text classification scenarios, such as sentiment classification (SST-2), subjectivity analysis (SUBJ), product review classification (CR) and news classification (TERC). By using these datasets, the classification performance of the present invention has been comprehensively evaluated in a variety of tasks.

[0163] In binary classification tasks, the proposed method achieved classification accuracy rates of 96.90%, 97.20%, and 94.02% on the SST-2, SUBJ, and CR datasets, respectively. On the six-category TERC dataset, the proposed method achieved a classification accuracy rate of 98.10%. Experimental results demonstrate that the proposed method exhibits excellent performance across various text classification tasks, particularly in complex multi-classification tasks like TERC. The high accuracy of the proposed method further demonstrates its robustness and broad applicability.

[0164] Comparing experimental results with existing optimal models, the present invention demonstrates superior classification performance overall. Compared to other models, the present invention not only achieves higher classification accuracy on a single dataset, but also maintains stable and consistent classification performance across multiple datasets. This advantage stems from the present invention's innovative design in feature extraction, model optimization, and classification strategy, ensuring more accurate capture of text features and more effective decision-making in classification tasks.

[0165] Experimental results fully demonstrate the superior performance of this invention in the field of text classification. Not only does it reach the leading level of the current state of the art in binary classification tasks, but it also demonstrates significant accuracy improvements in multi-classification tasks. This invention provides a more efficient and reliable technical solution for text classification tasks, and can be widely applied in scenarios such as sentiment analysis, news classification, and product evaluation, with significant value for practical industrial and research applications.

[0166] It should be noted that the embodiments of the present invention can be implemented by hardware, software, or a combination of software and hardware. The hardware portion can be implemented using dedicated logic; the software portion can be stored in a memory and executed by an appropriate instruction execution system, such as a microprocessor or dedicated design hardware. Those skilled in the art will appreciate that the above-mentioned devices and methods can be implemented using computer-executable instructions and / or contained in processor control code, for example, such as a carrier medium such as a disk, CD or DVD-ROM, a programmable memory such as a read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. The devices and modules of the present invention can be implemented by hardware circuits such as very large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, or programmable hardware devices such as field programmable gate arrays, programmable logic devices, etc., can also be implemented by software executed by various types of processors, or can be implemented by a combination of the above-mentioned hardware circuits and software, such as firmware.

[0167] The above description is only a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications, equivalent substitutions and improvements made by any technician familiar with this technical field within the technical scope disclosed by the present invention and within the spirit and principles of the present invention should be covered by the scope of protection of the present invention.

Claims

1. A multi-channel feature collaborative text classification method, characterized in that: include: S1. Feature extraction: In the encoding stage, the target text information is input into the multi-channel feature collaboration module, and two independent encoders are used to extract feature vectors with different position information emphases. The first encoder extracts global semantic features, and the second encoder extracts local context features. S2. Feature processing: Input the feature vector into the feature processing module, fuse and optimize the feature vectors from the two channels, and generate a unified output feature through feature alignment, weighting or dimensionality reduction methods; S3, threshold tuning: input the output features into the threshold tuning strategy module, screen effective features based on preset threshold conditions, filter noise or redundant information, and obtain optimized features; S4, classification decision: input the optimized features into the classifier, classify the target text, and generate the classification results; The threshold tuning strategy dynamically adjusts the distinction between semantically ambiguous and semantically simple text based on upper and lower thresholds. Using a threshold filtering function, different decisions are made based on the differences between these two types of text to improve the model's understanding. This approach is more consistent with human thinking, and its effectiveness has been verified through experiments. The semantic fuzzy text screening strategy is shown in Formula 15. Represents the fuzzy category screening threshold, that is, the upper threshold of the threshold tuning strategy module; assuming that The text is semantically ambiguous text, and its loss function is expressed as ,For semantically fuzzy text that satisfies the formula, the model will adopt a re-learning method to deepen the model’s understanding of this type of text; (15) Similarly, the semantically simple text category screening threshold is the lower threshold of the threshold tuning module. In order to avoid the model fitting being too biased towards semantically simple texts, the filtered semantically simple texts will be excluded from the current d training rounds. The semantically simple text screening strategy is shown in Equation 16: (16) When the upper threshold of the threshold filter function The value is 1.10, the lower threshold When the value is 0.93, the model achieves the best accuracy improvement on the SST-2 dataset.

2. The multi-channel feature collaborative text classification method according to claim 1, characterized in that: In text classification tasks, text position information is crucial to model understanding and classification performance. The order of words and sentences in a text not only carries semantics but also reflects dependencies between contexts. Accurately capturing and utilizing this position information can help models gain a deeper understanding of the structure and meaning of text and improve their ability to discern complex semantics. Based on the importance of text location information, a multi-channel feature collaboration module is designed; The multi-channel feature collaboration module consists of two main components: a feature encoder module and a feature processing module. The feature encoder module integrates two independent encoders - the RoBERTa encoder and the RoFormer encoder. During the encoding stage, in order to obtain richer text features, the two encoders receive different input texts and extract feature representations with specific focuses.

3. The multi-channel feature collaborative text classification method according to claim 1, characterized in that: Using the label-aware data enhancement method, all labels are inserted before the sentence of the RoBERTa encoder to form a new sequence ; Using this sequence for feature extraction can obtain the potential connection between labels and sentences, and obtain text features based on label enhancement , as shown in Formula 5-6: (5) (6)。 4. The multi-channel feature collaborative text classification method according to claim 1, characterized in that: For the RoFormer encoder, its unique rotational position encoding method increases the learning of relative position information between sequence elements; since the RoBEREa encoder has already incorporated label information, the RoFormer encoder is used only to extract features of text sentences to obtain features containing relative position information. ; As shown in Formula 7-8: (7) (8) Take the characteristics With text features Perform feature fusion for splicing to form splicing features , as shown in formula 9; (9)。 5. The multi-channel feature collaborative text classification method according to claim 1, characterized in that: Considering the dimensional disaster that may be caused by long features, the concatenated feature vector is processed through the feature processing module for three dimensionality reduction; this module includes linear layer, activation function layer, Layer and layer, as shown in formula 10-14; Splicing features First, after the first linear layer, the dimension parameter is reduced from 3328 to 2024, and the output feature is obtained. Through the activation function and Enter after the layer layer, the parameter is set to 0.45 after multiple experiments; (10) (11) The dimension parameter of the second linear layer is reduced from 2024 to 1024. The layer parameter was set to 0.3 after multiple experiments; (12) (13) After the last feature processing, the features The dimension is reduced to 768, and the output features suitable for subsequent text classification tasks can be obtained. ; (14)。 6. A multi-channel feature collaborative text classification system that implements the multi-channel feature collaborative text classification method according to any one of claims 1 to 5, characterized in that: include: The multi-channel feature collaboration module consists of two main components: a feature encoder module and a feature processing module. The feature encoder module integrates two independent encoders—the RoBERTa encoder and the RoFormer encoder. During the encoding phase, to obtain richer text features, the two encoders receive different input texts and extract feature representations with specific focuses. The threshold tuning strategy module makes judgments based on upper and lower thresholds and dynamically adjusts the definition of semantically ambiguous text and semantically simple text.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the multi-channel feature collaborative text classification method as described in any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of the multi-channel feature collaborative text classification method according to any one of claims 1 to 5.

9. An information data processing terminal, comprising the multi-channel feature collaborative text classification system according to claim 6.

Images