A text classification method and system based on multi-task learning

Abstract

This invention belongs to the field of natural language processing and artificial intelligence technology, and discloses a text classification method and system based on multi-task learning. Existing text classification methods suffer from problems such as isolated task processing and inconsistent category order during training and inference in campus public opinion scenarios. This invention uses a shared encoder to extract semantic features of the text, which are then processed by a subsequent regularization module. At least two parallel task classification modules simultaneously complete topic classification and sentiment classification. This multi-task text classification model uses joint loss for optimized training and introduces a category alignment mechanism to ensure deployment stability. This invention significantly improves processing efficiency and reduces resource consumption by outputting multi-dimensional results in a single inference by a single model; it achieves joint performance optimization by utilizing task relevance; and it fundamentally avoids prediction errors through the category alignment mechanism, greatly enhancing the reliability and practicality of the system.

Description

Technical Field

[0001] This invention relates to the fields of natural language processing and artificial intelligence, specifically to a text classification method and system based on multi-task learning. Background Technology

[0002] With the rapid development of internet technology, public opinion information in the campus cyberspace has experienced explosive growth. Students express their views, share experiences, and seek help through platforms such as forums, social media, and online communities, generating a wealth of campus public opinion data. This data contains key information such as students' learning status, psychological dynamics, and campus life needs, which is of great significance for administrators to promptly grasp students' ideological dynamics, warn of potential risks, and optimize campus services.

[0003] Text classification is one of the core tasks in the field of natural language processing, aiming to quickly and accurately categorize text content into predefined categories. In the context of campus public opinion monitoring, two classification tasks need to be completed simultaneously: topic classification (such as academic-related, complaints and feedback, interpersonal social issues, etc.) and sentiment analysis (positive, negative, neutral).

[0004] Currently, the mainstream solution is to train and deploy a dedicated deep learning model independently for each analytical dimension (i.e., each classification task). For example, using one BERT model for topic classification and another BERT (Bidirectional Encoder Representations from Transformers) model for sentiment analysis. While this approach is direct, it has significant drawbacks: First, classification tasks are treated in isolation, resulting in insufficient utilization of information. Existing technologies treat topic classification and sentiment analysis as two independent tasks, requiring the training of two separate models. This not only increases the complexity and time cost of model training, but more seriously, it completely ignores the inherent connection between the two tasks. Topic information helps in sentiment judgment, and sentiment information also helps in topic classification.

[0005] Secondly, the inference efficiency is low, failing to meet real-time monitoring requirements. Because two independent models need to be run separately, the system requires two inference operations to obtain complete analysis results (topics and sentiment). In scenarios with high real-time requirements, such as campus public opinion monitoring, this serial processing method leads to a significant increase in response time, making it difficult to meet practical application needs.

[0006] Secondly, the class order is inconsistent between the training and inference phases. In actual engineering deployments, the order of class labels determined during training may differ from the configuration order loaded by the inference service. This misalignment can lead to incorrect matching between the model's output index and the preset class names, resulting in subtle systemic errors that severely compromise service reliability. Existing solutions lack a built-in, automated mechanism to ensure consistent class alignment.

[0007] Furthermore, existing multi-task learning schemes are mostly limited to simple weighted summation of loss functions, lacking fine-grained control over dynamic interactions and gradient conflicts between tasks, and thus failing to achieve true joint optimization.

[0008] Therefore, there is an urgent need for a technical solution that can efficiently, collaboratively, and stably perform multi-dimensional joint analysis of text. Summary of the Invention

[0009] The purpose of this invention is to provide a text classification method and system based on multi-task learning to solve the problems mentioned in the background art.

[0010] The main design concept of this invention is as follows: To address the aforementioned technical problems, this invention provides a text classification method and system based on multi-task learning, thereby solving the problems of low efficiency, resource waste, insufficient utilization of correlation, and poor deployment stability caused by isolated processing of multi-dimensional text analysis tasks in the prior art.

[0011] To achieve the above objectives, the present invention provides the following technical solution: In a first aspect, the present invention provides a text classification method based on multi-task learning, comprising the following steps: Obtain the original text to be classified; The original text is preprocessed to obtain normalized text; Normalized text is input into a trained multi-task text classification model for multi-task text classification processing; The multi-task text classification model includes a shared encoder, a regularization module, and at least two parallel task classification modules; The shared encoder is used to perform semantic encoding on the input normalized text and output a shared feature vector; The regularization module is used to receive the shared feature vector and perform random deactivation processing on the shared feature vector during the training phase of the multi-task text classification model to obtain a regularized feature vector. The task classification module is used to receive regularized feature vectors and classify them into text, so as to output the corresponding classification results; Output at least two classification results generated by the multi-task text classification model.

[0012] More preferably, the shared encoder is a BERT pre-trained language model based on the Transformer architecture.

[0013] More preferably, the regularization module is a shared Dropout module, used to randomly discard a portion of the data in the shared feature vector with a preset probability during the training phase of the multi-task text classification model.

[0014] More preferably, the at least two parallel task classification modules include a topic classification head and an emotion classification head; The categories corresponding to the topic category headers include: academic related, venting and feedback, hobbies and interests, resource trading, interpersonal social interaction, mutual assistance and help, life and entertainment, campus interaction and events; The sentiment classification head corresponds to the following categories: positive, negative, and neutral.

[0015] More preferably, the training process of the multi-task text classification model is as follows: Calculate the loss function for each task classification module; The loss functions are weighted and fused to obtain the joint loss; Based on the joint loss, the parameters of the multi-task text classification model are updated using the backpropagation algorithm.

[0016] More preferably, a category alignment step is included before each task classification module outputs the corresponding classification result. The category alignment step includes: When training a multi-task text classification model, record the first category order list corresponding to the output layer of each task classification module; When performing multi-task text classification using a multi-task text classification model, obtain the second category order list corresponding to the output layer of each task classification module in the running system configuration of the multi-task text classification model. Compare the first category order list with the second category order list; If the comparison results are consistent, the classification result of the task classification module is directly output; if the comparison results are inconsistent, the classification result is remapped according to the comparison results so that the final output classification result is consistent with the second category order list.

[0017] Secondly, the present invention provides a text classification system based on multi-task learning, comprising the following modules: The input module is used to obtain the original text to be classified; The preprocessing module is used to preprocess the original text to obtain normalized text; The multi-task text classification module is used to input normalized text into a trained multi-task text classification model for multi-task text classification processing. The multi-task text classification model includes: A shared encoder is used to semantically encode the input normalized text and output a shared feature vector. The regularization module is used to receive the shared feature vector and perform random deactivation processing on the shared feature vector during the training phase of the multi-task text classification model to obtain a regularized feature vector. At least two parallel task classification modules are provided, wherein the task classification modules are used to receive regularized feature vectors and classify them into text, so as to output the corresponding classification results; The output module is used to output at least two classification results generated by the multi-task text classification model.

[0018] Thirdly, the present invention provides an electronic device including a processor and a memory, wherein the memory stores computer instructions, and when the computer instructions are executed by the processor, the electronic device performs the text classification method based on multi-task learning.

[0019] Fourthly, the present invention provides a computer program product containing computer-executable instructions, which, when executed, implement the text classification method based on multi-task learning.

[0020] Fifthly, the present invention provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the text classification method based on multi-task learning.

[0021] Compared with the prior art, the beneficial effects of the present invention are: 1. This invention utilizes a multi-task text classification model with a shared encoder and parallel task heads to integrate multi-dimensional classification tasks that originally required multiple independent models for serial processing into a unified model for one-time parallel inference. The number of inferences is reduced from 2 to 1, and the number of parameters is reduced from 220M to 110M, resulting in a 50% reduction in system response time. This also reduces model storage and runtime overhead, significantly improves processing throughput, and meets the real-time requirements of campus public opinion monitoring scenarios.

[0022] 2. This invention achieves deep joint optimization of topic classification and sentiment classification. The model learns within a unified shared feature space, enabling intrinsically related tasks to mutually reinforce each other. Experiments show that the topic classification performance (F1 score 97.97%) is essentially on par with the single-task model (98.34%), while the sentiment classification F1 score improves from 99.76% to 99.89%. This demonstrates that the invention effectively utilizes the correlation between tasks, enhancing the model's overall discriminative ability.

[0023] 3. This invention innovatively introduces a class alignment mechanism, solving the prediction error problem caused by inconsistencies in class order between the training and application environments of machine learning models. This mechanism ensures accurate correspondence between prediction results and preset classes by persisting the class order during training and automatically detecting, comparing, and remapping during application. This significantly improves system stability, avoids model failures due to configuration errors, and greatly enhances the system's robustness and maintainability.

[0024] 4. This invention constructs a dedicated multi-task text classification dataset covering eight campus theme categories and three sentiment categories. This dataset provides a high-quality, targeted training and evaluation foundation for campus public opinion analysis. Furthermore, the entire technical solution is closely integrated with the needs of campus public opinion analysis, forming a complete and practical solution with strong applicability. Attached Figure Description

[0025] Figure 1 This is a flowchart of the text classification method based on multi-task learning of the present invention; Figure 2 This is a block diagram of the text classification system based on multi-task learning according to the present invention; Figure 3 This is a block diagram of the multi-task text classification model of the present invention; Figure 4 This is a flowchart of the category alignment mechanism of the present invention. Detailed Implementation

[0026] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0027] In the description of this invention, it should be noted that the terms "upper," "lower," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Furthermore, the terms "installation" and "connection" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0028] Example 1 like Figure 1 and Figure 2 As shown, this embodiment provides a text classification method based on multi-task learning, including the following steps: Obtain the original text to be classified; The original text is preprocessed to obtain normalized text; Normalized text is input into a trained multi-task text classification model for multi-task text classification processing; The multi-task text classification model includes a shared encoder, a regularization module, and at least two parallel task classification modules; The shared encoder is used to perform semantic encoding on the input normalized text and output a shared feature vector; The regularization module is used to receive the shared feature vector and perform random deactivation processing on the shared feature vector during the training phase of the multi-task text classification model to obtain a regularized feature vector. The task classification module is used to receive regularized feature vectors and classify them into text, so as to output the corresponding classification results; Output at least two classification results generated by the multi-task text classification model.

[0029] This method can be implemented on a local server or a cloud server to process text data in scenarios such as campus forums and social media, and specifically includes the following processes.

[0030] S10: Obtain the original text to be classified. Specifically, the original text can come from web page content crawled in real time by web crawlers, user-submitted content received by application programming interfaces (APIs), historical text data stored in databases, etc. The text language is primarily Chinese, but this method is also applicable to other languages, such as English.

[0031] S11: Preprocess the original text The acquired raw text is preprocessed to obtain a normalized text representation. Preprocessing aims to remove noise, standardize formatting, and transform unstructured text into a numerical representation that the model can process. Specific implementation includes: 1. Text Cleaning: Remove HTML tags, URL links, special characters (such as @, #), emojis, or irrelevant advertising information from the text. For Chinese characters, it may also be necessary to convert full-width characters to half-width characters. This can be achieved using regular expressions (Regex).

[0032] 2. Word Segmentation: Split continuous sentences into independent word sequences. For Chinese, Chinese word segmentation tools such as Jieba, HanLP or LTP are required. For BERT-based models, its accompanying WordPiece or BERTTokenizer is usually used for subword tokenization, which can better handle out-of-vocabulary words.

[0033] 3. Stop Word Filtering: Remove common function words that contribute little to classification, such as "de", "le", "ye", etc. A predefined stop word list can be loaded for filtering. However, in context-based pre-trained models (BERT), this step can sometimes be omitted because the model can learn the contextual meaning of these words.

[0034] 4. Vectorization: Convert the tokenized sequence into the normalized text format required for model input, i.e., token sequence, attention mask, segment identifier, etc. This step is achieved by calling the encoding (encode or encode_plus) method of the tokenizer.

[0035] S12: Classify the cleaned original text using a multi-task text classification model Input the normalized text representation into a pre-trained multi-task text classification model. The multi-task classification model is the core of this invention, and its structure is as Figure 3 shown. The model includes a shared encoder, a shared Dropout module, and two parallel task classification heads (topic classification head and sentiment classification head).

[0036] Shared Encoder: Its function is to extract deep, context-related semantic features of the input normalized text. Preferably, the shared encoder uses the BERT pre-trained language model based on the Transformer architecture.

[0037] In this embodiment, the bert-base-chinese model is used. The shared encoder receives the token sequence, attention mask, etc. generated in the previous step as input, and after 12 layers of TransformerEncoder calculations, outputs a 768-dimensional feature sequence containing rich semantic information.

[0038] Generally, take the final hidden state corresponding to the first special token (a 768-dimensional vector for bert-base-chinese) as the representation vector of the entire sentence, called the shared feature vector H. The parameters of this encoder are shared among multiple tasks, enabling different tasks to learn in a unified semantic space.

[0039] The shared dropout module is connected after the shared encoder. During model training, it works by randomly dropping a certain proportion (e.g., 0.3, or 30%) of the dimension of the input feature vector during the forward propagation. Its output is a randomized shared feature vector H. This step is a powerful regularization technique that serves the following purposes: To prevent overfitting: By randomly discarding some feature dimensions, each subsequent task classification head is forced to avoid over-reliance on a few fixed feature dimensions and instead learn to make judgments from more robust and diverse feature combinations. This enhances the model's resistance to noise in the training data and its generalization ability.

[0040] Facilitating feature sharing between tasks: Since all task heads receive the same shared feature vector H processed by the Dropout module, this regularization constraint is applied globally, which helps to learn more fundamental feature representations that are universal to multiple tasks, thereby better realizing knowledge transfer and joint optimization between tasks.

[0041] During the model inference / testing phase, the Dropout module is turned off, and all feature dimensions are involved in the calculation, but the expected activation values ​​that are reduced due to dropping during training are usually compensated by scaling the weights.

[0042] Task classification heads: The shared feature vector H, processed by the shared Dropout module, is simultaneously fed to all parallel task classification heads. Each classification head is a relatively lightweight neural network module responsible for completing its specific classification task based on the shared feature vector H.

[0043] Each classification head typically consists of one or more fully connected linear layers and a softmax activation function. This invention relates to topic classification heads and sentiment classification heads.

[0044] Topic Classification Header: For the campus public opinion scenario in this embodiment, its output dimension is 8, corresponding to 8 preset topic categories. Its structure can be designed as: Linear(768→512)→ReLU()→Dropout(0.3)→Linear(512→8). Finally, an 8-dimensional logits vector is output, and after Softmax, the probability distribution of each topic category is obtained.

[0045] Sentiment classification head: Its output dimension is 3, corresponding to positive, negative, and neutral sentiment. Its structure can be designed as: Linear(768→512)→ReLU()→Dropout(0.3)→Linear(512→3). The final output is a 3-dimensional logits vector, which is then processed by Softmax to obtain the sentiment probability distribution.

[0046] S13: Output the classification results generated by the multi-task text classification model.

[0047] The feature vector H output by the shared encoder is simultaneously fed into both the topic classification head and the sentiment classification head. The two classification heads are computed in parallel, producing topic classification results and sentiment classification results respectively. The classification results can be probability distributions or category indices obtained by taking the maximum args.

[0048] For example, given the input text, the model might simultaneously output: Topic = "complaint feedback" (index 1, probability 0.85) and Sentiment = "negative" (index 0, probability 0.92).

[0049] The topic classification and sentiment classification results will be output in the form of structured data (such as JSON). For example: {"text":"...","theme":{"category":"complaints and feedback","confidence":0.85}","sentiment":{"polarity":"negative","confidence":0.92}}. This result can be used for public opinion dashboards, risk warnings, or further data analysis.

[0050] Example 2 Based on Example 1, this example elaborates on the training and joint optimization process of the multi-task text classification model, such as... Figure 3 As shown. Model training requires a multi-label dataset where each sample contains the original text, topic label, and sentiment label. The specific steps include:

[0051] S20: Data preparation and model initialization.

[0052] Load and split the training, validation, and test sets. Initialize 8 sentiment topic categories and 3 sentiment categories. Load the pre-trained BERT encoder weights (bert-base-chinese) and randomly initialize the weights of the two task classification heads. Define the optimizer (e.g., AdamW) and the learning rate scheduler.

[0053] The pseudocode for model initialization is shown below: model=MultiTaskBERT( n_sentiment_classes=3, n_theme_classes=8, dropout_rate=0.3, freeze_layers=8 ) S21: Forward Propagation Take a batch of data, preprocess it, and then input it into the model. The model performs forward propagation: the text passes through a shared encoder to obtain a shared feature vector H, and H then passes through a topic classification head and a sentiment classification head to obtain topic classification logits and sentiment classification logits, respectively.

[0054] S22: Calculate task loss Calculate the loss functions for topic classification and sentiment classification separately. For classification tasks, cross-entropy loss is typically used.

[0055] The topic classification loss function is L_theme=CrossEntropy(theme_logits,theme_labels), where theme_labels represents the true labels of the topic.

[0056] The sentiment classification loss function is L_sentiment=CrossEntropy(sentiment_logits, sentiment_labels), where sentiment_labels represents the true labels of sentiment.

[0057] The core example pseudocode for calculating task loss is as follows: theme_criterion=nn.CrossEntropyLoss(weight=theme_weights) sentiment_criterion=nn.CrossEntropyLoss(weight=sentiment_weights) S23: Calculate the joint loss The losses from multiple tasks are combined into a joint loss to guide the overall optimization of the model. This embodiment uses a weighted summation method to calculate the joint loss, namely: Where L_total represents the joint loss, and λ1 and λ2 represent hyperparameters used to control the degree of influence of different tasks on model updates. In this embodiment, λ1=λ2=1.0 is set, meaning that the two classification tasks are considered equally important. Preferably, a dynamic weighting strategy can be used to better balance multi-task learning.

[0058] The core example pseudocode for joint loss is as follows: theme_loss=theme_criterion(theme_logits,theme_labels) sentiment_loss=sentiment_criterion(sentiment_logits,sentiment_labels) total_loss=theme_loss+sentiment_loss S24: Backpropagation and Parameter Update Based on the joint loss L_total, the backpropagation algorithm is executed to calculate the gradient of the loss with respect to all model parameters (including the parameters of the shared encoder and the two classifier heads). Then, the optimizer uses these gradients to update all parameters.

[0059] This step is crucial for achieving "joint optimization" and "knowledge sharing." The gradient of the shared encoder is jointly determined by the gradients of the two tasks, thus forcing it to learn general feature representations useful for both tasks. The two classification heads, in turn, focus on learning the task-specific patterns of their respective tasks.

[0060] Its core example pseudocode is as follows: optimizer.zero_grad() total_loss.backward() optimizer.step() S25: Iterative Training and Evaluation Repeat steps S20 to S24 above until the model's performance on the validation set stabilizes or reaches the preset number of training epochs. After training is complete, evaluate the model's final performance on the test set.

[0061] Example 3 Building upon Examples 1 and 2, this example elaborates on the specific implementation of the category alignment mechanism, such as... Figure 4 As shown. This mechanism aims to resolve prediction errors caused by inconsistent class order during model deployment and is crucial for ensuring application stability.

[0062] S30: Training phase, save the category encoder. Before model training begins, a list of class orders is defined for each classification task. The order of this list determines the correspondence between neurons in the model's output layer and the class labels.

[0063] List of topic categories (training order): ['Academic related', 'Complaints and Feedback', 'Hobbies', 'Resource Trading', 'Interpersonal Social Interactions', 'Mutual Aid and Assistance', 'Life and Entertainment', 'Campus Interactions and Events',].

[0064] List of sentiment categories (training order): ['positive', 'negative', 'neutral'].

[0065] During training, the i-th neuron in the model's output layer corresponds to the i-th category in the list above. After training is complete, these two category lists and their order must be persistently saved. This example uses Python's pickle or joblib library to save the category list to a file.

[0066] Alternatively, you can use sklearn's LabelEncoder to fit the training labels and save the entire encoder object, as it records the category information internally.

[0067] S31: Inference phase, loading the category encoder When deploying a model for actual inference applications, engineers may rearrange the category order list for each classification task to make the category list more organized, forming a new category order list (expected classification order). This new category order list is then configured as an application variable for the application environment so that the new category order list for each classification task can be quickly loaded before each model is loaded for inference or when the service starts.

[0068] S32: Detection Sequence Consistency and Remapping Before the task classification module outputs the original classification results, it first checks whether the class order list from the model training phase corresponds to the class order list currently configured in the system. If they match, the classification result from the task module is directly output. If they do not match, remapping is performed based on the comparison results, the probability distribution vector is rearranged, or the class indices are transformed to match the new class order list. This mechanism ensures that the model's final output can be correctly associated with the expected category name regardless of how the category configuration order changes in the application environment, greatly improving the system's robustness.

[0069] The pseudocode example for category alignment is as follows: defrealign_predictions(theme_probs,theme_encoder): #Get the class order during training training_classes=list(theme_encoder.classes_) #Get the current preset category order current_classes=categories #Check if alignment is needed iftraining_classes!=current_classes: # Create mapping relationship mapping={} fori,train_classinenumerate(training_classes): forj,current_classinenumerate(current_classes): iftrain_class==current_class: mapping[i]=j #Realign prediction results realigned_probs=np.zeros_like(theme_probs) fortrain_idx,current_idxinmapping.items(): realigned_probs[current_idx]=theme_probs[train_idx] returnrealigned_probs returntheme_probs

[0070] Example 4 This embodiment provides a specific application example of the method in a campus public opinion monitoring scenario, and supplements the details of dataset construction and ablation experiment.

[0071] S40: Building the dataset To train and evaluate the multi-task text classification model of this invention, a dedicated campus public opinion multi-task text classification dataset was constructed. The data was obtained from campus forum crawling, screening of publicly available public opinion datasets, and manual annotation, totaling 4238 high-quality annotated data entries. Specific categories and sample numbers are described below.

[0072] Theme categories (8 categories): Academic related (991 items), Feedback and complaints (714 items), Hobbies and interests (583 items), Resource trading (534 items), Interpersonal social interaction (504 items), Mutual assistance and help (349 items), Life and entertainment (313 items), Campus interaction and events (250 items).

[0073] Sentiment categories (3): positive (1767 items), negative (1238 items), neutral (1233 items).

[0074] The dataset is divided into a training set (3051 records), a validation set (339 records), and a test set (848 records) in a ratio of 7:1:2.

[0075] S41: Model Training and Related Parameter Explanation Shared encoder: bert-base-chinese (12 layers, 768 hidden dimensions, 12 heads).

[0076] Classification head: Single-layer fully connected layer, 768-dimensional input, outputs are 8-dimensional (topic) and 3-dimensional (sentiment). Dropout rate is 0.1.

[0077] Optimizer: AdamW, initial learning rate 2e-5, linear decay after linear warm-up.

[0078] Batch size: 16.

[0079] Training epochs: 10 epochs.

[0080] Loss weights: λ1=λ2=1.0.

[0081] Hardware: Single NVIDIA Tesla V100 16GB card.

[0082] After three training epochs, the model achieves optimal performance on the validation set: Based on the above model training and related parameter settings, after three rounds of training on the training set, the multi-task text classification model achieved optimal performance on the validation set. Its training convergence results are shown in Table 1 below.

[0083] Table 1: Convergence Results of Model Training in This Invention As shown in Table 1, the joint loss value continuously decreased with the increase of training epochs (from 2.3456 to 0.9876), while the performance of both classification tasks improved rapidly and tended to stabilize. Specifically, the F1 score for topic classification increased from 86.5% to 97.97%, and the F1 score for sentiment classification increased from 92.3% to 99.89%. This data intuitively demonstrates the effectiveness and rapid convergence of the training method of the multi-task text classification model of this invention, showing that the model can achieve excellent performance in a relatively small number of training epochs.

[0084] S42: Experimental Results The performance evaluation results on the test set are shown in Table 2 below: Table 2: Performance Evaluation Experiment Results Based on the trained model, the average time for a single inference is 0.31 seconds, simultaneously outputting topic and sentiment results, which is twice as efficient as existing technologies (approximately 0.62 seconds for two inferences). The category alignment mechanism avoids prediction errors caused by inconsistent category order during training and inference, improving system stability by 1.73 percentage points and ensuring the reliability of the model in practical deployment.

[0085] Furthermore, the accuracy rate for negative sentiment reaches 100%, for neutral sentiment it reaches 99.8%, and for positive sentiment it reaches 99.86%. In addition, this invention merges two independent models into a single multi-task classification model, reducing the number of parameters from approximately 220M to 110M, a reduction of about 50%. This not only reduces model storage requirements but also reduces memory usage during inference.

[0086] S43: Comparative Experiment Furthermore, a comparative experiment was conducted on the test set between the multi-task text classification model of the present invention and existing models, and the results of the technical performance comparison are shown in Table 3 below.

[0087] Table 3: Comparison Results of Technological Effects in Comparative Experiments As can be seen from Table 3, the core advantage of this invention lies in the fact that while maintaining high performance in topic classification (F1 value 97.97% vs 98.34%) and training efficiency (147.4 minutes vs 147.6 minutes), it achieves significant optimization in multiple dimensions.

[0088] Specifically, this manifests in the following ways: improved sentiment classification performance; doubled inference efficiency (reducing the number of inferences from 2 to 1); and halved model complexity (consolidating the number of models from 2 to 1, with a corresponding reduction in the number of parameters). This demonstrates that the present invention, through a multi-task learning architecture, successfully achieves a doubling of efficiency and resource efficiency with almost no loss of accuracy, resulting in excellent overall performance.

[0089] S44: Ablation Experiment To verify the effectiveness of the technical solution of the present invention, an ablation experiment was designed, and the results of the ablation experiment are shown in Table 4 below.

[0090] Table 4: Ablation Experiment Results As shown in Table 4, removing multi-task learning requires training two separate single-task models, doubling the number of inferences while offering only a slight performance improvement, demonstrating the engineering value of multi-task learning. Removing pre-training fine-tuning keeps model performance stable, but training time decreases by 7.9%, indicating that this dataset has distinct features, and pre-training knowledge contributes only a limited amount. Removing category alignment reduces topic classification performance by 0.74 percentage points, highlighting the importance of the alignment mechanism.

[0091] Example 5 Based on the text classification method based on multi-task learning in Embodiment 1, this embodiment provides a text classification system based on multi-task learning, including the following modules: The input module is used to obtain the original text to be classified; The preprocessing module is used to preprocess the original text to obtain normalized text; The multi-task text classification module is used to input normalized text into a trained multi-task text classification model for multi-task text classification processing. The multi-task text classification model includes: A shared encoder is used to semantically encode the input normalized text and output a shared feature vector. The regularization module is used to receive the shared feature vector and perform random deactivation processing on the shared feature vector during the training phase of the multi-task text classification model to obtain a regularized feature vector. At least two parallel task classification modules are provided, wherein the task classification modules are used to receive regularized feature vectors and classify them into text, so as to output the corresponding classification results; The output module is used to output at least two classification results generated by the multi-task text classification model.

[0092] Example 6 This embodiment also provides an electronic device. The electronic device includes a bus, a processor, a memory, and a communication interface. The processor, memory, and communication interface communicate with each other via the bus. The computing device can be a server or a terminal device. It should be understood that this application does not limit the number of processors and memories in the computing device.

[0093] A bus can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, a bus can include a path for transmitting information between various components of a computing device (e.g., memory, processor, communication interfaces).

[0094] Processors include any one or more of the following: central processing unit (CPU), graphics processing unit (GPU), tensor processing unit (TPU), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), microprocessor (MP), or digital signal processor (DSP).

[0095] The memory may include volatile memory, such as random access memory (RAM). The processor may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD).

[0096] The memory stores executable program code, which the processor executes to implement the functions of the aforementioned units, thereby implementing the method described in Embodiment 1, etc. That is, the memory may store instructions for the methods and functions relating to the computing device in any of the above embodiments.

[0097] The communication interface uses transceiver modules such as, but not limited to, network interface cards and transceivers to enable communication between computing devices and other devices or communication networks.

[0098] Example 7 This embodiment provides at least one computer program product tangibly stored on a non-transitory computer-readable storage medium. The computer program product includes computer-executable instructions, such as instructions included in program modules, which execute in a device on a target real or virtual processor to perform the processes / methods as described above with reference to the accompanying drawings. Typically, program modules include routines, programs, libraries, objects, classes, components, data structures, etc., that perform specific tasks or implement specific abstract data types. In various embodiments, the functionality of program modules can be combined or divided among program modules as needed. The machine-executable instructions for the program modules can execute within a local or distributed device. In a distributed device, the program modules can reside in both local and remote storage media.

[0099] Computer program code used to implement the methods of this disclosure may be written in one or more programming languages. This computer program code may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus, such that when executed by the computer or other programmable data processing apparatus, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be performed. The program code may be executed entirely on a computer, partially on a computer, as a stand-alone software package, partially on a computer and partially on a remote computer, or entirely on a remote computer or server.

[0100] In the context of this disclosure, computer program code or related data may be carried on any suitable carrier to enable a device, apparatus, or processor to perform the various processes and operations described above. Examples of carriers include signals, computer-readable media, and so on. Examples of signals may include electrical, optical, radio, sound, or other forms of propagation signals, such as carrier waves, infrared signals, etc.

[0101] Computer-readable media can be any tangible medium that contains or stores programs for or relating to an instruction execution system, apparatus, or device, or a data storage device such as a data center containing one or more available media. Computer-readable media can be computer-readable signal media or computer-readable storage media. Computer-readable media can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. More detailed examples of computer-readable storage media include electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0102] Example 8 This embodiment provides a computer-readable storage medium storing computer instructions that, when executed by a processor, cause the processor to perform the methods and functions of the computing device involved in any of the above embodiments.

[0103] Generally, the various embodiments of this disclosure can be implemented in hardware or dedicated circuitry, software, logic, or any combination thereof. Some aspects can be implemented in hardware, while others can be implemented in firmware or software, which can be executed by a controller, microprocessor, or other computing device. Although various aspects of the embodiments of this disclosure are shown and described as block diagrams, flowcharts, or represented using some other illustration, it should be understood that the blocks, apparatuses, systems, techniques, or methods described herein can be implemented as, as non-limiting examples, in hardware, software, firmware, dedicated circuitry or logic, general-purpose hardware or controllers or other computing devices, or some combination thereof.

[0104] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered illustrative and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention, and no reference numerals in the claims should be construed as limiting the scope of the claims.

[0105] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A text classification method based on multi-task learning, characterized in that, Includes the following steps: Obtain the original text to be classified; The original text is preprocessed to obtain normalized text; Normalized text is input into a trained multi-task text classification model for multi-task text classification processing; The multi-task text classification model includes a shared encoder, a regularization module, and at least two parallel task classification modules; The shared encoder is used to perform semantic encoding on the input normalized text and output a shared feature vector; The regularization module is used to receive the shared feature vector and perform random deactivation processing on the shared feature vector during the training phase of the multi-task text classification model to obtain a regularized feature vector. The task classification module is used to receive regularized feature vectors and classify them into text, so as to output the corresponding classification results; Output at least two classification results generated by the multi-task text classification model.

2. The text classification method based on multi-task learning according to claim 1, characterized in that, The shared encoder is a BERT pre-trained language model based on the Transformer architecture.

3. The text classification method based on multi-task learning according to claim 2, characterized in that, The regularization module is a shared Dropout module, which is used to randomly discard a portion of the data in the shared feature vector with a preset probability during the training phase of the multi-task text classification model.

4. The text classification method based on multi-task learning according to claim 1, characterized in that, The at least two parallel task classification modules include a topic classification head and an emotion classification head; The categories corresponding to the topic category headers include: academic related, venting and feedback, hobbies and interests, resource trading, interpersonal social interaction, mutual assistance and help, life and entertainment, campus interaction and events; The sentiment classification head corresponds to the following categories: positive, negative, and neutral.

5. The text classification method based on multi-task learning according to claim 1, characterized in that, The training process of the multi-task text classification model is as follows: Calculate the loss function for each task classification module; The loss functions are weighted and fused to obtain the joint loss; Based on the joint loss, the parameters of the multi-task text classification model are updated using the backpropagation algorithm.

6. The text classification method based on multi-task learning according to claim 1, characterized in that, Before each task classification module outputs its corresponding classification result, a category alignment step is included, which includes: When training a multi-task text classification model, record the first category order list corresponding to the output layer of each task classification module; When performing multi-task text classification using a multi-task text classification model, obtain the second category order list corresponding to the output layer of each task classification module in the running system configuration of the multi-task text classification model. Compare the first category order list with the second category order list; If the comparison results are consistent, the classification result of the task classification module is directly output; if the comparison results are inconsistent, the classification result is remapped according to the comparison results so that the final output classification result is consistent with the second category order list.

7. A text classification system based on multi-task learning, characterized in that, Includes the following modules: The input module is used to obtain the original text to be classified; The preprocessing module is used to preprocess the original text to obtain normalized text; The multi-task text classification module is used to input normalized text into a trained multi-task text classification model for multi-task text classification processing. The multi-task text classification model includes: A shared encoder is used to semantically encode the input normalized text and output a shared feature vector. The regularization module is used to receive the shared feature vector and perform random deactivation processing on the shared feature vector during the training phase of the multi-task text classification model to obtain a regularized feature vector. At least two parallel task classification modules are provided, wherein the task classification modules are used to receive regularized feature vectors and classify them into text, so as to output the corresponding classification results; The output module is used to output at least two classification results generated by the multi-task text classification model.

8. An electronic device, characterized in that, It includes a processor and a memory, the memory storing computer instructions that, when executed by the processor, cause the electronic device to perform the method of any one of claims 1 to 6.

9. A computer program product, characterized in that, The computer program product includes computer-executable instructions that, when executed, implement the method of any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions that, when executed by a processor, implement the method of any one of claims 1 to 6.

Images