Text classification method and device, computer device, storage medium and program product

Abstract

The application relates to the technical field of natural language processing, and discloses a text classification method and device, computer equipment, a storage medium and a program product. The text classification method comprises the following steps: predicting a target entity in a text to be classified based on a first prediction model to obtain a first classification result of a classification to which the target entity belongs; extracting an aspect category corresponding to the target entity based on a second prediction model, and respectively predicting based on the aspect category to obtain a second classification result; and determining a classification result corresponding to the text to be classified based on the first classification result and the second classification result. The application can simultaneously complete coarse-grained element classification prediction at a target entity level and fine-grained requirement classification prediction at an aspect entity level, realizes comprehensive identification and prediction of elements at different levels and different granularities, provides more comprehensive and accurate results for element analysis, improves the accuracy of classification prediction while improving the flexibility of the model, and meets the use requirements of users.

Description

Technical Field

[0001] This invention relates to the field of natural language processing technology, specifically to text classification methods, apparatus, computer equipment, storage media, and program products. Background Technology

[0002] Feature classification has always been a fundamental task in machine learning. By designing and using computer algorithms to determine the category of features, it has been widely applied in fields such as news classification, sentiment classification, topic classification, image classification, video classification, ad filtering, content moderation, comment analysis, question-and-answer, natural language processing, data mining, recommendation systems, and advertising systems. This massive amount of data contains netizens' opinions, attitudes, and decisions regarding various issues, including evaluations and viewpoints on trending events, products, and individuals. Mining and analyzing this data can yield useful information. For example, analyzing user reviews of a product helps providers understand consumer psychology, thereby improving product quality and service. It also helps other consumers understand the true nature of products, providing a reference for their purchasing decisions.

[0003] However, in relevant text classification schemes, features often need to be constructed manually. The final classification effect depends to a large extent on the quality of feature engineering and human prior knowledge, resulting in poor model flexibility of traditional methods and classification results that cannot meet the user's needs. Summary of the Invention

[0004] In view of this, the present invention provides a text classification method, apparatus, computer device, storage medium and program product to solve the problem that the classification model of traditional methods has poor flexibility and cannot meet the user's needs.

[0005] In a first aspect, the present invention provides a text classification method, the method comprising:

[0006] Based on the first prediction model, the target entity in the text to be classified is predicted, and the first classification result of the target entity is obtained.

[0007] The second prediction model extracts the aspect categories corresponding to the target entity, and predictions are made based on the aspect categories to obtain the second classification result. The aspect category is a predefined attribute category used to evaluate the target entity.

[0008] Based on the first and second classification results, the classification result corresponding to the text to be classified is determined.

[0009] In one alternative implementation, the second prediction model includes: a first interactive attention mechanism;

[0010] The second prediction model extracts the aspect categories corresponding to the target entity, and predictions are made based on each aspect category to obtain the second classification results, including:

[0011] Get the text to be categorized;

[0012] Based on a pre-trained model, the aspect categories of the text to be classified are identified;

[0013] The text to be classified is identified based on the start identifier and the end identifier to obtain the first identification result. The first identification result is then vectorized using a preset matrix to obtain the text vector.

[0014] The aspect categories are identified based on the start identifier and the end identifier to obtain the second identification result. The second identification result is then vectorized using a preset matrix to obtain the category vector.

[0015] Based on aspect categories, sub-tasks are created;

[0016] Based on the first interaction attention mechanism, target sentiment word vectors that match the aspect categories of the subtask are extracted from the text vectors;

[0017] The sub-tasks are analyzed based on the target sentiment word vectors to obtain the sub-classification results of the sub-tasks;

[0018] The second classification result is determined based on the subclassification results.

[0019] In this embodiment of the invention, a second prediction model can be used to perform fine-grained element classification prediction of the aspect type corresponding to the target entity in the text to be classified, thereby providing a technical basis for realizing multi-level and multi-granularity analysis of the target entity.

[0020] In one optional implementation, determining the second classification result based on the sub-classification result includes:

[0021] Analyze the weights of subtasks based on their aspect categories;

[0022] The subclassification results are weighted and summed according to their weights to obtain the second classification result.

[0023] In this embodiment of the invention, the sub-classification results of the sub-tasks corresponding to each aspect category can be weighted and summed, thereby making full use of the dataset of fine-grained element analysis in complex contexts and improving the accuracy of fine-grained classification results.

[0024] In one optional implementation, based on a first interactive attention mechanism, target sentiment word vectors that match the aspect category of the subtask are extracted from the text vector, including:

[0025] Based on the first interaction attention mechanism, extract the category features of aspect categories;

[0026] Extract sentence features that match the category features from the text vectors, and determine the target sentiment word vectors based on the sentence features.

[0027] In this embodiment of the invention, a second prediction model can be used to perform fine-grained element classification prediction of the aspect type corresponding to the target entity in the text to be classified, thereby providing a technical basis for realizing multi-level and multi-granularity analysis of the target entity.

[0028] In one alternative implementation, the first prediction model includes: a self-attention mechanism and a second interactive attention mechanism;

[0029] Based on the first prediction model, the target entity in the text to be classified is predicted, and the first classification result of the target entity's category is obtained, including:

[0030] Based on the self-attention mechanism, semantic features are extracted from the text to be classified, and the target entity is determined based on the semantic features;

[0031] Based on the second interactive attention mechanism, the first classification result of the target entity is predicted.

[0032] In this embodiment of the invention, a first prediction model can be used to perform coarse-grained element classification prediction of target entities in the text to be classified, thereby providing a technical basis for realizing multi-level and multi-granularity analysis of target entities.

[0033] In one alternative implementation, the first prediction model includes a multi-head attention mechanism;

[0034] Based on the second interactive attention mechanism, the first classification result of the target entity's category is predicted, including:

[0035] When the target entity contains multiple sub-entities, the target features of each sub-entity are extracted through a second interactive attention mechanism;

[0036] Based on the multi-head attention mechanism, the classification result of each target feature is predicted to obtain the first classification result.

[0037] In this embodiment of the invention, a first prediction model can be used to perform coarse-grained element classification prediction of target entities in the text to be classified, thereby providing a technical basis for realizing multi-level and multi-granularity analysis of target entities.

[0038] In one alternative implementation, the method further includes:

[0039] The first loss function corresponding to the first prediction model and the second loss function corresponding to the second prediction model are determined based on the cross-entropy loss function.

[0040] Based on the first loss function and the second loss function, a target loss function is constructed, which is used to guide the optimization of classification results.

[0041] In this embodiment of the invention, since the above classification prediction model includes a first prediction model and a second prediction model, the loss functions of the two models can be calculated separately, and the target loss function can be constructed by comprehensively considering the first loss function and the second loss function. This allows for better training of the classification prediction model, measurement of model performance, and optimization of model parameters, thereby improving the accuracy of the output results of the classification prediction model.

[0042] Secondly, the present invention provides a text classification device, the device comprising:

[0043] The first prediction module is used to predict the target entity in the text to be classified based on the first prediction model, and to obtain the first classification result of the target entity's category.

[0044] The second prediction module is used to extract the aspect categories corresponding to the target entity based on the second prediction model, and to make predictions based on the aspect categories to obtain the second classification result. The aspect categories are predefined attribute categories used to evaluate the target entity.

[0045] The determination module is used to determine the classification result corresponding to the text to be classified based on the first classification result and the second classification result.

[0046] Thirdly, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the computer instructions to perform the text classification method of the first aspect or any corresponding embodiment described above.

[0047] Fourthly, the present invention provides a computer-readable storage medium storing computer instructions for causing a computer to perform the text classification method of the first aspect or any corresponding embodiment thereof.

[0048] Fifthly, the present invention provides a computer program product, including computer instructions for causing a computer to execute the text classification method of the first aspect or any corresponding embodiment thereof. Attached Figure Description

[0049] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0050] Figure 1 This is a schematic diagram illustrating element classification based on dictionary and rule-based methods;

[0051] Figure 2 This is a schematic diagram of feature classification based on machine learning methods;

[0052] Figure 3 This is a flowchart illustrating a text classification method according to an embodiment of the present invention;

[0053] Figure 4 This is a flowchart illustrating another text classification method according to an embodiment of the present invention;

[0054] Figure 5 This is a schematic diagram of a classification prediction model;

[0055] Figure 6 This is a flowchart illustrating another text classification method according to an embodiment of the present invention;

[0056] Figure 7 This is a structural block diagram of a text classification device according to an embodiment of the present invention;

[0057] Figure 8 This is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. Detailed Implementation

[0058] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0059] The application scenarios on which the text classification method depends are described here.

[0060] Feature classification has always been a fundamental task in machine learning. By designing and using computer algorithms to determine the category of features, it has been widely applied in fields such as news classification, sentiment classification, topic classification, image classification, video classification, ad filtering, content moderation, comment analysis, question-and-answer, natural language processing, data mining, recommendation systems, and advertising systems. This massive amount of data contains netizens' opinions, attitudes, and decisions regarding various issues, including evaluations and viewpoints on trending events, products, and individuals. Mining and analyzing this data can yield useful information. For example, analyzing user reviews of a product helps providers understand consumer psychology, thereby improving product quality and service. It also helps other consumers understand the true nature of products, providing a reference for their purchasing decisions.

[0061] However, in relevant text classification schemes, features often need to be constructed manually. The final classification effect depends to a large extent on the quality of feature engineering and human prior knowledge, resulting in poor model flexibility of traditional methods and classification results that cannot meet the user's needs.

[0062] For example, such as Figure 1 The diagram illustrates feature classification based on a dictionary and rule-based approach. First, the polarity and degree of each word in the text are calculated and statistically analyzed using a dictionary. Then, corresponding polarity judgment rules are designed for feature classification. Specifically, a dictionary needs to be constructed first, containing feature words and their corresponding polarities, such as positive or negative. This dictionary can be generated manually or through automatic data mining. Next, the text to be analyzed needs to be preprocessed, including word segmentation, stop word removal, and lemmatization, to facilitate subsequent feature analysis. Then, features need to be extracted from the preprocessed text. Commonly used features include word frequency, word position, negation words, and degree adverbs. Next, rules are formulated to adjust scores based on specific scenarios and contexts. For example, when a sentiment word is preceded by a negation word, the sentiment score is adjusted to the opposite polarity. Finally, the sentiment score of the text is calculated by summing the sentiment word scores and considering rule adjustments.

[0063] However, dictionary-based and rule-based methods rely on dictionaries, whose construction requires manual annotation or automatic mining, making it difficult to cover all vocabulary. Furthermore, due to the diverse contexts and linguistic variations in element classification, rules may not be applicable in all situations. Additionally, dictionary construction typically only considers the features of element words, ignoring the contextual information of the text. This necessitates rebuilding the dictionary and rules when facing new domains or languages, requiring significant manpower and time.

[0064] For example, such as Figure 2The diagram illustrates feature classification based on machine learning methods. Deep learning automatically learns the internal semantics of text from massive datasets, synthesizes different levels of semantics within the text, and obtains the final feature representation of the corresponding text using word vectors. The general process of deep learning-based feature classification can be divided into five stages: text preprocessing, initial word vector representation, feature extraction and contextual semantic representation, deep learning model training and testing, and experimental result analysis and evaluation. Compared to traditional classification methods, deep learning-based methods can avoid a significant amount of manual feature extraction.

[0065] However, machine learning methods require manually constructed features, and classification performance heavily relies on the quality of feature engineering and prior human knowledge, resulting in poor model flexibility. Furthermore, deep learning methods only consider opinion words in the context as important clues, ignoring the potential influence of other words in the context. Especially in complex contexts, relying solely on opinion words for feature classification may overlook the potential impact of other words in the context, leading to inaccurate or biased classification results. Simultaneously, fine-grained text feature analysis datasets are limited in size, and readily available labeled information is not fully utilized, resulting in limited extracted text semantics. This prevents the simultaneous extraction of multi-granular, higher-level semantic features, thus limiting the performance of fine-grained feature analysis models.

[0066] Based on this, embodiments of the present invention provide a text classification method. This method first predicts the target entities in the text to be classified using a first prediction model, obtaining a first classification result for the target entity's category. Then, it extracts the aspect categories corresponding to the target entities using a second prediction model, and performs predictions based on these aspect categories to obtain a second classification result. Here, aspect categories are predefined attribute categories used to evaluate target entities. Next, based on the first and second classification results, the classification result corresponding to the text to be classified is determined. This simultaneously completes coarse-grained element classification prediction at the target entity level and fine-grained requirement classification prediction at the aspect entity level, achieving comprehensive identification and prediction of elements at different levels and granularities. This provides more comprehensive and accurate results for element analysis, improving both model flexibility and classification prediction accuracy, thus meeting user needs.

[0067] According to an embodiment of the present invention, a text classification method embodiment is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0068] This embodiment provides a text classification method that can be used in servers, personal computers, mobile devices, and embedded systems, etc. Figure 3 This is a flowchart of a text classification method according to an embodiment of the present invention, such as... Figure 3 As shown, the process includes the following steps:

[0069] Step S301: Based on the first prediction model, predict the target entity in the text to be classified to obtain the first classification result of the category to which the target entity belongs.

[0070] Step S302: Extract the aspect categories corresponding to the target entity based on the second prediction model, and make predictions based on the aspect categories respectively to obtain the second classification result, wherein the aspect categories are predefined attribute categories used to evaluate the target entity.

[0071] In this embodiment of the invention, the classification prediction model may include a main task learning module and an auxiliary task learning module. The main task learning module includes the first prediction model described above, and the auxiliary task learning module includes the second prediction model described above. Specifically, the main task learning module is used to perform coarse-grained feature classification prediction for target entities in the text to be classified, and the auxiliary task learning module is used to perform fine-grained feature classification prediction for aspect categories of the target entities.

[0072] For example, in sentiment analysis, the text to be classified includes the sentence "The sandwitch is good, but a little expensive." The target entity is "sandwitch," and in the sentence, this target entity can be categorized into two aspects: food (FOOD) and price (PRICE).

[0073] When performing coarse-grained element classification prediction on the target entity using the first prediction model described above, the first classification result of the target entity's category can be determined based on, for example, the sentiment words in the sentence. Here, the sentiment words in the example sentence can be "good" and "expensive". The sentiment polarity corresponding to "good" can be "positive", and the sentiment polarity corresponding to "expensive" can be "negative". Therefore, after comprehensively considering the sentiment polarity of the sentiment words, the first classification result corresponding to the target entity can be "neutral".

[0074] When performing fine-grained element classification prediction on the aspect categories corresponding to the target entity using the second prediction model mentioned above, specifically, when performing fine-grained classification prediction on the above aspect categories, a sub-task can be established for each aspect category. For example, in sub-task 1, the sentiment word "good" corresponding to the aspect category FOOD can be analyzed to obtain the second classification result of sub-task 1 as "positive". In sub-task 2, the sentiment word "expensive" corresponding to the aspect category PRICE can be analyzed to obtain the second classification result of sub-task 2 as "negative".

[0075] Step S303: Based on the first classification result and the second classification result, determine the classification result corresponding to the text to be classified.

[0076] In this embodiment of the invention, the target entity can be analyzed at multiple levels using the first classification result and the second classification result to obtain the classification result. This enables comprehensive identification and prediction of elements at different levels and granularities, providing more comprehensive and accurate results for element analysis.

[0077] For example, in the medical field, semantic hierarchical classification of patient medical records helps to better understand the disease progression and treatment effects; in social media analytics, semantic hierarchical classification of user interaction data on social media helps to more accurately understand user interests and needs; in the financial field, semantic hierarchical classification of various indicators in company financial reports helps to better reveal the company's operating status; and in sentiment analysis, semantic hierarchical classification of sentiment words helps to more precisely understand the sentiment tendency in text.

[0078] As described above, in this embodiment of the invention, the target entity in the text to be classified is first predicted based on a first prediction model to obtain a first classification result. Then, the aspect categories corresponding to the target entity are extracted based on a second prediction model, and predictions are made based on these aspect categories to obtain a second classification result. Here, the aspect categories are predefined attribute categories used to evaluate the target entity. Next, based on the first and second classification results, the classification result corresponding to the text to be classified is determined. This simultaneously completes coarse-grained element classification prediction at the target entity level and fine-grained requirement classification prediction at the aspect entity level, achieving comprehensive identification and prediction of elements at different levels and granularities. This provides more comprehensive and accurate results for element analysis, improving both model flexibility and classification prediction accuracy, thus meeting user needs.

[0079] This embodiment provides another text classification method, which can be used in servers, personal computers, mobile devices, and embedded systems, etc. Figure 4This is a flowchart of another text classification method according to an embodiment of the present invention, such as... Figure 4 As shown, the process includes the following steps:

[0080] Step S401: Based on the first prediction model, predict the target entity in the text to be classified to obtain the first classification result of the target entity's category.

[0081] Specifically, the first prediction model mentioned above includes a self-attention mechanism and a second interactive attention mechanism, and step S401 includes:

[0082] Step S4011: Based on the self-attention mechanism, extract semantic features from the text to be classified, and determine the target entity based on the semantic features.

[0083] Step S4012: Based on the second interactive attention mechanism, predict the first classification result of the category to which the target entity belongs.

[0084] In embodiments of the present invention, such as Figure 5 The diagram illustrates the classification prediction model described above. The main task learning module includes the first prediction model, and the auxiliary task learning module includes the second prediction model. The first prediction model incorporates a self-attention mechanism (i.e.,...). Figure 1 The first prediction model uses SA (hereinafter referred to as SA), and the second interactive attention mechanism (cross-attention, i.e.) Figure 1 CA in the first prediction model (hereinafter referred to as CA).

[0085] After obtaining the context embeddings (containing word vectors corresponding to each word in the text) from BERT (Bidirectional Encoder Representations from Transformers) processing of the text to be classified, the relationship between each word and all other words in the text can be calculated using SA (Attention Score). Specifically, this can be done by calculating an "attention score," which measures the importance of each word to the other words. These scores are then normalized and used to weighted average all word vectors to obtain a new context vector.

[0086] Next, point-wise convolution transformation (PCT) can be used to process the temporal information of the context vector to obtain contextual features (i.e., the semantic features mentioned above), and the target entity can be identified based on these contextual features. PCT is a technique that assumes time is a piecewise constant, meaning that the change in time is constant within each time interval. This can be achieved by encoding the temporal information into an additional feature vector and adding it to the context vector.

[0087] After identifying the target entity, classification prediction can be performed. Specifically, the target entity can be processed using BERT to obtain a target embedding containing word vectors of the target entity. Then, semantic association (CA) can be used to calculate the semantic correlation between the target embedding and other words in the text to be processed, i.e., learning and extracting the semantic association between the two texts at the word level. Next, temporal information of the target embedding can be processed using PCT to reduce the complexity of the model and obtain the target features corresponding to the target entity. Based on these target features, the first classification result of the target entity's category can be predicted.

[0088] Specifically, the formula for calculating CA is as follows:

[0089] Where K and Q are matrices composed of word vectors, k i q j These are two word vectors used to calculate semantic relevance, meaning that contextual semantics are learned from the word level. tanh is the activation function, and softmax is the weight normalization function.

[0090] Additionally, the formula for calculating PCT is as follows:

[0091] Among them, W pc Let b be the weight matrix. pc These are bias vectors, which are all learnable parameters during the model training process.

[0092] In this embodiment of the invention, the first prediction model includes a multi-head attention mechanism, and step S4012 includes:

[0093] Step a1: When the target entity contains multiple sub-entities, the target features of each sub-entity are extracted through the second interactive attention mechanism.

[0094] Step a2: Based on the multi-head attention mechanism, predict the classification result of each target feature to which it belongs, and obtain the first classification result.

[0095] In this embodiment of the invention, considering that a target entity may include multiple sub-entities, for example, when the target entity is XX Camera, the target entity includes the sub-entity "XX (brand)" and the sub-entity "Camera". It should be understood that the contribution of each sub-entity in the target entity to the semantic expression of the entire target entity will also be different. For example, when discussing the quality of a camera, the semantics of "Camera" are more critical than that of "XX". Therefore, a multi-head attention mechanism (hereinafter referred to as MA) can be used to learn the semantic relationship between the entity and the context from multiple perspectives and at different granularities.

[0096] Specifically, multiple attention heads can be pre-designed in the MA mentioned above, with each head responsible for a sub-entity. Then, all heads can process in parallel, extracting features from different perspectives to output the target sentence vector corresponding to each sub-entity. Next, the target sentence features can be processed using softmax to obtain the first classification result corresponding to the target entity.

[0097] Here, the formula for calculating MA is as follows:

[0098] MA(K,Q)=[CA 1 CA 2 ,...,CA nh ]·W ma , where CA 1 CA 2 ,...,CA nh Attention head W in MA ma The weight matrix represents the learnable parameters during model training.

[0099] Step S402: Extract the aspect categories corresponding to the target entity based on the second prediction model, and perform predictions based on each aspect category to obtain the second classification result. Here, the aspect categories are predefined attribute categories used to evaluate the target entity. For details, please refer to [link to relevant documentation]. Figure 1 Step S102 of the illustrated embodiment will not be described again here.

[0100] Step S403: Based on the first classification result and the second classification result, determine the classification result corresponding to the text to be classified. For details, please refer to [link to relevant documentation]. Figure 1 Step S103 of the illustrated embodiment will not be described again here.

[0101] In this embodiment of the invention, a first prediction model can be used to perform coarse-grained element classification prediction of target entities in the text to be classified, thereby providing a technical basis for realizing multi-level and multi-granularity analysis of target entities.

[0102] This embodiment provides yet another text classification method, which can be used in servers, personal computers, mobile devices, and embedded systems, etc. Figure 6 This is a flowchart of another text classification method according to an embodiment of the present invention, such as... Figure 6 As shown, the process includes the following steps:

[0103] Step S601: Based on the first prediction model, predict the target entity in the text to be classified to obtain the first classification result of the target entity's category. For details, please refer to [link to relevant documentation]. Figure 1 Step S101 of the illustrated embodiment will not be described again here.

[0104] Step S602: Extract the aspect categories corresponding to the target entity based on the second prediction model, and make predictions based on the aspect categories to obtain the second classification result. The aspect category is a predefined attribute category used to evaluate the target entity.

[0105] Specifically, the second prediction model mentioned above includes: a first interactive attention mechanism, and step S602 includes:

[0106] Step S6021: Obtain the text to be classified.

[0107] Step S6022: Based on the pre-trained model, identify the aspect categories of the text to be classified.

[0108] Step S6023: The text to be classified is identified based on the start identifier and the end identifier to obtain the first identification result, and the first identification result is vectorized through a preset matrix to obtain the text vector.

[0109] Step S6024: Identify the aspect category based on the start identifier and the end identifier to obtain the second identification result, and vectorize the second identification result through a preset matrix to obtain the category vector.

[0110] In this embodiment of the invention, the pre-trained model can be the BERT model mentioned above, that is, the input layer of the classification prediction model. This input layer can convert the input text into vector form, thereby mapping words from the semantic space to the vector space.

[0111] Here, BERT can first label the text to be classified based on the start identifier and the end identifier to obtain the first label result. Then, it can learn and supervise the fusion of the entire text semantics through these two labels from the beginning and end of the text, respectively. The start identifier can be "[CLS]" and the end identifier can be "[SEP]".

[0112] For example, if the text to be classified includes the example sentence "The food is good, but a little expensive," then the first identifier result corresponding to this example sentence can be represented as "[CLS]+The food is good, but alittle expensive.+[SEP]".

[0113] Next, the first identifier result can be vectorized using a preset matrix to obtain a text vector. Specifically, the first identifier result can be represented as a text sequence s = {} of length l, which, after passing through the preset matrix A, yields a text vector X = {}.

[0114] Similarly, based on the start and end identifiers, the aspect categories are identified to obtain a second identification result. This second identification result is then vectorized using a preset matrix to obtain a category vector. Here, we take... Figure 5 The example sentence "The sandwitch is delicious, but the waiters are rude" can be used to illustrate the category of an example sentence, which may include "FOOD", "SERVICE", "AMBIENCE", etc.

[0115] Specifically, the method for identifying aspect categories to obtain the second identification result is similar to the process of determining the first identification result mentioned above, and will not be repeated here. Here, if there are N aspect entities A = {}, the category vector X can be obtained by vectorizing the second identification result corresponding to the aspect entity using a preset matrix. a ={}.

[0116] It should be understood that BERT can also vectorize the third representation of the target entity to obtain an entity vector. Specifically, the method for identifying the target entity to obtain the third identification result is similar to the process of determining the first identification result, and will not be elaborated here. Here, for an aspect entity T of length M... i The vectorization result of ={} can be the entity vector X. t ={}, where T i It can be a subsequence of the above text sequence s.

[0117] Step S6025: Based on aspect category, create subtasks.

[0118] Step S6026: Based on the first interactive attention mechanism, extract the target sentiment word vectors from the text vectors that match the aspect categories of the subtask.

[0119] Step S6027: Analyze the sub-task based on the target sentiment word vector to obtain the sub-classification results of the sub-task.

[0120] Step S6028: Determine the second classification result based on the sub-classification result.

[0121] In embodiments of the present invention, such as Figure 5 As shown, a corresponding subtask can be established for each aspect category to obtain the sub-classification result of that subtask. When executing the subtask for each aspect category, a multi-layer attention mechanism can be used, with each layer using the output of the previous layer as the query input for the next layer, thereby progressively refining the category features. Here, the aforementioned first interactive attention mechanism can include two CA processes, i.e., a two-layer attention mechanism.

[0122] Taking subtask 1 corresponding to the aspect category FOOD as an example, we first calculate the semantic correlation between the category 1 embedding and other words in the text to be processed using CA, obtaining the category 1 feature. This category 1 feature is then used as the query input for the next CA to obtain the category 1 sentence feature. Then, we process this category 1 sentence feature using softmax as described above to obtain the subclassification result corresponding to the aspect category FOOD.

[0123] Next, in step S6026 above, based on the first interactive attention mechanism, the target sentiment word vectors that match the aspect category of the subtask are extracted from the text vector, including:

[0124] Step b1: Extract category features of aspect categories based on the first interaction attention mechanism.

[0125] Step b2: Extract sentence features that match the category features from the text vector, and determine the target sentiment word vector based on the sentence features.

[0126] In this embodiment of the invention, the category feature is as described above. Figure 5 The category 1 features, ..., category N features shown above, and the target sentiment word vectors are included in the above. Figure 5 In the sentence features of category 1, ..., category N shown, that is, the category features are the output results of the first CA corresponding to the subtask, and the category sentence features containing the target sentiment word vector are the output results of the second CA corresponding to the subtask.

[0127] Step S603: Based on the first classification result and the second classification result, determine the classification result corresponding to the text to be classified. For details, please refer to [link to relevant documentation]. Figure 1 Step S103 of the illustrated embodiment will not be described again here.

[0128] In this embodiment of the invention, a second prediction model can be used to perform fine-grained element classification prediction of the aspect type corresponding to the target entity in the text to be classified, thereby providing a technical basis for realizing multi-level and multi-granularity analysis of the target entity.

[0129] In some optional implementations, step S2022 above includes:

[0130] Step c1: Analyze the weights of subtasks based on their aspect categories.

[0131] Step c2: The subclassification results are weighted and summed according to their weights to obtain the second classification result.

[0132] In this embodiment of the invention, the weight parameters of the classification prediction model can be adjusted according to the corresponding classification domain. Here, we take... Figure 5 Taking the corresponding sentiment analysis field as an example, the weight of the aspect category "FOOD" can be set to the highest, for example, 50%, followed by the weight of the aspect category "SERVICE", for example, 25%, and then the weight of the aspect category "AMBIENCE", for example, 25%.

[0133] Here, if the subclassification result corresponding to the aspect category "FOOD" is A1, the subclassification result corresponding to the aspect category "SERVICE" is A2, and the subclassification result corresponding to the aspect category "AMBIENCE" is A3, then the second classification result can be expressed as: 0.5A1 + 0.25A2 + 0.25A3.

[0134] In this embodiment of the invention, the sub-classification results of the sub-tasks corresponding to each aspect category can be weighted and summed, thereby making full use of the dataset of fine-grained element analysis in complex contexts and improving the accuracy of fine-grained classification results.

[0135] In some alternative implementations, the above Figure 1 Corresponding embodiments also include:

[0136] Step S11: Determine the first loss function corresponding to the first prediction model and the second loss function corresponding to the second prediction model based on the cross-entropy loss function.

[0137] Step S12: Based on the first loss function and the second loss function, construct the target loss function, which is used to guide the optimization of the classification results.

[0138] In this embodiment of the invention, the first loss function corresponding to the first prediction model determined by the cross-entropy loss function can be expressed as: Furthermore, the second loss function corresponding to the second prediction model determined by the cross-entropy loss function can be expressed as:

[0139] Based on this, the constructed objective loss function can be expressed as: Here, λ and θ are preset to be constants.

[0140] In this embodiment of the invention, since the above classification prediction model includes a first prediction model and a second prediction model, the loss functions of the two models can be calculated separately, and the target loss function can be constructed by comprehensively considering the first loss function and the second loss function. This allows for better training of the classification prediction model, measurement of model performance, and optimization of model parameters, thereby improving the accuracy of the output results of the classification prediction model.

[0141] In summary, in this embodiment of the invention, the target entity in the text to be classified can first be predicted based on the first prediction model to obtain the first classification result of the target entity's category. Then, the aspect categories corresponding to the target entity can be extracted based on the second prediction model, and predictions can be made based on the aspect categories to obtain the second classification result. Here, the aspect categories are predefined attribute categories used to evaluate the target entity. Next, the classification result corresponding to the text to be classified can be determined based on the first and second classification results, thereby simultaneously completing coarse-grained element classification prediction at the target entity level and fine-grained requirement classification prediction at the aspect entity level. This achieves comprehensive identification and prediction of elements at different levels and granularities, providing more comprehensive and accurate results for element analysis. While improving the model's flexibility, it also improves the accuracy of classification prediction, meeting the user's needs.

[0142] This embodiment also provides a text classification device for implementing the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0143] This embodiment provides a text classification device, such as... Figure 7 As shown, it includes:

[0144] The first prediction module 701 is used to predict the target entity in the text to be classified based on the first prediction model, and to obtain the first classification result of the category to which the target entity belongs.

[0145] The second prediction module 702 is used to extract the aspect categories corresponding to the target entity based on the second prediction model, and to make predictions based on the aspect categories to obtain the second classification result. The aspect categories are predefined attribute categories used to evaluate the target entity.

[0146] The determination module 703 is used to determine the classification result corresponding to the text to be classified based on the first classification result and the second classification result.

[0147] In some optional implementations, the second prediction model includes: a first interactive attention mechanism; the second prediction module 702 is further configured to:

[0148] Get the text to be categorized;

[0149] Based on a pre-trained model, the aspect categories of the text to be classified are identified;

[0150] The text to be classified is identified based on the start identifier and the end identifier to obtain the first identification result. The first identification result is then vectorized using a preset matrix to obtain the text vector.

[0151] The aspect categories are identified based on the start identifier and the end identifier to obtain the second identification result. The second identification result is then vectorized using a preset matrix to obtain the category vector.

[0152] Based on aspect categories, sub-tasks are created;

[0153] Based on the first interaction attention mechanism, target sentiment word vectors that match the aspect categories of the subtask are extracted from the text vectors;

[0154] The sub-tasks are analyzed based on the target sentiment word vectors to obtain the sub-classification results of the sub-tasks;

[0155] The second classification result is determined based on the subclassification results.

[0156] In some optional implementations, the second prediction module 702 is further configured to:

[0157] Analyze the weights of subtasks based on their aspect categories;

[0158] The subclassification results are weighted and summed according to their weights to obtain the second classification result.

[0159] In some optional implementations, the second prediction module 702 is further configured to:

[0160] Based on the first interaction attention mechanism, extract the category features of aspect categories;

[0161] Extract sentence features that match the category features from the text vectors, and determine the target sentiment word vectors based on the sentence features.

[0162] In some optional implementations, the first prediction model includes: a self-attention mechanism and a second interactive attention mechanism; the first prediction module 701 is further configured to:

[0163] Based on the self-attention mechanism, semantic features are extracted from the text to be classified, and the target entity is determined based on the semantic features;

[0164] Based on the second interactive attention mechanism, the first classification result of the target entity is predicted.

[0165] In some optional implementations, the first prediction model includes a multi-head attention mechanism; the first prediction module 701 is further configured to:

[0166] When the target entity contains multiple sub-entities, the target features of each sub-entity are extracted through a second interactive attention mechanism;

[0167] Based on the multi-head attention mechanism, the classification result of each target feature is predicted to obtain the first classification result.

[0168] In some alternative implementations, the device is also used for:

[0169] The first loss function corresponding to the first prediction model and the second loss function corresponding to the second prediction model are determined based on the cross-entropy loss function.

[0170] Based on the first loss function and the second loss function, a target loss function is constructed, which is used to guide the optimization of classification results.

[0171] Further functional descriptions of the above modules and units are the same as those in the corresponding embodiments described above, and will not be repeated here.

[0172] In this embodiment, the text classification device is presented in the form of a functional unit. Here, a unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0173] This invention also provides a computer device having the above-described features. Figure 7 The text classification device shown.

[0174] Please see Figure 8 , Figure 8 This is a schematic diagram of the structure of a computer device provided in an optional embodiment of the present invention, such as... Figure 8As shown, the computer device includes one or more processors 10, memory 20, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components communicate with each other via different buses and can be mounted on a common motherboard or otherwise installed as needed. The processors can process instructions executed within the computer device, including instructions stored in or on memory to display graphical information of a GUI on external input / output devices (such as display devices coupled to the interfaces). In some alternative implementations, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple computer devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system). Figure 8 Take a processor 10 as an example.

[0175] Processor 10 may be a central processing unit, a network processor, or a combination thereof. Processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The programmable logic device may be a complex programmable logic device (CAMP), a field-programmable gate array (FPGA), a general-purpose array logic (GDA), or any combination thereof.

[0176] The memory 20 stores instructions executable by at least one processor 10 to cause the at least one processor 10 to perform the method shown in the above embodiments.

[0177] The memory 20 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the computer device. Furthermore, the memory 20 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some alternative embodiments, the memory 20 may optionally include memory remotely located relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0178] The memory 20 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, hard disk or solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0179] The computer device also includes a communication interface 30 for communicating with other devices or communication networks.

[0180] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the methods shown in the above embodiments.

[0181] A portion of this invention can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide the methods and / or technical solutions according to the invention through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, installation package files, etc. Correspondingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions, or the computer compiling the instructions and then executing the corresponding compiled program, or the computer reading and executing the instructions, or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0182] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A text classification method, characterized in that, The method includes: Based on the first prediction model, the target entity in the text to be classified is predicted to obtain the first classification result of the category to which the target entity belongs; The second prediction model is used to extract the aspect categories corresponding to the target entity, and predictions are made based on the aspect categories to obtain the second classification result. The aspect categories are predefined attribute categories used to evaluate the target entity. Based on the first classification result and the second classification result, the classification result corresponding to the text to be classified is determined; The first prediction model includes: a self-attention mechanism and a second interactive attention mechanism; The step of predicting the target entity in the text to be classified based on the first prediction model to obtain the first classification result of the category to which the target entity belongs includes: Based on the self-attention mechanism, semantic features are extracted from the text to be classified, and the target entity is determined based on the semantic features; Based on the second interactive attention mechanism, predict the first classification result of the category to which the target entity belongs; The first prediction model includes: a multi-head attention mechanism; The first classification result for predicting the category to which the target entity belongs based on the second interactive attention mechanism includes: When the target entity contains multiple sub-entities, the target features of each sub-entity are extracted through the second interactive attention mechanism; Based on the multi-head attention mechanism, the classification result of each target feature is predicted to obtain the first classification result; The second prediction model includes: a first interactive attention mechanism; The step of extracting the aspect categories corresponding to the target entity based on the second prediction model, and making predictions based on the aspect categories respectively to obtain the second classification result includes: Get the text to be categorized; Based on a pre-trained model, the aspect categories of the text to be classified are identified; The text to be classified is identified based on the start identifier and the end identifier to obtain a first identification result, and the first identification result is vectorized through a preset matrix to obtain a text vector; The aspect categories are identified based on the start identifier and the end identifier to obtain a second identification result, and the second identification result is vectorized through a preset matrix to obtain a category vector; Based on the aforementioned aspect categories, sub-tasks are established; Based on the first interactive attention mechanism, target sentiment word vectors that match the aspect category of the subtask are extracted from the text vector; The subtask is analyzed based on the target sentiment word vectors to obtain the sub-classification results of the subtask; The second classification result is determined based on the sub-classification results.

2. The method according to claim 1, characterized in that, Determining the second classification result based on the sub-classification result includes: Based on the aspect categories of the subtasks, analyze the weights of the subtasks; The sub-classification results are weighted and summed according to the weights to obtain the second classification result.

3. The method according to claim 1, characterized in that, The step of extracting target sentiment word vectors from the text vector that match the aspect category of the subtask based on the first interactive attention mechanism includes: Based on the first interactive attention mechanism, the category features of the aspect categories are extracted; Sentence features that match the category features are extracted from the text vector, and the target sentiment word vector is determined based on the sentence features.

4. The method according to claim 1, characterized in that, The method further includes: The first loss function corresponding to the first prediction model and the second loss function corresponding to the second prediction model are determined based on the cross-entropy loss function. Based on the first loss function and the second loss function, a target loss function is constructed, wherein the target loss function is used to guide the optimization of the classification result.

5. A text classification device, characterized in that, The device includes: The first prediction module is used to predict the target entity in the text to be classified based on the first prediction model, and to obtain the first classification result of the category to which the target entity belongs; The second prediction module is used to extract the aspect categories corresponding to the target entity based on the second prediction model, and to make predictions based on the aspect categories to obtain a second classification result, wherein the aspect categories are predefined attribute categories used to evaluate the target entity; The determination module is used to determine the classification result corresponding to the text to be classified based on the first classification result and the second classification result; The first prediction module is specifically used to extract semantic features from the text to be classified based on the self-attention mechanism, and determine the target entity based on the semantic features; and to predict the first classification result of the category to which the target entity belongs based on the second interactive attention mechanism; wherein, the first prediction model includes a self-attention mechanism and a second interactive attention mechanism; The first prediction module is specifically used to extract the target features of each sub-entity through the second interactive attention mechanism when the target entity contains multiple sub-entities; and to predict the classification result of the category to which each target feature belongs based on the multi-head attention mechanism to obtain the first classification result; wherein, the first prediction model includes a multi-head attention mechanism; The second prediction module is specifically used to: acquire the text to be classified; identify the aspect categories of the text to be classified based on a pre-trained model; identify the text to be classified based on start and end identifiers to obtain a first identification result, and vectorize the first identification result using a preset matrix to obtain a text vector; identify the aspect categories based on the start and end identifiers to obtain a second identification result, and vectorize the second identification result using a preset matrix to obtain a category vector; establish a subtask based on the aspect categories; extract target sentiment word vectors from the text vectors that match the aspect categories of the subtask based on the first interactive attention mechanism; analyze the subtask based on the target sentiment word vectors to obtain the sub-classification result of the subtask; and determine the second classification result based on the sub-classification result; wherein, the second prediction model includes: the first interactive attention mechanism.

6. A computer device, characterized in that, include: A memory and a processor are communicatively connected, the memory storing computer instructions, and the processor executing the computer instructions to perform the text classification method of any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to perform the text classification method according to any one of claims 1 to 4.

8. A computer program product, characterized in that, Includes computer instructions for causing a computer to perform the text classification method according to any one of claims 1 to 4.

Images