Text classification method and device, computer equipment, storage medium and program product

By using the first and second prediction models in text classification combined with interactive attention, self-attention and multi-head attention mechanisms, multi-level and multi-grained analysis of the text is achieved, and the traditional method has poor flexibility and poor classification effect is solved. Excellent problem improves classification accuracy and flexibility.

CN119088967AActive Publication Date: 2024-12-06SHANDONG YUNHAI GUOCHUANG CLOUD COMPUTING EQUIP IND INNOVATION CENT CO LTD

Patent Information

Application Number
CN202411203600.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-29
Publication Date
2024-12-06
Estimated Expiration
2044-08-29

AI Technical Summary

Technical Problem

The traditional text classification method has poor flexibility and relies on artificial construction of features, which leads to the classification effect not meeting user needs, and it is difficult to fully utilize the data sets analyzed by fine-grained factor in complex contexts, resulting in inaccurate or biased classification results.

Method used

Using the method based on the first and second prediction models, the target entities in the classified text are predicted, and the coarse-grained and fine-grained elements are classified respectively. The emotional word vector is extracted through the interactive attention mechanism, combining self-attention and multi-head attention. mechanism to realize multi-level and multi-grained analysis, and optimize model parameters using cross-entropy loss function.

Benefits of technology

It improves the flexibility and accuracy of the text classification model, and can make full use of the data sets of fine-grained factor analysis in complex contexts, provide more comprehensive and accurate classification results, and meet users' usage needs.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119088967A_ABST
    Figure CN119088967A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of natural language processing, and discloses a text classification method and device, computer equipment, a storage medium and a program product, the text classification method comprises the following steps: predicting a target entity in a to-be-classified text based on a first prediction model to obtain a first classification result of a classification to which the target entity belongs; extracting aspect categories corresponding to the target entity based on a second prediction model, and performing prediction based on the aspect categories to obtain a second classification result; and based on the first classification result and the second classification result, determining a classification result corresponding to the to-be-classified text. According to the method, coarse-grained element classification prediction from a target entity level and fine-grained requirement classification prediction from an aspect entity level can be completed at the same time, comprehensive identification and prediction of elements of different levels and different granularities are realized, and a more comprehensive and accurate result is provided for element analysis; the accuracy of classification prediction is improved while the flexibility of the model is improved, and the use requirements of users are met.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular to a text classification method, device, computer equipment, storage medium and program product. Background Art

[0002] Feature classification has always been a basic task of machine learning. By designing and using computer algorithms to determine the category to which a feature belongs, it has been widely used in news classification, sentiment classification, theme classification, image classification, video classification, advertising filtering, content review, comment analysis, question answering and other natural language processing, data mining, recommendation systems, advertising systems and other fields. These massive data contain netizens' views, attitudes and judgments on various things, including their evaluations and opinions on hot events, commodities, and people. By mining and analyzing these data, some useful information can be obtained. For example, analyzing users' evaluation information on a certain product is helpful for providers to understand consumers' psychology, thereby improving product quality and service. It is also helpful for other consumers to understand more real products and provide reference for purchasing decisions.

[0003] However, in related text classification schemes, features often need to be constructed manually, and the final classification effect depends to a large extent on the construction quality of feature engineering and human prior knowledge, which leads to poor model flexibility of traditional methods and the classification effect cannot meet user needs. Summary of the invention

[0004] In view of this, the present invention provides a text classification method, apparatus, computer device, storage medium and program product to solve the problem that the classification model of the traditional method has poor flexibility and cannot meet the user's usage requirements.

[0005] In a first aspect, the present invention provides a text classification method, the method comprising:

[0006] Predicting a target entity in the to-be-classified text based on the first prediction model to obtain a first classification result of the category to which the target entity belongs;

[0007] Extracting aspect categories corresponding to the target entity based on the second prediction model, and performing predictions based on the aspect categories respectively to obtain second classification results, wherein the aspect categories are predefined attribute categories for evaluating the target entity;

[0008] Based on the first classification result and the second classification result, a classification result corresponding to the text to be classified is determined.

[0009] In an optional embodiment, the second prediction model includes: a first interactive attention mechanism;

[0010] The aspect categories corresponding to the target entity are extracted based on the second prediction model, and predictions are made based on the aspect categories to obtain the second classification results, including:

[0011] Get the text to be classified;

[0012] Based on the pre-trained model, identify the aspect categories of the text to be classified;

[0013] The text to be classified is identified based on the start identifier and the end identifier to obtain a first identification result, and the first identification result is vectorized through a preset matrix to obtain a text vector;

[0014] Identify the aspect category based on the start identifier and the end identifier to obtain a second identification result, and vectorize the second identification result through a preset matrix to obtain a category vector;

[0015] Based on the aspect categories, subtasks are established;

[0016] Based on the first interactive attention mechanism, extract the target sentiment word vector that matches the aspect category of the subtask in the text vector;

[0017] Analyze the subtasks based on the target sentiment word vectors to obtain the sub-classification results of the subtasks;

[0018] A second classification result is determined according to the sub-classification result.

[0019] In the embodiment of the present invention, the second prediction model can be used to perform fine-grained element classification prediction on the aspect type corresponding to the target entity in the classified text, thereby providing a technical basis for realizing multi-level and multi-granularity analysis of the target entity.

[0020] In an optional implementation, determining the second classification result according to the sub-classification result includes:

[0021] Analyze the weights of subtasks based on their aspect categories;

[0022] The sub-classification results are weighted and summed according to the weights to obtain the second classification result.

[0023] In the embodiment of the present invention, the sub-classification results of the sub-tasks corresponding to each aspect category may be weighted and summed, thereby making full use of the data set of fine-grained factor analysis in a complex context and improving the accuracy of the fine-grained classification results.

[0024] In an optional implementation, based on the first interactive attention mechanism, extracting a target sentiment word vector that matches the aspect category of the subtask in the text vector includes:

[0025] Based on the first interactive attention mechanism, the category features of the aspect categories are extracted;

[0026] Sentence features that match the category features are extracted from the text vector, and the target sentiment word vector is determined based on the sentence features.

[0027] In the embodiment of the present invention, the second prediction model can be used to perform fine-grained element classification prediction on the aspect type corresponding to the target entity in the classified text, thereby providing a technical basis for realizing multi-level and multi-granularity analysis of the target entity.

[0028] In an optional implementation, the first prediction model includes: a self-attention mechanism and a second interactive attention mechanism;

[0029] The target entity in the to-be-classified text is predicted based on the first prediction model to obtain a first classification result of the category to which the target entity belongs, including:

[0030] Based on the self-attention mechanism, the semantic features in the text to be classified are extracted, and the target entity is determined according to the semantic features;

[0031] Based on the second interactive attention mechanism, predict the first classification result of the category to which the target entity belongs.

[0032] In the embodiment of the present invention, the first prediction model can be used to perform coarse-grained element classification prediction on the target entity in the text to be classified, thereby providing a technical basis for realizing multi-level and multi-granularity analysis of the target entity.

[0033] In an optional implementation, the first prediction model includes: a multi-head attention mechanism;

[0034] Based on the second interactive attention mechanism, the first classification result of the category to which the target entity belongs is predicted, including:

[0035] When the target entity contains multiple sub-entities, the target features of each sub-entity are extracted through the second interactive attention mechanism;

[0036] Based on the multi-head attention mechanism, the classification result of each target feature is predicted to obtain the first classification result.

[0037] In the embodiment of the present invention, the first prediction model can be used to perform coarse-grained element classification prediction on the target entity in the text to be classified, thereby providing a technical basis for realizing multi-level and multi-granularity analysis of the target entity.

[0038] In an optional embodiment, the method further includes:

[0039] Determine a first loss function corresponding to the first prediction model and a second loss function corresponding to the second prediction model according to the cross entropy loss function;

[0040] Based on the first loss function and the second loss function, a target loss function is constructed, wherein the target loss function is used to guide the optimization of the classification result.

[0041] In an embodiment of the present invention, since the above-mentioned classification prediction model includes a first prediction model and a second prediction model, the loss functions of the two models can be calculated separately, and the first loss function and the second loss function can be comprehensively considered to construct a target loss function, so as to better train the classification prediction model, measure model performance and optimize model parameters, thereby improving the accuracy of the output results of the classification prediction model.

[0042] In a second aspect, the present invention provides a text classification device, the device comprising:

[0043] A first prediction module, used to predict a target entity in the to-be-classified text based on a first prediction model, and obtain a first classification result of a category to which the target entity belongs;

[0044] A second prediction module is used to extract the aspect categories corresponding to the target entity based on the second prediction model, and perform predictions based on the aspect categories to obtain a second classification result, wherein the aspect categories are predefined attribute categories used to evaluate the target entity;

[0045] The determination module is used to determine the classification result corresponding to the text to be classified based on the first classification result and the second classification result.

[0046] In a third aspect, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the text classification method of the first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0047] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the text classification method of the first aspect or any corresponding embodiment thereof.

[0048] In a fifth aspect, the present invention provides a computer program product, comprising computer instructions for causing a computer to execute the text classification method of the first aspect or any corresponding embodiment thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0050] Figure 1 It is a schematic diagram of feature classification based on dictionary and rule methods;

[0051] Figure 2 It is a schematic diagram of feature classification based on machine learning methods;

[0052] Figure 3 is a flowchart of a text classification method according to an embodiment of the present invention;

[0053] Figure 4 is a flowchart of another text classification method according to an embodiment of the present invention;

[0054] Figure 5 It is a schematic diagram of the classification prediction model;

[0055] Figure 6 is a flowchart of another text classification method according to an embodiment of the present invention;

[0056] Figure 7 is a structural block diagram of a text classification device according to an embodiment of the present invention;

[0057] Figure 8 It is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0058] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0059] In conjunction with the application scenarios on which the execution of the text classification method depends, the application scenarios are described here.

[0060] Feature classification has always been a basic task of machine learning. By designing and using computer algorithms to determine the category to which a feature belongs, it has been widely used in news classification, sentiment classification, theme classification, image classification, video classification, advertising filtering, content review, comment analysis, question answering and other natural language processing, data mining, recommendation systems, advertising systems and other fields. These massive data contain netizens' views, attitudes and judgments on various things, including their evaluations and opinions on hot events, commodities, and people. By mining and analyzing these data, some useful information can be obtained. For example, analyzing users' evaluation information on a certain product is helpful for providers to understand consumers' psychology, thereby improving product quality and service. It is also helpful for other consumers to understand more real products and provide reference for purchasing decisions.

[0061] However, in related text classification schemes, features often need to be constructed manually, and the final classification effect depends to a large extent on the construction quality of feature engineering and human prior knowledge, which leads to poor model flexibility of traditional methods and the classification effect cannot meet user needs.

[0062] For example, Figure 1 The figure shows a schematic diagram of factor classification based on the dictionary and rule method, in which the polarity and degree of each word in the text can be calculated and counted with the help of the dictionary, and then the corresponding polarity judgment rules can be designed for factor classification. Specifically, it is necessary to first build a dictionary, which contains some factor words and their corresponding polarities, such as positive or negative. The dictionary can be generated by manual annotation or automatic mining. Then, the text to be analyzed needs to be preprocessed, including operations such as word segmentation, stop word removal, and word form restoration, so as to facilitate subsequent factor analysis. Next, features need to be extracted from the preprocessed text. Commonly used features include word frequency, word position, negative words, degree adverbs, etc. Secondly, some rules are formulated to adjust the score according to specific scenarios and contexts. For example, when there is a negative word before the sentiment word, the sentiment score is adjusted to the opposite polarity. Finally, the sentiment score of the text is calculated by adding the sentiment word scores and considering the rule adjustment.

[0063] However, the dictionary-based and rule-based methods rely on dictionaries, and the construction of dictionaries requires manual annotation or automatic mining, which makes it difficult to cover all vocabulary. In addition, due to the diverse contexts and languages ​​of feature classification, rules may not be applicable to all situations. In addition, when constructing dictionaries, only the features of feature vocabulary are usually considered, ignoring the contextual information of the text. This leads to the need to rebuild dictionaries and rules when facing new fields or languages, which requires a lot of manpower and time.

[0064] For example, Figure 2The figure shows a schematic diagram of feature classification based on machine learning methods, in which the deep learning method automatically learns the internal semantics of the text from massive data sets, synthesizes the different levels of semantics in the text, and uses word vectors to obtain the final feature representation of the corresponding text. The general process of feature classification based on deep learning can be divided into five stages: text preprocessing, initial word vector representation, feature extraction and contextual semantic representation, deep learning model training and testing, and experimental results analysis and evaluation. Compared with traditional classification methods, methods based on deep learning can avoid a lot of manual labor consumption to manually extract features.

[0065] However, machine learning methods require artificial feature construction, and the classification effect depends largely on the quality of feature engineering and human prior knowledge, which leads to poor model flexibility. In addition, deep learning methods only use opinion words in the context as important clues, but ignore the potential impact of other words in the context. Especially in complex contexts, relying solely on opinion words as important clues for feature classification may ignore the potential impact of other words in the context, resulting in inaccurate or large deviations in classification results. At the same time, the data set for fine-grained feature analysis of texts is limited in data volume, and the easily available annotated information in the data set is not fully utilized. The extracted text semantics is limited, and multi-granular, higher-level semantic features cannot be extracted at the same time, resulting in limited performance of fine-grained feature analysis models.

[0066] Based on this, an embodiment of the present invention provides a text classification method, which can first predict the target entity in the text to be classified based on the first prediction model to obtain a first classification result of the category to which the target entity belongs. Then, the aspect category corresponding to the target entity can be extracted based on the second prediction model, and predictions can be made based on the aspect categories to obtain a second classification result, wherein the aspect category is a predefined attribute category for evaluating the target entity. Next, the classification result corresponding to the text to be classified can be determined based on the first classification result and the second classification result, thereby completing the coarse-grained element classification prediction at the target entity level and the fine-grained requirement classification prediction at the aspect entity level at the same time, realizing comprehensive identification and prediction of elements at different levels and granularities, providing more comprehensive and accurate results for element analysis, improving the accuracy of classification prediction while improving the flexibility of the model, and meeting the user's usage needs.

[0067] According to an embodiment of the present invention, a text classification method embodiment is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0068] In this embodiment, a text classification method is provided, which can be used in servers, personal computers, mobile devices, and embedded systems. Figure 3 is a flowchart of a text classification method according to an embodiment of the present invention. Figure 3 As shown, the process includes the following steps:

[0069] Step S301, predicting a target entity in the text to be classified based on a first prediction model, and obtaining a first classification result of the category to which the target entity belongs.

[0070] Step S302: extracting aspect categories corresponding to the target entity based on the second prediction model, and performing predictions based on the aspect categories to obtain second classification results, wherein the aspect categories are predefined attribute categories for evaluating the target entity.

[0071] In an embodiment of the present invention, the classification prediction model may include a main task learning module and an auxiliary task learning module, wherein the main task learning module includes the above-mentioned first prediction model, and the auxiliary task learning module includes the above-mentioned second prediction model. Specifically, the main task learning module is used to perform coarse-grained element classification prediction for the target entity in the text to be classified, and the auxiliary task learning module is used to perform fine-grained element classification prediction for the aspect category of the target entity.

[0072] For example, taking sentiment analysis as an example, the text to be classified includes the sentence "The sandwitch is good, but a little expensive". The target entity is "sandwitch", and the two aspect categories corresponding to the target entity in the sentence can be food (ie, FOOD) and price (ie, PRICE).

[0073] When the target entity is predicted by the coarse-grained element classification through the above-mentioned first prediction model, the first classification result of the category to which the target entity belongs can be determined according to the sentiment words in, for example. Here, the sentiment words in the example sentence can be "good" and "expensive", where the sentiment polarity corresponding to "good" can be "positive", and the sentiment polarity corresponding to "expensive" can be "negative". Therefore, after comprehensively considering the sentiment polarity of the sentiment words, the first classification result corresponding to the target entity can be "neutral".

[0074] When fine-grained element classification prediction is performed on the aspect categories corresponding to the target entity through the above-mentioned second prediction model, specifically, when fine-grained classification prediction is performed for the above-mentioned aspect categories respectively, subtasks can be established for each aspect category, among which, the sentiment word "good" corresponding to the aspect category FOOD in subtask 1 can be analyzed to obtain the second classification result of subtask 1 as "positive", and the sentiment word "expensive" corresponding to the aspect category PRICE in subtask 2 can be analyzed to obtain the second classification result of subtask 2 as "negative".

[0075] Step S303: Determine a classification result corresponding to the text to be classified based on the first classification result and the second classification result.

[0076] In an embodiment of the present invention, a multi-level analysis can be performed on the target entity through the first classification result and the second classification result to obtain a classification result, thereby achieving comprehensive identification and prediction of elements at different levels and granularities, and providing more comprehensive and accurate results for element analysis.

[0077] For example, in the medical field, by semantically classifying patients' medical records, we can better understand the development of the disease and the effectiveness of treatment; in the field of social media analysis, by semantically classifying users' interaction data on social media, we can more accurately understand users' interests and needs; in the financial field, by semantically classifying various indicators in a company's financial report, we can better reveal the company's operating conditions; in the field of sentiment analysis, by classifying the semantic levels of sentiment words, we can more finely understand the sentiment tendencies in the text.

[0078] It can be seen from the above description that in an embodiment of the present invention, the target entity in the text to be classified can first be predicted based on the first prediction model to obtain the first classification result of the category to which the target entity belongs. Then, the aspect category corresponding to the target entity can be extracted based on the second prediction model, and predictions can be made based on the aspect categories to obtain the second classification result, wherein the aspect category is a predefined attribute category for evaluating the target entity. Next, the classification result corresponding to the text to be classified can be determined based on the first classification result and the second classification result, thereby completing the coarse-grained element classification prediction at the target entity level and the fine-grained requirement classification prediction at the aspect entity level at the same time, realizing the comprehensive identification and prediction of elements of different levels and different granularities, providing more comprehensive and accurate results for element analysis, improving the accuracy of classification prediction while improving the flexibility of the model, and meeting the user's usage needs.

[0079] In this embodiment, another text classification method is provided, which can be used in servers, personal computers, mobile devices, and embedded systems, etc. Figure 4is a flowchart of another text classification method according to an embodiment of the present invention. Figure 4 As shown, the process includes the following steps:

[0080] Step S401, predicting a target entity in the text to be classified based on a first prediction model, and obtaining a first classification result of the category to which the target entity belongs.

[0081] Specifically, the first prediction model includes: a self-attention mechanism and a second interactive attention mechanism, and the step S401 includes:

[0082] Step S4011, based on the self-attention mechanism, extract the semantic features in the text to be classified, and determine the target entity according to the semantic features.

[0083] Step S4012: Based on the second interactive attention mechanism, predict the first classification result of the category to which the target entity belongs.

[0084] In the embodiment of the present invention, Figure 5 The schematic diagram of the above classification prediction model is shown, wherein the main task learning module includes the above first prediction model, and the auxiliary task learning module includes the above second prediction model. The first prediction model includes a self-attention mechanism (i.e. Figure 1 The first prediction model is SA, hereinafter referred to as SA), the second interactive attention mechanism (cross-attention, i.e. Figure 1 The CA in the first prediction model is referred to as CA hereafter).

[0085] After obtaining the context embedding (containing the word vectors corresponding to each word in the text to be classified) obtained by BERT (Bidirectional Encoder Representations from Transformers), SA can be used to calculate the relationship between each word and all other words in the text. Specifically, this can be done by calculating an "attention score", which measures the importance of each word to other words. These scores are then normalized and used to weighted average all word vectors to obtain a new context vector.

[0086] Next, PCT (point-wise convolution transformation) can be used to process the temporal information of the context vector to obtain context features (i.e., the semantic features mentioned above), and identify the target entity based on the context features, where PCT is a technique that assumes that time is piecewise constant, i.e., the change in time is constant within each time period. This can be achieved by encoding the temporal information as an additional feature vector and adding it to the context vector.

[0087] After the target entity is identified, the target entity can be classified and predicted. Specifically, the target entity can be processed by the above BERT to obtain a target embedding containing the word vector of the target entity. Then, the semantic association between the target embedding and other words in the text to be processed can be calculated by CA, that is, the associated semantics between the two texts can be learned and extracted from the word level. Next, the temporal information of the target embedding can be processed by the above PCT to reduce the complexity of the model and obtain the target feature corresponding to the target entity, so as to predict the first classification result of the category to which the target entity belongs based on the target feature.

[0088] Specifically, the calculation formula of CA is as follows:

[0089] Among them, K and Q are matrices composed of word vectors, k i ,q j To calculate the two word vectors of semantic association, it means learning contextual semantics from the word level, tanh is the activation function, and softmax is the weight normalization function.

[0090] In addition, the PCT calculation formula is as follows:

[0091] Among them, W pc is the weight matrix, b pc are bias vectors, which are all learnable parameters in the model training process.

[0092] In an embodiment of the present invention, the first prediction model includes: a multi-head attention mechanism, and the step S4012 includes:

[0093] Step a1: When the target entity contains multiple sub-entities, the target features of each sub-entity are extracted through the second interactive attention mechanism.

[0094] Step a2: Based on the multi-head attention mechanism, predict the classification result of the category to which each target feature belongs to obtain the first classification result.

[0095] In the embodiment of the present invention, it is considered that the target entity may include multiple sub-entities. For example, when the target entity is XX Camera, the target entity includes the sub-entity "XX (brand)" and the sub-entity "Camera". It should be understood that the contribution of each sub-entity in the target entity to the semantic expression of the entire target entity may also be different. For example, when discussing the quality of the camera, the semantics of "Camera" is more critical than "XX". Therefore, the semantic association relationship between the elements of the entity and the context can be learned from multiple angles and granularities through the multi-head attention mechanism (hereinafter referred to as MA).

[0096] Specifically, multiple attention heads can be pre-designed in the above MA, each head is responsible for a sub-entity. Then, all heads can be processed in parallel to extract features from different angles to output the target sentence vector corresponding to each sub-entity. Next, the target sentence features can be processed by softmax to obtain the first classification result corresponding to the target entity.

[0097] Here, the calculation formula of the above MA is as follows:

[0098] MA(K,Q)=[CA 1 ,CA 2 ,...,CA nh ]·W ma , among which, CA 1 ,CA 2 ,...,CA nh is the attention head W in MA ma is the weight matrix, which is a learnable parameter in the model training process.

[0099] Step S402: extract the aspect categories corresponding to the target entity based on the second prediction model, and perform predictions based on the aspect categories to obtain a second classification result, wherein the aspect categories are predefined attribute categories used to evaluate the target entity. Figure 1 Step S102 of the illustrated embodiment will not be described in detail here.

[0100] Step S403: Determine the classification result corresponding to the text to be classified based on the first classification result and the second classification result. Figure 1 Step S103 of the illustrated embodiment will not be described in detail here.

[0101] In the embodiment of the present invention, the first prediction model can be used to perform coarse-grained element classification prediction on the target entity in the text to be classified, thereby providing a technical basis for realizing multi-level and multi-granularity analysis of the target entity.

[0102] In this embodiment, another text classification method is provided, which can be used in servers, personal computers, mobile devices, and embedded systems, etc. Figure 6 is a flowchart of another text classification method according to an embodiment of the present invention. Figure 6 As shown, the process includes the following steps:

[0103] Step S601: predict the target entity in the to-be-classified text based on the first prediction model to obtain a first classification result of the category to which the target entity belongs. Figure 1 Step S101 of the illustrated embodiment will not be described in detail here.

[0104] Step S602: extracting aspect categories corresponding to the target entity based on the second prediction model, and performing predictions based on the aspect categories to obtain a second classification result, wherein the aspect categories are predefined attribute categories for evaluating the target entity.

[0105] Specifically, the second prediction model includes: a first interactive attention mechanism, and the step S602 includes:

[0106] Step S6021, obtaining the text to be classified.

[0107] Step S6022: Identify the aspect category of the text to be classified based on the pre-trained model.

[0108] Step S6023, identifying the text to be classified based on the start identifier and the end identifier to obtain a first identification result, and vectorizing the first identification result through a preset matrix to obtain a text vector.

[0109] Step S6024: Identify the aspect category based on the start identifier and the end identifier to obtain a second identification result, and vectorize the second identification result through a preset matrix to obtain a category vector.

[0110] In an embodiment of the present invention, the pre-trained model may be the above-mentioned BERT model, that is, the input layer of the classification prediction model, which may convert the input text into a vector form, thereby mapping words from the semantic space to the vector space.

[0111] Here, BERT can first identify the text to be classified based on the start identifier and the end identifier to obtain the first identification result, and then learn and supervise the vectorization of the entire text semantics from the text head and tail respectively through these two marks, where the start identifier can be "[CLS]" and the end identifier can be "[SEP]".

[0112] For example, if the text to be classified includes the example sentence "The food is good, but a little expensive", then the first identification result corresponding to the example sentence can be expressed as "[CLS]+The food is good, but a little expensive.+[SEP]".

[0113] Next, the first identification result can be vectorized by a preset matrix to obtain a text vector. Specifically, the first identification result can be represented as a text sequence s={} with a length of l, and the text can be obtained after passing through the preset matrix A to obtain a text vector X={}.

[0114] Similarly, the aspect category is identified based on the start identifier and the end identifier to obtain a second identification result, and the second identification result is vectorized through a preset matrix to obtain a category vector. Figure 5 Taking the example sentence “The sandwitch is delicious, but the waiter are rude” in FIG. 1 as an example, the aspect categories of this sentence for an object may include “FOOD (food)”, “SERVICE (service)”, “AMBIENCE (environment)”, etc.

[0115] Specifically, the method of identifying the aspect category to obtain the second identification result is similar to the process of determining the first identification result, which will not be repeated here. Here, if N aspect entities A = {}, the second identification result corresponding to the aspect entity is vectorized by a preset matrix to obtain the category vector X a ={}.

[0116] It should be understood that the above BERT can also vectorize the third representation result of the above target entity to obtain an entity vector. Specifically, the method of identifying the target entity to obtain the third identification result is similar to the process of determining the first identification result, which will not be repeated here. Here, for the aspect entity T with a length of M i ={}The vectorization result can be the entity vector X t ={}, where T i It can be a subsequence of the above text sequence s.

[0117] Step S6025, establishing subtasks based on aspect categories.

[0118] Step S6026, based on the first interactive attention mechanism, extract the target sentiment word vector in the text vector that matches the aspect category of the subtask.

[0119] Step S6027, analyzing the subtask based on the target sentiment word vector to obtain the sub-classification result of the subtask.

[0120] Step S6028, determining a second classification result according to the sub-classification result.

[0121] In the embodiment of the present invention, Figure 5 As shown in , a corresponding subtask can be established for each aspect category to obtain the sub-classification result corresponding to the subtask. When executing the subtask corresponding to each aspect category, a multi-layer attention mechanism can be used, and each layer uses the output of the previous layer as the query input of the next layer, thereby gradually refining the category features. Here, the first interactive attention mechanism can include two CA processes, that is, a double-layer attention mechanism is used.

[0122] Here, subtask 1 corresponding to the above aspect category FOOD is taken as an example. First, the semantic association between the embedding of category 1 and other words in the text to be processed can be calculated through CA to obtain the category 1 feature, and the category 1 feature is used as the query input of the next CA to obtain the category 1 sentence feature. Then, the category 1 sentence feature can be processed through softmax as described above to obtain the sub-classification result corresponding to the aspect category FOOD.

[0123] Next, the above step S6026, based on the first interactive attention mechanism, extracts the target sentiment word vector that matches the aspect category of the subtask in the text vector, including:

[0124] Step b1, based on the first interactive attention mechanism, extract the category features of the aspect category.

[0125] Step b2: extract sentence features that match the category features in the text vector, and determine the target sentiment word vector based on the sentence features.

[0126] In the embodiment of the present invention, the category feature is the above Figure 5 The category 1 features, ..., category N features shown in , and the target sentiment word vector are included in the above Figure 5 Among the category 1 sentence features,…, category N sentence features shown in , that is, the category features are the output results of the first CA corresponding to the subtask, and the category sentence features containing the target sentiment word vector are the output results of the second CA corresponding to the subtask.

[0127] Step S603: Determine the classification result corresponding to the text to be classified based on the first classification result and the second classification result. Figure 1 Step S103 of the illustrated embodiment will not be described in detail here.

[0128] In the embodiment of the present invention, the second prediction model can be used to perform fine-grained element classification prediction on the aspect type corresponding to the target entity in the classified text, thereby providing a technical basis for realizing multi-level and multi-granularity analysis of the target entity.

[0129] In some optional implementations, the above step S2022 includes:

[0130] Step c1, analyzing the weights of subtasks based on their aspect categories.

[0131] Step c2, performing weighted summation on the sub-classification results according to the weights to obtain a second classification result.

[0132] In the embodiment of the present invention, the weight parameters of the classification prediction model can be adjusted according to the corresponding classification field. Figure 5 Taking the corresponding sentiment analysis field as an example, the weight of the aspect category "FOOD" can be set to the highest, for example, to 50%, followed by the weight of the aspect category "SERVICE", for example, to 25%, and then the weight of the aspect category "AMBIENCE", for example, to 25%.

[0133] Here, if the sub-classification result corresponding to the aspect category "FOOD" is A1, the sub-classification result corresponding to the aspect category "SERVICE" is A2, and the sub-classification result corresponding to the aspect category "AMBIENCE" is A3, then the second classification result can be expressed as: 0.5A1+0.25A2+0.25A3.

[0134] In the embodiment of the present invention, the sub-classification results of the sub-tasks corresponding to each aspect category may be weighted and summed, thereby making full use of the data set of fine-grained factor analysis in a complex context and improving the accuracy of the fine-grained classification results.

[0135] In some optional embodiments, the above Figure 1 The corresponding embodiments also include:

[0136] Step S11, determining a first loss function corresponding to the first prediction model and a second loss function corresponding to the second prediction model according to the cross entropy loss function.

[0137] Step S12, constructing a target loss function based on the first loss function and the second loss function, wherein the target loss function is used to guide the optimization of the classification result.

[0138] In the embodiment of the present invention, the first loss function corresponding to the first prediction model determined according to the cross entropy loss function can be expressed as: In addition, the second loss function corresponding to the second prediction model determined by the cross entropy loss function can be expressed as:

[0139] On this basis, the constructed target loss function can be expressed as Among them, λ and θ are preset as constants.

[0140] In an embodiment of the present invention, since the above-mentioned classification prediction model includes a first prediction model and a second prediction model, the loss functions of the two models can be calculated separately, and the first loss function and the second loss function can be comprehensively considered to construct a target loss function, so as to better train the classification prediction model, measure model performance and optimize model parameters, thereby improving the accuracy of the output results of the classification prediction model.

[0141] In summary, in an embodiment of the present invention, the target entity in the text to be classified can first be predicted based on the first prediction model to obtain the first classification result of the category to which the target entity belongs. Then, the aspect category corresponding to the target entity can be extracted based on the second prediction model, and predictions can be made based on the aspect categories to obtain the second classification result, wherein the aspect category is a predefined attribute category for evaluating the target entity. Next, the classification result corresponding to the text to be classified can be determined based on the first classification result and the second classification result, thereby completing the coarse-grained element classification prediction at the target entity level and the fine-grained requirement classification prediction at the aspect entity level at the same time, realizing the comprehensive identification and prediction of elements of different levels and granularities, providing more comprehensive and accurate results for element analysis, improving the accuracy of classification prediction while improving the flexibility of the model, and meeting the user's usage needs.

[0142] In this embodiment, a text classification device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0143] This embodiment provides a text classification device, such as Figure 7 As shown, including:

[0144] A first prediction module 701 is used to predict a target entity in the to-be-classified text based on a first prediction model, and obtain a first classification result of the category to which the target entity belongs;

[0145] A second prediction module 702 is used to extract the aspect categories corresponding to the target entity based on the second prediction model, and perform predictions based on the aspect categories to obtain a second classification result, wherein the aspect categories are predefined attribute categories used to evaluate the target entity;

[0146] The determination module 703 is used to determine the classification result corresponding to the text to be classified based on the first classification result and the second classification result.

[0147] In some optional implementations, the second prediction model includes: a first interactive attention mechanism; the second prediction module 702 is further used to:

[0148] Get the text to be classified;

[0149] Based on the pre-trained model, identify the aspect categories of the text to be classified;

[0150] The text to be classified is identified based on the start identifier and the end identifier to obtain a first identification result, and the first identification result is vectorized through a preset matrix to obtain a text vector;

[0151] Identify the aspect category based on the start identifier and the end identifier to obtain a second identification result, and vectorize the second identification result through a preset matrix to obtain a category vector;

[0152] Based on the aspect categories, subtasks are established;

[0153] Based on the first interactive attention mechanism, extract the target sentiment word vector that matches the aspect category of the subtask in the text vector;

[0154] Analyze the subtasks based on the target sentiment word vectors to obtain the sub-classification results of the subtasks;

[0155] A second classification result is determined according to the sub-classification result.

[0156] In some optional implementations, the second prediction module 702 is further configured to:

[0157] Analyze the weights of subtasks based on their aspect categories;

[0158] The sub-classification results are weighted and summed according to the weights to obtain the second classification result.

[0159] In some optional implementations, the second prediction module 702 is further configured to:

[0160] Based on the first interactive attention mechanism, the category features of the aspect categories are extracted;

[0161] Sentence features that match the category features are extracted from the text vector, and the target sentiment word vector is determined based on the sentence features.

[0162] In some optional implementations, the first prediction model includes: a self-attention mechanism and a second interactive attention mechanism; the first prediction module 701 is further used to:

[0163] Based on the self-attention mechanism, the semantic features in the text to be classified are extracted, and the target entity is determined according to the semantic features;

[0164] Based on the second interactive attention mechanism, predict the first classification result of the category to which the target entity belongs.

[0165] In some optional implementations, the first prediction model includes: a multi-head attention mechanism; the first prediction module 701 is further used to:

[0166] When the target entity contains multiple sub-entities, the target features of each sub-entity are extracted through the second interactive attention mechanism;

[0167] Based on the multi-head attention mechanism, the classification result of each target feature is predicted to obtain the first classification result.

[0168] In some optional embodiments, the device is also used for:

[0169] Determine a first loss function corresponding to the first prediction model and a second loss function corresponding to the second prediction model according to the cross entropy loss function;

[0170] Based on the first loss function and the second loss function, a target loss function is constructed, wherein the target loss function is used to guide the optimization of the classification result.

[0171] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0172] The text classification device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0173] The embodiment of the present invention also provides a computer device having the above Figure 7 The text classification device shown.

[0174] See also Figure 8 , Figure 8 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Figure 8As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 8 A processor 10 is taken as an example.

[0175] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0176] The memory 20 stores instructions executable by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.

[0177] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0178] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0179] The computer device further comprises a communication interface 30 for the computer device to communicate with other devices or a communication network.

[0180] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0181] A part of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of the computer program instruction in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc., and accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.

[0182] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A text classification method, characterized in that: The method comprises: Predicting a target entity in the to-be-classified text based on a first prediction model to obtain a first classification result of the category to which the target entity belongs; Extracting aspect categories corresponding to the target entity based on the second prediction model, and performing predictions based on the aspect categories respectively to obtain second classification results, wherein the aspect categories are predefined attribute categories for evaluating the target entity; Based on the first classification result and the second classification result, a classification result corresponding to the text to be classified is determined.

2. The method according to claim 1, characterized in that The second prediction model includes: a first interactive attention mechanism; The extracting the aspect categories corresponding to the target entity based on the second prediction model and performing predictions based on the aspect categories to obtain a second classification result includes: Get the text to be classified; Based on the pre-trained model, identifying the aspect category of the text to be classified; Identify the text to be classified based on the start identifier and the end identifier to obtain a first identification result, and vectorize the first identification result through a preset matrix to obtain a text vector; Identify the aspect category based on the start identifier and the end identifier to obtain a second identification result, and vectorize the second identification result through a preset matrix to obtain a category vector; Based on the aspect categories, subtasks are established; Based on the first interactive attention mechanism, extracting a target sentiment word vector that matches the aspect category of the subtask in the text vector; Analyzing the subtask based on the target sentiment word vector to obtain a subclassification result of the subtask; The second classification result is determined according to the sub-classification result.

3. The method according to claim 2, characterized in that The determining the second classification result according to the sub-classification result includes: Analyzing weights of the subtasks based on the aspect categories of the subtasks; The sub-classification results are weighted and summed according to the weights to obtain the second classification result.

4. The method according to claim 2, characterized in that: The step of extracting a target sentiment word vector matching the aspect category of the subtask from the text vector based on the first interactive attention mechanism includes: Extracting category features of the aspect category based on the first interactive attention mechanism; Sentence features matching the category features are extracted from the text vector, and the target sentiment word vector is determined according to the sentence features.

5. The method according to claim 1, characterized in that The first prediction model includes: a self-attention mechanism and a second interactive attention mechanism; The step of predicting a target entity in the to-be-classified text based on the first prediction model to obtain a first classification result of the category to which the target entity belongs includes: Based on the self-attention mechanism, extracting semantic features in the text to be classified, and determining the target entity according to the semantic features; Based on the second interactive attention mechanism, predict a first classification result of the category to which the target entity belongs.

6. The method according to claim 5, characterized in that The first prediction model includes: a multi-head attention mechanism; The predicting, based on the second interactive attention mechanism, a first classification result of the category to which the target entity belongs, comprises: When the target entity includes multiple sub-entities, extracting target features of each of the sub-entities through the second interactive attention mechanism; Based on the multi-head attention mechanism, the classification result of the category to which each of the target features belongs is predicted to obtain the first classification result.

7. The method according to claim 1, characterized in that The method further comprises: Determine a first loss function corresponding to the first prediction model and a second loss function corresponding to the second prediction model according to the cross entropy loss function; Based on the first loss function and the second loss function, a target loss function is constructed, wherein the target loss function is used to guide the optimization of the classification result.

8. A text classification device, characterized in that: The device comprises: A first prediction module, configured to predict a target entity in the to-be-classified text based on a first prediction model, and obtain a first classification result of a category to which the target entity belongs; A second prediction module, configured to extract aspect categories corresponding to the target entity based on a second prediction model, and perform predictions based on the aspect categories to obtain a second classification result, wherein the aspect categories are predefined attribute categories for evaluating the target entity; A determination module is used to determine a classification result corresponding to the text to be classified based on the first classification result and the second classification result.

9. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the text classification method according to any one of claims 1 to 7 by executing the computer instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the text classification method according to any one of claims 1 to 7.

11. A computer program product, characterized in that The method comprises computer instructions, wherein the computer instructions are used to cause a computer to execute the text classification method according to any one of claims 1 to 7.

Citation Information

Patent Citations

  • Method and device for analyzing emotional tendency

    CN102236636A

  • Specific aspect sentiment analysis method of multi-granularity attention model based on Transformer

    CN112232087A

  • Data processing method and device and storage medium

    CN113254635A

  • Aspect-level sentiment classification method based on enhanced semantic syntactic information

    CN114676687A

  • Chinese attribute level sentiment analysis method, system and equipment based on multi-task learning and medium

    CN116029305A

Cited By

  • Text classification method and device, equipment, storage medium and program product

    CN119493860A

  • Object category identification method and device based on multi-modal large model, and object category display method and device based on multi-modal large model

    CN121598215A