Comment information classification method and device, electronic equipment and storage medium
By obtaining text vector representations of comment information, mining entities and topics, constructing heterogeneous information graphs and performing convolution operations, and training a classification model, the problems of feature sparsity and high noise in comment information classification are solved, and the classification accuracy is improved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- AGRICULTURAL BANK OF CHINA
- Filing Date
- 2022-12-08
- Publication Date
- 2026-06-02
AI Technical Summary
Comment information exists in the form of short texts, and the sparse features and high noise result in low classification accuracy.
By obtaining text vector representations of sample comment information, entities and topics are mined, a heterogeneous information graph is constructed, and convolution operations are performed to train a comment information classification model.
It improves the accuracy of comment information classification and solves the problems of feature sparsity and high noise.
Smart Images
Figure CN115757911B_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present invention relate to data processing technology, and more particularly to a method, apparatus, electronic device and storage medium for classifying comment information. Background Technology
[0002] With the development of social networks, tens of thousands of comments are generated every day, containing various information such as users' potential needs and interests. For businesses, these comments involve evaluations and feedback on the services, products, and businesses they provide, especially negative comments. Analyzing and categorizing this information can help businesses understand customer pain points and diverse needs, thereby facilitating targeted improvements.
[0003] These comments are usually short texts, characterized by sparse features and high noise, which makes classifying these comments extremely difficult and results in low accuracy. Summary of the Invention
[0004] This invention provides a method, apparatus, electronic device, and storage medium for classifying comment information, which can solve the problems of sparse features and high noise, and improve the accuracy of classification results.
[0005] In a first aspect, the present invention provides a method for classifying comment information, including:
[0006] Obtain the text vector representation of sample comment information;
[0007] Mine the entities and topics represented by the text vectors, and construct a heterogeneous information graph based on the entities, the topics, and the text vector representations;
[0008] Perform convolution operation on the heterogeneous information graph to obtain the text node representation of the sample comment information;
[0009] The text node representations are used to train the model, resulting in a comment information classification model.
[0010] The target comment information is classified using the aforementioned comment information classification model.
[0011] Secondly, the present invention provides a comment information classification device, comprising:
[0012] The vector acquisition module is used to acquire the text vector representation of sample comment information;
[0013] A heterogeneous graph construction module is used to mine the entities and topics represented by the text vectors, and construct a heterogeneous information graph based on the entities, the topics, and the text vector representations.
[0014] The convolution operation module is used to perform convolution operations on the heterogeneous information graph to obtain the text node representation of the sample comment information;
[0015] The model training module is used to train the model using the text node representations to obtain a comment information classification model;
[0016] The classification module is used to classify target comment information using the comment information classification model.
[0017] Thirdly, the present invention provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the comment information classification method as described in any embodiment of the present invention.
[0018] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the comment information classification method as described in any embodiment of the present invention.
[0019] In this invention, text vector representations of sample comment information are obtained; entities and topics within the text vector representations are mined, and a heterogeneous information graph is constructed based on the entities, topics, and text vector representations; convolution operations are performed on the heterogeneous information graph to obtain text node representations of the sample comment information; the text node representations are used for model training to obtain a comment information classification model; and the comment information classification model is used to classify target comment information. In other words, this invention can vectorize samples, construct heterogeneous information graphs based on entities and topics mined from the samples, and achieve semantic enhancement of samples from multiple dimensions, solving the problems of feature sparsity and high noise. Based on the convolution operation of the heterogeneous information graph, features required for model training are obtained, and these features are used to train a model for classifying comment information, which can improve the accuracy of the classification results. Attached Figure Description
[0020] To more clearly illustrate the technical solution of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.
[0021] Figure 1 This is a flowchart illustrating the comment information classification method provided by the present invention;
[0022] Figure 2 This is another flowchart illustrating the comment information classification method provided by the present invention;
[0023] Figure 3a This is an example diagram of a fusion text representation method for the comment information classification method provided by the present invention;
[0024] Figure 3b This is an example diagram of the model training process for the comment information classification method provided by this invention;
[0025] Figure 4 This is a schematic diagram of the comment information classification device provided by the present invention;
[0026] Figure 5 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation
[0027] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.
[0028] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.
[0029] Before introducing the solution of the present invention, the relevant terms involved in the embodiments of the present invention will be explained:
[0030] Few-shot learning: Few-shot learning is a learning technique that uses only a small number of labeled samples to train a model. It can classify and recognize samples with a small number of labeled samples and has the ability to quickly understand new concepts and generalize them.
[0031] Data augmentation: Data augmentation is a technique used to generate new training samples. These techniques are generally achieved by randomly jittering and perturbing the original data, while the class labels remain unchanged.
[0032] Semantic enhancement: To address the issues of semantic sparsity and insufficient semantic feature extraction in short texts, methods such as introducing external knowledge to enrich the text's feature representation and extracting latent topic information can be used to enhance the semantic features of short texts. These methods can be considered semantic enhancement methods.
[0033] XLNet: XLNet is a general autoregressive pre-training method that combines the advantages of autoregressive language models and autoencoder language models. It learns bidirectional contextual information and has excellent performance on a variety of natural language processing tasks.
[0034] Heterogeneous graph convolutional networks: Heterogeneous graph convolutional networks are used to extract spatial features on graphs with non-Euclidean structures, which are heterogeneous graphs composed of various types of nodes and edges.
[0035] Figure 1 This is a flowchart illustrating a comment information classification method provided by the present invention. This method can be executed by a comment information classification device provided by the present invention, which can be implemented using software and / or hardware. In a specific embodiment, the device can be integrated into an electronic device, such as a computer or server. The following embodiments will illustrate this using the integration of the device into an electronic device as an example. Figure 1 The method may specifically include the following steps:
[0036] Step 101: Obtain the text vector representation of the sample comment information.
[0037] The sample comment information can be a small sample short text dataset D = {d1,…,d2} containing a small number of labeled samples and a large number of unlabeled samples. l ,…,d n The first l samples are labeled samples and undergo data preprocessing. Data augmentation is performed on a small number of labeled samples, and the short text is converted into an effective text vector representation matrix. First, a data augmentation algorithm that integrates word importance is used to augment the labeled samples, appropriately increasing the number of labeled samples. Second, a semantic augmentation algorithm that integrates XLNet multi-layer features is used to convert the augmented labeled samples and a large number of unlabeled samples into a short text representation matrix in vector form, which serves as the text vector representation of the sample comment information.
[0038] Step 102: Mine the entities and topics represented by the text vectors, and construct a heterogeneous information graph based on the entities, topics and text vector representations.
[0039] Specifically, entity information and latent topics are mined from short text datasets, and these are used as additional semantic features of the short texts. A heterogeneous information graph of short texts is constructed by fusing entity information, latent topics, and text representations from the representation layer. Three edge types—text-entity, text-topic, and entity-entity—can be constructed using short texts, entities, and topics as nodes. Furthermore, a heterogeneous information graph is built using short texts, entities, and topics as nodes and text-entity, text-topic, and entity-entity edges.
[0040] Step 103: Perform convolution operation on the heterogeneous information graph to obtain the text node representation of the sample comment information.
[0041] Specifically, convolution operations are performed on the heterogeneous short text infographic to learn effective text node representations, thereby improving classification performance. For the heterogeneous short text infographic output in step 102, a random neighbor removal strategy is used to increase the diversity of the infographic, thus reducing model convergence speed, alleviating overfitting, and obtaining the adjacency matrix A, degree matrix D, and short text feature matrix X. The traditional graph convolution propagation rule is improved by fully considering message propagation of various types of nodes and relationships, achieving effective updating of text node features in the heterogeneous infographic and further alleviating the small sample size problem. An attention weight matrix is constructed based on a dual attention mechanism, where the dual-layer attention mechanism simultaneously incorporates type-level and node-level attention mechanisms for calculation. After learning through the dual-layer heterogeneous graph convolution operation, the text node representations of the sample comment information are obtained.
[0042] Step 104: Use text node representations to train the model and obtain the comment information classification model.
[0043] Specifically, the model's output layer outputs the class labels for n nodes using the softmax function, Y. i This layer represents the category label of node i (i = 1, 2, ..., n). The input to this layer is the text node representation learned by the graph convolutional layer, and the output is the category labels of all text nodes. The model is trained using these text node representations. Training stops when a certain cutoff condition is met, and the model updated in the last round is used as the comment information classification model. The cutoff condition may be, for example, reaching a preset number of training rounds or the training loss falling below a certain value.
[0044] Step 105: Classify the target comment information using a comment information classification model.
[0045] Specifically, the target comment information can be any comment targeting a company, business, or process obtained from a social media platform. The target comment information is input into a comment classification model. First, data augmentation is performed by fusing word importance, followed by semantic augmentation by fusing XLNet multi-layer features to obtain a text vector representation. Next, entity information is extracted from the text vector representation, and topic information is mined. A heterogeneous information graph is constructed based on text, entities, and topics. Random neighbor removal is performed on the heterogeneous information graph to obtain an adjacency matrix, degree matrix, and short text feature matrix. Then, heterogeneous graph convolution is performed based on a dual attention mechanism. Finally, the target comment classification output is obtained, and the output category can be positive or negative. If the target comment information is already negative, these negative comments can be further classified, such as mild, moderate, or severe negative comments, depending on the category labels labeled during model training.
[0046] The present invention obtains text vector representations of sample comment information; mines entities and topics from the text vector representations, and constructs a heterogeneous information graph based on entities, topics, and text vector representations; performs convolution operations on the heterogeneous information graph to obtain text node representations of the sample comment information; trains a model using the text node representations to obtain a comment information classification model; and classifies the target comment information using the comment information classification model. In other words, the present invention can vectorize samples, construct a heterogeneous information graph based on entities and topics mined from the samples, and achieve semantic enhancement of samples from multiple dimensions, solving the problems of feature sparsity and high noise. Based on the convolution operation of the heterogeneous information graph, it obtains the features required for model training, and uses these features to train a model for classifying comment information, which can improve the accuracy of the classification results.
[0047] Figure 2 This is another flowchart illustrating the comment information classification method provided by the present invention, such as... Figure 2 As shown, the method may include the following steps:
[0048] Step 201: Identify keywords and non-keywords in the labeled sample comment information.
[0049] Here, sample comment information refers to pre-labeled sample comments randomly obtained from various social media platforms. Keywords are words that express the central content of the text, while non-keywords are other less significant, non-essential words. In practice, keywords play a crucial role in the semantics of short texts. Distinguishing keywords and using different random augmentation strategies can effectively reduce introduced noise while maintaining the text labels unchanged. Therefore, designing a data augmentation algorithm that integrates word importance is important for fully considering the importance of different words and reducing randomly injected noise during data augmentation.
[0050] Step 202: Use random insertion to augment the keywords and random replacement and / or random deletion to augment the non-keywords to obtain the first sample comment information.
[0051] Specifically, for short text corpora, keyword extraction tools are used to extract keywords from short texts to form a keyword list. For example, the YAKE tool can be used. Then, keywords are randomly inserted, non-keywords are randomly replaced and / or randomly deleted, and all non-stop words are randomly swapped. Through these four random operations, the small sample short text data is enhanced to obtain the first sample comment information.
[0052] Step 203: Use a multi-layer feature extractor of the language understanding model to extract features from the first sample comment information and the unlabeled sample comment information to obtain multi-layer features.
[0053] The language understanding model can be the XLNet model. A multi-layered feature extractor from the language understanding model is used to extract features from the first sample comment information and the unlabeled sample comment information. For example, the multi-layered feature extractor can include layers 3, 7, and 11 of XLNet. After extraction, the feature encoding output by the multi-layered encoder in the XLNet model, which stores phrase-level information, syntactic information, and semantic information, is obtained.
[0054] Step 204: Fuse the multi-layer features to obtain the second sample comment information.
[0055] Specifically, the text representation method that integrates XLNet multi-layer features differs from methods that only use a single layer of feature encoding as text features. This method integrates the feature encoding outputs of the multi-layer encoders in the XLNet model, which store phrase-level information, syntactic information, and semantic information. By concatenating the three layers of feature encodings, a text feature matrix is obtained, and a one-dimensional max pooling operation is performed on the text feature matrix to retain key features, thereby obtaining the second sample comment information that integrates XLNet multi-layer features.
[0056] Step 205: Vectorize the second sample comment information to obtain a text vector representation.
[0057] For example, text representation methods that fuse XLNet multi-layer features, such as... Figure 3a As shown, the input text sequence is d = {token1, ..., token} i ,…,token m}, for any token in it i Each Transformer-XL feature extractor extracts a feature matrix. By concatenating the output vectors of Transformer-XL layers 3, 7, and 11, a token can be obtained.i The multi-layer feature fusion matrix is subjected to one-dimensional max pooling to retain the key features of each feature vector, thus obtaining the token. i The vector matrix of all tokens is obtained by averaging the vector matrices. This yields the text vector of the input text d after fusing XLNet multi-layer features, which is the text vector representation matrix of the short text d.
[0058] Step 206: Mine the entities and topics represented by the text vectors, and construct the first type of edges based on the entities contained in the text vector representations. The first type of edges are the edges from text to entities.
[0059] The short text is based on the text representation matrix obtained in step 205, and the entities are entity information obtained from the text using the TagMe tool. Specifically, entities are considered key information in the short text, and text-entity edge types are constructed as the first type of edge to enhance the model's attention to key information when learning the semantics of the short text. If the short text contains k entities, then the short text nodes will have k edges connecting them to the entity nodes.
[0060] Step 207: Construct a second type of edge based on the topics contained in the text vector representation. The second type of edge is the edge from the text to the topic.
[0061] Here, the topic is the latent topic information of the short text obtained based on the BTM topic model. The topic is the extraction and abstraction of the important semantics of the short text. The text-topic edge type is constructed as the second type of edge to further expand the semantic features of the short text and solve the problem of feature sparsity of the short text. The top p topics with the highest probability belonging to the short text are selected as the latent topics of the short text, that is, each short text node will have p edges connected to the topic nodes.
[0062] Step 208: Construct a third type of edge based on the similarity between entities. The third type of edge is the edge from one entity to another.
[0063] Specifically, when the similarity between entity nodes is higher than a predetermined threshold, an edge is established between the two entities to merge more semantic information, resulting in a third type of edge.
[0064] Step 209: Use text vector representations, entities, and topics as nodes, and use first-type edges, second-type edges, and third-type edges as edges to construct a heterogeneous information graph.
[0065] Specifically, heterogeneous information graphs can be constructed using the Short Text Heterogeneous Information Network (STHIN). The STHIN uses short texts, entities, and topics as nodes, and first-class, second-class, and third-class edges as edges to construct the heterogeneous information graph.
[0066] Step 210: Randomly remove edges from the heterogeneous information graph to obtain the target heterogeneous graph.
[0067] Specifically, due to the limited number of labeled samples in few-shot learning, the model extracts fewer data features during learning, weakening its generalization ability and making it prone to overfitting. To alleviate overfitting and oversmoothing issues, edges are randomly removed from the short text heterogeneous information graph, and an appropriate amount of noise is introduced to increase the randomness and diversity of the heterogeneous information network, thereby reducing the model's convergence speed and obtaining the target heterogeneous graph.
[0068] If the heterogeneous information graph STHIN=(V,E), where V={v1,v2,…,v…} n Let E be a set of nodes, where E = {e1, e2, ..., e} k Let} be the set of edges, then we can obtain the adjacency matrix A∈R. n×n Sum degree matrix D∈R n×n An adjacency matrix is a storage structure for graphs, used to represent the adjacency relationships between vertices. If the rate of random neighbor removal is r, then the new adjacency matrix is formed by A. new express:
[0069] A new =AA r
[0070] Among them, A r The elements represent the positions where non-zero elements in the adjacency matrix are randomly set to 0 with probability r. The new degree matrix is represented as D. new Introducing renormalization operation and self-connection matrix I n Adjacency matrix A new Adding self-loops allows you to summarize the features of the nodes themselves and obtain a normalized symmetric adjacency matrix.
[0071]
[0072] Step 211: Calculate the type-level attention coefficient based on the word embeddings and types of each node in the target heterogeneous graph.
[0073] Specifically, given a specific node in the STHIN short text heterogeneous information network, neighboring nodes of different types have different influences on it. For example, when entity type nodes carry more information than topic type nodes, entity type neighboring nodes should be given higher weights to provide more effective neighbor information for classification. Furthermore, different neighboring nodes of the same type also have different importance; for example, different topic nodes of the same type contain different levels of potential topic information. Therefore, to simultaneously capture the different importance at the node level and the type level, an attention mechanism is introduced.
[0074] Type-level attention mechanisms are used to learn the weights of neighboring nodes of different types, calculating type-level attention coefficients based on the word embeddings and type embeddings of the current node. Given a node v, the type embedding of its neighboring nodes of type s is represented as:
[0075]
[0076] Where N is the set of neighboring nodes of node v, and the node type is s, x v′ Word embeddings for nodes. Word embeddings x based on node v. v and type embedding r s Calculate type-level attention score a s :
[0077]
[0078] Where LeakyReLU is a non-linear activation function, η s This is the attention weight matrix for category s, where || represents the weight of x. v and r s The concatenation operation is then performed. Next, the softmax function is introduced to regularize the type-level attention scores, obtaining the type-level attention coefficient α. s .
[0079] Step 212: Calculate the node-level attention coefficient based on the type-level attention coefficient, the word embeddings of each node in the target heterogeneous graph, and the type.
[0080] Specifically, a node-level attention mechanism is used to capture the importance of different neighboring nodes and reduce the weights of noisy nodes. Based on the type-level attention coefficients, the word embeddings of the nodes are obtained, and node-level attention coefficients are calculated. Given a node v of type s, its neighboring nodes v′∈N v The word embedding x of type s′ is based on node v. v The word embedding x of neighbor node v′ v′ and type embedding r s′ Calculate the node-level attention score b vv′ :
[0081] b vv′ =LeakyReLU(ω T α s′ [x v ||x v′ ])
[0082] Here, ω is the attention weight matrix, and the softmax function is introduced to regularize the node-level attention scores to obtain the node-level attention coefficients β. vv′ .
[0083] Step 213: Determine the attention weight matrix based on the node-level attention coefficients.
[0084] Specifically, an attention weight matrix γ is constructed for the heterogeneous information network of short texts. The elements of this matrix consist of node-level attention coefficients, i.e., γ ii =β ii , representing node v j For node v i The degree of importance.
[0085] Step 214: Perform convolution operation on the target heterogeneous graph based on the attention weight matrix to obtain the text node representation of the sample comment information.
[0086] A heterogeneous graph convolution method that integrates multiple types of information integrates various types of node information and projects their respective transformation matrices into a common space, capturing various types of node information and relationships in STHIN.
[0087] To summarize the characteristics of each node, the adjacency matrix A needs to be filled with self-loops, A′=A+I. n To integrate various types of node information and project their respective transformation matrices into a common space, and to improve the traditional graph convolution propagation rules, a two-layer heterogeneous graph convolutional network for few-sample short text classification can be represented as:
[0088]
[0089]
[0090] Where S = (D, E, T) represents three node types: text, entity, and topic. Let X ∈ R be a normalized symmetric adjacency matrix of type s. n×m Given a matrix containing all nodes and their features, each row x v ∈R m W is a feature vector of a node v, where n is the number of nodes and m is the dimension of the feature vector. (0) ∈R m×k W is the weight matrix from the input layer to the hidden layer. (1) ∈R k×c This is the weight matrix from the hidden layer to the output layer, where k is the number of feature maps in the hidden layer. Unlike the traditional graph convolution propagation regularity, Consider the differences in feature spaces for different types of information. ReLU and softmax functions are the activation functions for the first and output layers, respectively, Z∈R. n×c This is the output text category information, where c is the number of categories.
[0091] The improved formula for heterogeneous graph convolution after fusing the attention weight matrix is as follows:
[0092]
[0093] An improved heterogeneous graph convolution formula can be used to perform convolution operations on the target heterogeneous graph to obtain the text node representation of the sample comment information.
[0094] Step 215: Use text node representations to train the model and obtain the comment information classification model.
[0095] Step 216: Classify the target comment information using a comment information classification model.
[0096] The present invention obtains text vector representations of sample comment information; mines entities and topics from the text vector representations, and constructs a heterogeneous information graph based on entities, topics, and text vector representations; performs convolution operations on the heterogeneous information graph to obtain text node representations of the sample comment information; trains a model using the text node representations to obtain a comment information classification model; and classifies the target comment information using the comment information classification model. In other words, the present invention can vectorize samples, construct a heterogeneous information graph based on entities and topics mined from the samples, and achieve semantic enhancement of samples from multiple dimensions, solving the problems of feature sparsity and high noise. Based on the convolution operation of the heterogeneous information graph, it obtains the features required for model training, and uses these features to train a model for classifying comment information, which can improve the accuracy of the classification results.
[0097] For example, Figure 3b This is an example diagram illustrating the model training process of the comment information classification method provided by this invention. (See diagram for example.) Figure 3b As shown, the model inputs data from a small sample short text dataset, which includes some labeled samples and some unlabeled samples. Data augmentation is performed on the sample comment information by fusing word importance, followed by semantic augmentation by fusing XLNet multi-layer features to obtain text vector representations. Next, entity information is extracted from the text vector representations based on TagMe, and topic information is mined based on the BTM topic model. A heterogeneous information graph is constructed based on text, entity, and topic features. Random neighbor removal is performed on the heterogeneous information graph to obtain the adjacency matrix, degree matrix, and short text feature matrix. Then, heterogeneous graph convolution is performed based on the attention weight matrix obtained through a dual attention mechanism. The model's output layer outputs the category labels of n nodes through a softmax function, Y. i The category label represents node i (i = 1, 2, ..., n).
[0098] Figure 4 This is a schematic diagram of a comment information classification device provided by the present invention, which is suitable for implementing the comment information classification method provided by the present invention. Figure 4 As shown, the device may specifically include:
[0099] Vector acquisition module 401 is used to acquire the text vector representation of sample comment information;
[0100] The heterogeneous graph construction module 402 is used to mine the entities and topics represented by the text vectors, and construct a heterogeneous information graph based on the entities, the topics, and the text vector representations.
[0101] The convolution operation module 403 is used to perform convolution operations on the heterogeneous information graph to obtain the text node representation of the sample comment information;
[0102] Model training module 404 is used to train the model using the text node representation to obtain a comment information classification model;
[0103] The classification module 405 is used to classify the target comment information using the comment information classification model.
[0104] In one embodiment, the sample comment information includes labeled sample comment information and unlabeled sample comment information, and the vector acquisition module 401 is specifically used for:
[0105] Data augmentation is performed on the labeled sample comment information to obtain the first sample comment information;
[0106] Semantic enhancement is performed on the first sample comment information and the unlabeled sample comment information to obtain the second sample comment information;
[0107] The second sample comment information is vectorized to obtain the text vector representation.
[0108] In one embodiment, the vector acquisition module 401 performs data augmentation on the labeled sample comment information to obtain first sample comment information, including:
[0109] Identify the keywords and non-keywords in the labeled sample comment information;
[0110] The keywords are augmented using random insertion, and the non-keywords are augmented using random replacement and / or random deletion to obtain the first sample comment information.
[0111] In one embodiment, the vector acquisition module 401 performs semantic enhancement on the first sample comment information and the unlabeled sample comment information to obtain second sample comment information, including:
[0112] A multi-layer feature extractor based on a language understanding model is used to extract features from the first sample comment information and the unlabeled sample comment information to obtain multi-layer features;
[0113] The multi-layer features are fused to obtain the second sample comment information.
[0114] In one embodiment, the heterogeneous graph construction module 402 is specifically used for:
[0115] A first type of edge is constructed based on the entities contained in the text vector representation, wherein the first type of edge is an edge from text to entity;
[0116] A second type of edge is constructed based on the topic contained in the text vector representation, whereby the second type of edge is an edge from the text to the topic;
[0117] A third type of edge is constructed based on the similarity between entities, wherein the third type of edge is an entity-to-entity edge;
[0118] The heterogeneous information graph is constructed by using the text vector representation, the entity, and the topic as nodes, and the first type of edge, the second type of edge, and the third type of edge as edges.
[0119] In one embodiment, the convolution operation module 403 is specifically used for:
[0120] Randomly remove edges from the heterogeneous information graph to obtain the target heterogeneous graph;
[0121] The target heterogeneous graph is convolved to obtain the text node representation of the sample comment information.
[0122] In one embodiment, the convolution operation module 403 performs convolution operations on the target heterogeneous graph to obtain the text node representation of the sample comment information, including:
[0123] Calculate the type-level attention coefficient based on the word embedding and type of each node in the target heterogeneous graph;
[0124] The node-level attention coefficient is calculated based on the type-level attention coefficient, the word embedding, and the type of each node in the target heterogeneous graph;
[0125] Determine the attention weight matrix based on the node-level attention coefficients;
[0126] The target heterogeneous graph is convolved with the attention weight matrix to obtain the text node representation of the sample comment information.
[0127] The apparatus of this invention can acquire text vector representations of sample comment information; mine entities and topics from the text vector representations, and construct a heterogeneous information graph based on entities, topics, and text vector representations; perform convolution operations on the heterogeneous information graph to obtain text node representations of the sample comment information; train a model using the text node representations to obtain a comment information classification model; and classify target comment information using the comment information classification model. In other words, this invention can vectorize samples, construct a heterogeneous information graph based on entities and topics mined from the samples, and achieve semantic enhancement of samples from multiple dimensions, solving the problems of feature sparsity and high noise. Based on the convolution operation of the heterogeneous information graph, it obtains the features required for model training, and uses these features to train a model for classifying comment information, which can improve the accuracy of the classification results.
[0128] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the comment information classification method provided in any of the above embodiments.
[0129] The present invention also provides a computer-readable medium having a computer program stored thereon, which, when executed by a processor, implements the comment information classification method provided in any of the above embodiments.
[0130] The following is for reference. Figure 5 It shows a schematic diagram of the structure of a computer system 500 suitable for implementing the electronic device of the present invention. Figure 5 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the invention.
[0131] like Figure 5 As shown, the computer system 500 includes a central processing unit (CPU) 501, which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) 502 or programs loaded from storage section 508 into random access memory (RAM) 503. The RAM 503 also stores various programs and data required for the operation of the computer system 500. The CPU 501, ROM 502, and RAM 503 are interconnected via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.
[0132] The following components are connected to I / O interface 505: an input section 506 including a keyboard, mouse, etc.; an output section 507 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 508 including a hard disk, etc.; and a communication section 509 including a network interface card such as a LAN card, modem, etc. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to I / O interface 505 as needed. A removable medium 511, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 510 as needed so that computer programs read from it can be installed into storage section 508 as needed.
[0133] In particular, according to the embodiments disclosed in this invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 509, and / or installed from removable medium 511. When the computer program is executed by central processing unit (CPU) 501, it performs the functions defined above in the system of this invention.
[0134] It should be noted that the computer-readable medium shown in this invention can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this invention, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.
[0135] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.
[0136] The modules and / or units described in this invention can be implemented in software or hardware. The described modules and / or units can also be housed in a processor; for example, a processor may be described as including a vector acquisition module, a heterogeneous graph construction module, a convolution operation module, a model training module, and a classification module. The names of these modules do not necessarily limit the module itself.
[0137] In another aspect, the present invention also provides a computer-readable medium, which may be included in the device described in the above embodiments; or it may exist independently and not assembled into the device. The computer-readable medium carries one or more programs, which, when executed by the device, cause the device to include:
[0138] Obtain text vector representations of sample comment information; mine entities and topics from the text vector representations, and construct a heterogeneous information graph based on entities, topics, and text vector representations; perform convolution operations on the heterogeneous information graph to obtain text node representations of sample comment information; use the text node representations to train the model and obtain a comment information classification model; use the comment information classification model to classify target comment information.
[0139] According to the technical solution of this invention, text vector representations of sample comment information are obtained; entities and topics in the text vector representations are mined, and a heterogeneous information graph is constructed based on the entities, topics, and text vector representations; convolution operations are performed on the heterogeneous information graph to obtain text node representations of the sample comment information; the text node representations are used to train a model to obtain a comment information classification model; and the comment information classification model is used to classify target comment information. In other words, this invention can vectorize samples and construct a heterogeneous information graph based on the entities and topics mined from the samples, achieving semantic enhancement of the samples from multiple dimensions, solving the problems of feature sparsity and high noise. Based on the convolution operation of the heterogeneous information graph, the features required for model training are obtained, and the model for classifying comment information is trained using these features, which can improve the accuracy of the classification results.
[0140] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.
[0141] The technical solution of this application complies with the relevant provisions of national laws and regulations regarding data acquisition, storage, use, and processing. The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can occur depending on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.
Claims
1. A method for classifying comment information, characterized in that, include: Obtain the text vector representation of sample comment information; Mine the entities and topics represented by the text vectors, and construct a first type of edge based on the entities contained in the text vector representations. The first type of edge is the edge from text to entity. A second type of edge is constructed based on the topic contained in the text vector representation, whereby the second type of edge is an edge from the text to the topic; A third type of edge is constructed based on the similarity between entities, wherein the third type of edge is an entity-to-entity edge; Using the text vector representation, the entity, and the topic as nodes, and the first type of edge, the second type of edge, and the third type of edge as edges, a heterogeneous information graph is constructed. Randomly remove edges from the heterogeneous information graph to obtain the target heterogeneous graph; Calculate the type-level attention coefficient based on the word embedding and type of each node in the target heterogeneous graph; The node-level attention coefficient is calculated based on the type-level attention coefficient, the word embedding, and the type of each node in the target heterogeneous graph; Determine the attention weight matrix based on the node-level attention coefficients; The target heterogeneous graph is convolved with the attention weight matrix to obtain the text node representation of the sample comment information. The text node representations are used to train the model, resulting in a comment information classification model. The target comment information is classified using the aforementioned comment information classification model.
2. The method according to claim 1, characterized in that, The sample comment information includes labeled sample comment information and unlabeled sample comment information. The text vector representation of the sample comment information is obtained as follows: Data augmentation is performed on the labeled sample comment information to obtain the first sample comment information; Semantic enhancement is performed on the first sample comment information and the unlabeled sample comment information to obtain the second sample comment information; The second sample comment information is vectorized to obtain the text vector representation.
3. The method according to claim 2, characterized in that, The step of performing data augmentation on the labeled sample comment information to obtain the first sample comment information includes: Identify the keywords and non-keywords in the labeled sample comment information; The keywords are augmented using random insertion, and the non-keywords are augmented using random replacement and / or random deletion to obtain the first sample comment information.
4. The method according to claim 2, characterized in that, The step of semantically enhancing the first sample comment information and the unlabeled sample comment information to obtain the second sample comment information includes: A multi-layer feature extractor based on a language understanding model is used to extract features from the first sample comment information and the unlabeled sample comment information to obtain multi-layer features; The multi-layer features are fused to obtain the second sample comment information.
5. A device for classifying comment information, characterized in that, include: The vector acquisition module is used to acquire the text vector representation of sample comment information; The heterogeneous graph construction module is used to mine the entities and topics represented by the text vectors, and construct a first type of edge based on the entities contained in the text vector representations. The first type of edge is the edge from text to entity. A second type of edge is constructed based on the topic contained in the text vector representation, whereby the second type of edge is an edge from the text to the topic; A third type of edge is constructed based on the similarity between entities, wherein the third type of edge is an entity-to-entity edge; Using the text vector representation, the entity, and the topic as nodes, and the first type of edge, the second type of edge, and the third type of edge as edges, a heterogeneous information graph is constructed. The convolution operation module is used to randomly remove edges from the heterogeneous information graph to obtain the target heterogeneous graph; Calculate the type-level attention coefficient based on the word embedding and type of each node in the target heterogeneous graph; The node-level attention coefficient is calculated based on the type-level attention coefficient, the word embedding, and the type of each node in the target heterogeneous graph; Determine the attention weight matrix based on the node-level attention coefficients; The target heterogeneous graph is convolved with the attention weight matrix to obtain the text node representation of the sample comment information. The model training module is used to train the model using the text node representations to obtain a comment information classification model; The classification module is used to classify the target comment information using the comment information classification model.
6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the comment information classification method as described in any one of claims 1 to 4.
7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the comment information classification method as described in any one of claims 1 to 4.