A text classification method and device based on heterogeneous neural network

Abstract

The invention discloses a text classification method and device based on a heterogeneous neural network. The method comprises: step S1, combining the tree structure of the actual classification system in the training data set, constructing an N-layer text classification multi-branch tree corresponding to the tree structure, and according to the structure of the N-layer text classification multi-branch tree, writing the training data in the training data set into classification files corresponding to each level respectively, performing word segmentation on the Chinese text of each classification file, and performing feature selection and saving in the corresponding feature file; step S2, constructing a text heterogeneous neural network corresponding to the tree structure; step S3, setting the heterogeneous neural network algorithm parameters, and adjusting the parameters, iteratively generating each classification and sub-classification model of the training data and saving them, and using the verification data set to determine the accuracy. The method also adds categories and the hierarchical relationship between categories into the heterogeneous neural network, explicitly learning vector representation for categories, and solving the data sparsity problem.

Description

Technical Field

[0001] The present invention relates to the fields of natural language processing and artificial intelligence, and in particular to a text classification method and device based on heterogeneous neural networks. Background Art

[0002] Text classification is the process of computers automatically classifying natural languages ​​according to a certain classification system, and it has applications in many fields. With the increasing number of application scenarios, scenarios with large classification systems are becoming more and more common. Taking the customer service center platform as an example, merchants hope to use automatic text classification to accurately classify and quickly record customer visit work orders, thereby speeding up the service response efficiency of on-site operators and improving the overall service quality; and by improving the intelligence level of hotline overall operation data analysis to optimize management. However, due to the complexity of business logic, the classification system in this scenario is very large, which is different from traditional text classification problems.

[0003] A large classification system is often accompanied by richer relationships between categories, such as hierarchical relationships. This multi-level classification system means that there is a progressive relationship between the levels of multiple categories. Generally, they are divided into several major categories, each of which contains several medium categories, each of which contains several small categories, and each small category is further divided into small small categories, and so on; finally, each text can be classified into a certain category, and there is a top-down logical relationship between the major categories, medium categories, small categories, and small small categories. Therefore, in order to make full use of this relationship, the existing method with better performance is to use machine learning algorithms for hierarchical classification, that is, each layer uses a model for classification, and the next layer continues to classify based on the classification results of the previous layer until the final classification result is obtained. At present, this hierarchical classification method has achieved better results than traditional text classification methods. Summary of the invention

[0004] In order to overcome the shortcomings of the above-mentioned prior art, the purpose of the present invention is to provide a method and device for implementing text classification based on heterogeneous neural networks, wherein the method is aimed at texts with a large classification system, that is, the number of categories reaches hundreds or even thousands, and classifies them based on the framework of heterogeneous neural network representation learning.

[0005] To achieve the above object, the present invention proposes a text classification method based on heterogeneous neural network, comprising the following steps:

[0006] Step S0, obtaining a training data set, a verification data set and a test data set to be classified of the text;

[0007] Step S1, combining the tree structure of the actual classification system in the training data set, constructing an N-layer text classification multi-branch tree corresponding to the tree structure, and according to the structure of the N-layer text classification multi-branch tree, writing the training data in the training data set into the classification files corresponding to each level respectively, naming the files according to the file naming rule R, saving the naming rule of the category number and its corresponding Chinese category name as a classification file for easy reading, selecting the corresponding word segmentation tool to segment the Chinese text, and filtering the word segmentation results through the stop word library, and then performing feature selection and saving them in the corresponding feature file;

[0008] Step S2, constructing the entire document data set into a large text heterogeneous neural network, wherein the nodes in the network include the word nodes, document nodes, and category nodes obtained in step S1 after removing stop words. The adjacency weight matrix between any two nodes is obtained according to the edge weight calculation formula between the nodes;

[0009] Step S3, setting parameters of the heterogeneous neural network classification algorithm and adjusting the parameters, importing the feature file data saved in step S2 into the heterogeneous neural network algorithm, iteratively generating and saving each classification and sub-classification model of the training data;

[0010] Step S4, read the test data sorted in step 0 one by one, and start the hierarchical prediction from the first layer t=1 until the classification result of the last layer. The prediction formula is:

[0011]

[0012] Among them, p(.) is the probability predicted by the model, T is the number of levels, t is the tth level, C t is the total number of categories at level t, h d is the vector of document d, h d T is the transpose of the vector of document d, h c t is the vector of category c under the tth layer, u t is the category u, h under the tth layer u t is the vector of category u under the tth layer, is the vector of classification categories at the t-1th layer.

[0013] Preferably, the step S0 further comprises: using D = {d1, d2, ..., d m} represents the sample data set, using (c i 1 ,c i 2 ,...,c i T) represents the true hierarchical category set of document i; where c i t is the category of document i at level t, c i t ∈(1,2,…,C t ), C t is the total number of categories at the tth level.

[0014] Preferably, the step S1 further comprises:

[0015] Step S100, using a Chinese word segmentation tool to sequentially segment the Chinese text of the classification file;

[0016] Step S101, constructing a stop word library, and using the constructed stop word library to filter out the word segmentation result of step S100;

[0017] Step S102, extracting features from the word segmentation results, and using a feature selection tool to select the first m-dimensional features as a feature library.

[0018] Preferably, in step S101, the stop word library includes but is not limited to numbers, English characters, adverbs, modal particles and business-irrelevant words, and the stop word library can be added, deleted or modified according to the needs of the actual work order text data content.

[0019] Preferably, in step S2, the weight A of the edge between nodes x and z is x,z The calculation formula is:

[0020]

[0021] Among them, TF is the word frequency; IDF is the inverse document frequency; the word frequency is the number of times a word appears in a document, and the inverse document frequency is the reciprocal of the logarithm of the number of documents containing the word.

[0022] Preferably, in step S2, the edge weight PMI(i, j) between word nodes i and j is calculated using the following formula:

[0023]

[0024] Where #W is the total number of sliding windows in the corpus, #W(i, j) is the number of sliding windows that contain both words i and j, and #W(i) is the number of sliding windows in the corpus that contain word i.

[0025] Preferably, in step S3, the verification data set compiled in step S0 is read to determine the accuracy of the result of the classification algorithm. If it is higher than a given threshold, the model training ends, otherwise, the process returns to step S2.

[0026] Preferably, first, the vector h of the category c under the tth layer is displayed in the text heterogeneous neural network. c t and the vector h of document d d , then do the inner product of the two vectors and normalize them to get the probability distribution of the category, and then use the category vector obtained by the previous level classification It is added to assist the prediction and classification of this level, and so on until the prediction and classification of the last level, and then the test data and classification results kept in the memory are saved to the result file.

[0027] Preferably, in step S4, a multi-level classification loss is used, that is, the classification loss of each level is weighted and calculated into the final loss, and the final loss formula is as follows:

[0028]

[0029] Among them, L is the final loss, γ t is the weight of the classification loss at the tth level, Y t is the true category matrix of the t-th level, D train is the training set, i is the sample i in the training set, and p(.) is the probability predicted by the model.

[0030] In order to achieve the above object, the present invention also provides a device for implementing a multi-level Chinese text classification method of heterogeneous neural network representation learning, comprising:

[0031] A data set generation unit, used to obtain a training data set, a verification data set and a test data set to be classified of the text;

[0032] The word segmentation and feature processing unit is used to segment the Chinese text of each classification file, filter the word segmentation results through the stop word library, and perform feature selection and save them into the corresponding feature file;

[0033] A neural network construction unit is used to combine the structure of the actual classification system in the training data set, construct a multi-level text classification heterogeneous neural network corresponding to the structure, and write the training data in the training data set into the classification files corresponding to each level according to the structure of the multi-level text classification heterogeneous neural network;

[0034] A classification training unit, used to set parameters of the heterogeneous neural network classification algorithm and adjust the parameters, import the feature file data saved by the word segmentation and feature processing unit into the heterogeneous neural network classification algorithm, iteratively generate each classification and sub-classification model of the training data and save them;

[0035] The prediction and classification unit is used to read the test data set collated by the data set generation unit one by one in sequence, load the model files of each level generated by the classification training unit in sequence, perform prediction and classification, and save the results.

[0036] The beneficial effects of the present invention are as follows:

[0037] 1. Use heterogeneous neural networks to model text data. Take advantage of the network's natural advantages in modeling complex relationships and build word and document nodes together with category nodes into a heterogeneous neural network. Unlike existing modeling methods that only treat words and documents as nodes, this method also adds categories and hierarchical relationships between categories into the heterogeneous neural network, explicitly learning vector representations for categories and solving the problem of data sparsity.

[0038] 2. In order to capture node attributes and network structure information at the same time and make good use of labels, graph neural networks are used on the constructed heterogeneous neural networks to learn node representations. Taking into account the heterogeneity of nodes, different aggregation functions are used for different types of nodes to allow information to be effectively transmitted in the network. In the classification stage, considering the hierarchical relationship between categories, the classification results of the upper layer can guide the classification of the next layer. Therefore, for layers other than the first layer, the category vector obtained by the classification of the previous layer is added to assist the classification of this layer, so that the classification accuracy is further improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 A text heterogeneous neural network framework for text classification based on heterogeneous neural networks according to the present invention;

[0040] Figure 2 A classification system tree for text classification based on heterogeneous neural networks according to the present invention;

[0041] Figure 3 The present invention is a flowchart of the steps of a method for implementing text classification based on heterogeneous neural networks. DETAILED DESCRIPTION

[0042] The following describes the implementation of the present invention through specific examples and in conjunction with the accompanying drawings. Those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific examples, and the details in this specification can also be modified and changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention.

[0043] Using D = {d1, d2, ..., d m} represents the sample data set, using (c i 1 ,c i 2,...,c i T ) represents the true hierarchical category set of document i; where T is the number of levels, c i t is the category of document i at level t, c i t ∈(1,2,…,C t ), C t is the total number of categories at the t-th level. The ultimate goal of this problem is to classify new documents into the correct t-th level category. Figure 1-3 , the key technologies of the present invention are elaborated in detail.

[0044] Figure 1 The invention discloses a text heterogeneous neural network framework for text classification based on heterogeneous neural networks. The framework mainly includes three parts. The first is the construction of a text heterogeneous network, the purpose of which is to construct the entire text data set into a heterogeneous network, whose nodes include words, documents and categories, and because there are a large number of categories and there is a hierarchical relationship between categories, the hierarchical association model of the categories is modeled into the heterogeneous network to strengthen information transmission; the second part is to use a heterogeneous graph neural network to learn node representation, and different aggregation functions are used for different types of nodes due to the heterogeneity of nodes and relationships; the third part is hierarchical classification, the purpose of which is to obtain the score of each category by doing the inner product of the document vector and the category vector in hierarchical order until the classification of the last layer is obtained. Compared with other multi-model methods, the method proposed in the invention uses a single model for training and testing, which greatly reduces the cost of training and prediction while maintaining the accuracy.

[0045] The nodes in the text heterogeneous neural network include word nodes, document nodes, and category nodes. The edges in the network include the co-occurrence relationship between words, the co-occurrence relationship between words and documents, and the belonging relationship between documents and categories. Considering that there is a hierarchical association between categories, it also includes the hierarchical relationship between categories. Introducing this association helps information transmission in the network and alleviates data sparsity. In order to obtain global word co-occurrence information, a fixed-size sliding window is used to count all documents in the text dataset, and pointwise mutual information (PMI) is used to calculate the weight between two word nodes. Pointwise mutual information is a special case of mutual information. In text processing, it can be used to calculate the degree of association between two words. The advantage of PMI is that it can analyze whether there is semantic correlation between words from a statistical perspective. In specific experiments, it is found that using PMI has better results than using word co-occurrence counts. The calculation method is as follows:

[0046]

[0047] Where #W is the total number of sliding windows in the corpus, #W(i, j) is the number of sliding windows that contain both words i and j, and #W(i) is the number of sliding windows in the corpus that contain word i.

[0048] A positive PMI value means that the semantic relatedness of the words is high, while a negative PMI value indicates little or no semantic relatedness in the corpus. Therefore, edges are only established between word pairs with positive PMI values.

[0049] This paper defines the weight of the edge between the document node and the word node as the word frequency-inverse document frequency (TF-IDF) of the word in the document, where the word frequency is the number of times a word appears in a document, and the inverse document frequency is the reciprocal of the logarithm of the number of documents containing the word. The importance of a word increases in direct proportion to the word frequency, but at the same time decreases in inverse proportion to the document frequency.

[0050] Since the relationship between document and category nodes and between category and category value is affiliation, their weights have no special meaning, so they are defined as 1. Therefore, formally, the weight A of the edge between nodes x and z is x,z The calculation formula is:

[0051]

[0052] Among them, TF is the word frequency; IDF is the inverse document frequency; the word frequency is the number of times a word appears in a document, and the inverse document frequency is the reciprocal of the logarithm of the number of documents containing the word.

[0053] Since there are a large number of categories and there are hierarchical relationships between them, information transfer can be enhanced by constructing the entire text dataset into a heterogeneous neural network, especially by modeling the hierarchical association of categories into the heterogeneous neural network.

[0054] After building a heterogeneous neural network, the next step is to design a heterogeneous graph neural network algorithm to learn the node representation.

[0055] This method randomly initializes the vectors of the nodes, and these parameters are trained along with the entire network. For word nodes, their only neighbor nodes are document nodes. For document nodes, their neighbor nodes are word nodes and category nodes. However, since category nodes are label information that guides network learning, messages from category nodes cannot be passed in during information aggregation, otherwise information leakage will occur. Therefore, for word nodes and category nodes, their only neighbors are each other, and since their edges are weighted, the GCN aggregation formula can be used:

[0056]

[0057] In Formula 3, l represents the number of convolution layers, H is the node vector, W is the transfer matrix, σ is the activation function, and H l+1 and H l are the node vectors of the l+1th layer and the lth layer respectively, W l is the l-th layer transfer matrix vector, is a normalized symmetric adjacency matrix, A is the adjacency matrix introduced in Key Technology 1, and D is the degree matrix. However, the information contained in the weight matrix A is based on the importance of statistical information, but lacks the interaction of content information. For example, words with similar meanings may not be closely related in statistics, but they are closely related in content. Therefore, this paper considers further capturing the interaction of node content based on statistical information. Specifically, for node i and node j, the attention mechanism is used to calculate the content importance weight of the neighboring nodes. The calculation method is as follows:

[0058]

[0059] Among them, a T is the weight vector, represents the vector concatenation operation, LeakyReLu is the activation function, N i is the neighbor node of node i, h i 、h j and h k are the vectors of node i, node j and node k respectively, and W1 is the first training parameter matrix.

[0060] Next, the weight of the neighboring node is obtained by adding the statistical weight to the content weight, and then concatenated with the node's own vector to obtain the vector representation of the node in the convolutional layer l+1:

[0061]

[0062] h i l+1 and h i l are the vectors of node i in the l+1th layer and the lth layer, respectively, h j l is the vector of node j in layer l, W2 l is the second training parameter matrix.

[0063] The aggregation method for category nodes is different from that for word and document nodes. The reason for modeling hierarchical relationships into text heterogeneous neural networks is that information can be transmitted through hierarchical associations of categories to alleviate data sparsity. For example, the "construction and transportation category" includes two subcategories, "urban management law enforcement" and "traffic law enforcement". The "urban management law enforcement" category accounts for a high proportion in the data set, while the "traffic law enforcement" category accounts for a very low proportion. Therefore, the vector of the former is more fully trained than the latter. Since the parent categories of both in the figure are "construction and transportation category", the "traffic law enforcement" category can obtain information transmitted by the parent category. The model only needs to further learn fine-grained knowledge based on this information, thereby reducing the difficulty of model learning. For the category node of the t layer, this paper divides its neighbor nodes into two parts, the t-1 layer and the t+1 layer. After averaging and aggregating these two parts respectively, the neighbor vector is obtained by proportion, and then concatenated with the node's own vector and passed through the activation function to obtain the vector representation of the node in the convolution layer l+1. The following is the aggregation formula:

[0064]

[0065] Among them, h i t-1 represents the vector at level t-1 of node i, h i t+1 represents the vector at level t+1 of node i, h v represents the vector of node v, h Ni Represents the aggregation vector of i’s neighbor nodes, N t-1 (i) represents the neighbor nodes at level t-1 of node i, N t +1 (i) represents the neighbor nodes at level t+1 of node i, W3 l is the third training parameter matrix, MEAN(.) represents the averaging operation, and β is the input ratio. β is used to control the ratio of parent category information to subcategory information input. The larger β is, the more subcategory information is input, and the model will focus more on learning finer-grained node representations. Conversely, the model will pass more parent category information to solve the data sparsity problem faced by categories with small data volumes.

[0066] Figure 2 This is a classification system tree for text classification based on heterogeneous neural networks. It is a hierarchical classification method. At present, most text classification methods use softmax classifiers for classification after learning document representation. This method implicitly learns the category vector. In this paper, the category vector h is learned in the text heterogeneous neural network. c t and the document vector h d, so the inner product of the two vectors can be normalized to obtain the probability distribution of the category. Considering the hierarchical relationship between categories, the classification results of the upper layer can guide the classification of the next layer. Therefore, for all layers except the first layer, the category vector obtained by the classification of the previous layer is Join in to assist in the classification of this level,

[0067]

[0068] Among them, p() is the probability predicted by the model, T is the number of levels, t is the tth level, C t is the total number of categories at level t, h d is the vector of document d, h d T is the transpose of the vector of document d, h c t is the vector of category c under the tth layer, u t is the category u, h under the tth layer u t is the vector of category u under the tth layer, is the vector of classification categories at the t-1th layer.

[0069] However, since the classification space of the last layer of categories is very large, if only the loss of the last layer of classification is calculated, the advantages of the hierarchical structure cannot be fully utilized. Therefore, this paper uses multi-level classification losses, that is, the classification losses of each level are weighted and included in the final loss. The formula is as follows:

[0070]

[0071] Among them, L is the final loss, γ t is the weight of the classification loss at the tth level, Y t is the true category matrix of the t-th level, D train is the training set, i is the sample i in the training set, and p(.) is the probability predicted by the model.

[0072] Figure 3 The present invention is a flowchart of a method for implementing text classification based on heterogeneous neural networks, comprising the following steps:

[0073] Step S0, obtaining a training data set, a verification data set and a test data set to be classified; using D = {d1, d2, ..., d m} represents the sample data set, using (c i 1 ,c i 2 ,...,c i T ) represents the true hierarchical category set of document i; where ci t is the category of document i at level t, c i t ∈(1,2,…,C t ), C t is the total number of categories at the tth level.

[0074] Step S1, combining the tree structure of the actual classification system in the training data set, constructing an N-layer text classification multi-branch tree corresponding to the tree structure, and according to the structure of the N-layer text classification multi-branch tree, writing the training data in the training data set into the classification files corresponding to each level respectively, naming the files according to the file naming rule R, saving the naming rule of the category number and its corresponding Chinese category name as a classification file for easy reading, selecting the corresponding word segmentation tool to segment the Chinese text, and filtering the word segmentation results through the stop word library, and then performing feature selection and saving them in the corresponding feature file, wherein step S1 specifically includes:

[0075] Step S100, using a Chinese word segmentation tool to sequentially segment the Chinese text of the classification file;

[0076] Step S101, constructing a stop word library, the stop word library includes but is not limited to numbers, English characters, adverbs, modal particles and words irrelevant to the business, and the stop word library can be added, deleted or modified according to the needs of the actual work order text data content, and the constructed stop word library is used to filter out the word segmentation results of step S100;

[0077] Step S102, extracting features from the word segmentation results, and using a feature selection tool to select the first m-dimensional features as a feature library.

[0078] Step S2, constructing the entire document data set into a large text heterogeneous neural network, wherein the nodes in the network include the word nodes, document nodes, and category nodes obtained in step S1 after removing stop words. Use pointwise mutual information (PMI) to calculate the weight between two word nodes (Formula 1), and define the formula (Formula 2) based on the weight of the edge between node i and node j to calculate the adjacency weight matrix between any two nodes;

[0079] Step S3, set the parameters of the heterogeneous neural network classification algorithm, and adjust the parameters, import the feature file data saved in step S1 into the heterogeneous neural network algorithm, read the verification data set sorted in step S0 to determine the accuracy of the classification algorithm result, if it is higher than a given threshold, the model training ends, otherwise return to execute step S2, and then iteratively generate each classification and sub-classification model of the training data and save it.

[0080] Step S4, read the test data sorted in step S0 one by one, first display the vector h of category c under the tth layer learned in the text heterogeneous neural network c t and the vector h of document d d , then the inner product of the two vectors is normalized to obtain the probability distribution of the category (Formula 7); then, considering the hierarchical relationship between categories, the category vector obtained by the previous level classification is Join in to assist the prediction and classification of this level; start from t=1 until the prediction and classification of the last level is completed, and then save the test data and classification results held in the memory into the result file; use multi-level classification loss, that is, weight the classification loss of each level into the final loss, and the final loss formula is as shown in Formula 8.

[0081] In order to achieve the above object, the present invention also provides a device for implementing a multi-level Chinese text classification method of heterogeneous neural network representation learning, comprising:

[0082] The data set generating unit 201 is used to obtain a training data set ST, a verification data set V and a test data set U to be classified of text.

[0083] In a specific embodiment of the present invention, taking the work order text of a call center user complaint as an example, the data set generation unit 201 obtains a training data set ST, a verification data set V and a test data set U to be classified of the work order text.

[0084] The word segmentation and feature processing unit 202 is used to segment the Chinese text of each classification file, filter the word segmentation results through the stop word library, and perform feature selection and save them into the corresponding feature file;

[0085] Specifically, the word segmentation and feature processing unit 202 further includes:

[0086] The word segmentation module is used to segment the Chinese text of the classification file in sequence using the Chinese word segmentation tool. For example, the jieba Chinese word segmentation tool is used to segment the Chinese text of the classification file in sequence.

[0087] The stop word library construction module is used to construct a stop word library, and the constructed stop word library is used to filter the word segmentation results of the step word segmentation module. In a specific embodiment of the present invention, the stop word library includes numbers, English characters, adverbs, modal particles, and business-irrelevant words, etc., and the stop word library can be added, deleted, and modified according to the needs of the actual work order text data content.

[0088] The feature selection module is used to extract features from the word segmentation results and use the feature selection tool to select the first m-dimensional features as the feature library. The m value is set according to the model results.

[0089] The neural network construction unit 203 is used to construct a multi-level text classification heterogeneous neural network corresponding to the structure of the actual classification system in the training data set ST, and write the training data in the training data set into the classification files corresponding to each level according to the structure of the multi-level text classification heterogeneous neural network;

[0090] The classification training unit 204 is used to set the parameters of the heterogeneous neural network classification algorithm and adjust the parameters, import the feature file data saved by the word segmentation and feature processing unit 202 into the classification algorithm, iteratively generate and save each classification and sub-classification model of the training data. Read the verification data set V sorted in the data set generation unit 201 to determine the accuracy. If it is higher than the given threshold Δ, the model training ends, otherwise it returns to execute the neural network construction unit 203;

[0091] Preferably, a device for implementing multi-level and multi-category Chinese text classification of the present invention further includes:

[0092] The prediction and classification unit 205 is used to read the test data set U sorted by the data set generation unit 201 one by one, load the model files of each level generated by the classification training unit 204 in turn, perform prediction and classification, and save the results. Specifically, first load the first-level model file corresponding to the classification algorithm, perform the first-level prediction and classification, and save it; then load the corresponding second-level model file to perform the second-level prediction and classification, and save it; then load the corresponding third-level model file to predict and classify and save it; until the corresponding N-th level model prediction and classification is completed, then save the test data and classification results stored in the memory into the result file.

[0093] Example

[0094] In this embodiment, taking the call center user complaint work order text classification as an example, the implementation method of the call center user complaint classification is as follows:

[0095] Step S0, obtaining a training data set of 100,000 work order texts, a verification data set of 20,000 work order texts, and a test data set of 4,000 work order texts.

[0096] Step S1, in combination with the tree structure of the actual classification system in the training data set, construct an N-layer text classification multi-branch tree corresponding to the tree structure, and according to the structure of the N-layer text classification multi-branch tree, write the training data in the training data set into the classification files corresponding to each level respectively, name the files according to the file naming rule R, save the naming rule of the category number and its corresponding Chinese category name as a classification file for easy reading, select the corresponding word segmentation tool to segment the Chinese text, and filter the word segmentation results through the stop word library, and then perform feature selection and save them in the corresponding feature file, as follows:

[0097] Step S100, using the jieba Chinese word segmentation tool to segment the Chinese text of the file in sequence.

[0098] Step S102, construction of a stop word library, adding, deleting and modifying the stop word library according to business needs, the stop word library includes numbers, English characters, adverbs, modal particles and words irrelevant to the business.

[0099] Step S2, arranges the training data of call center work order text data and its corresponding four-level classification, wherein the first level has 9 categories, the second level has 63 categories, the third level has 226 categories, and the fourth level has 676 categories (in the embodiment, according to business needs, the example cited is a 4-level category structure). The entire text data set is constructed into a large text heterogeneous network. The nodes in the network include word nodes, document nodes and category nodes after removing stop words obtained in step 1. According to formula 2, the adjacency matrix weight between any two nodes is calculated.

[0100] Step S3, set the algorithm parameters to train the training data set organized in step 0. First, randomly initialize the vector of each node. These parameters are trained together with the entire network. For word nodes and document nodes, use formula 5 for aggregation. For category nodes, use formula 6 for aggregation. Use the function shown in formula 8 as the loss function of the model. Read the verification data set organized in step 0 for accuracy determination. If the verification data set is no longer improved, the model training ends, otherwise continue training.

[0101] Step S4, read the test data sorted in step 0 one by one, use formula 7, and perform hierarchical prediction starting from t=1 until the classification result of the 4th layer is obtained.

[0102] In summary, the present invention discloses a text classification method and device based on heterogeneous neural networks, which uses heterogeneous networks to model multi-level and multi-category texts, and uses heterogeneous network representation learning methods to learn node representations, and finally achieves text classification with high accuracy. The present invention can be applied to online customer service work order systems, and can achieve accurate classification of customer visit work orders, thereby accelerating the service response efficiency of on-site operators, improving the overall service quality, and improving the intelligence of the overall hotline operation data analysis, and optimizing the management effect.

[0103] The above embodiments are merely illustrative of the principles and effects of the present invention, and are not intended to limit the present invention. Any person skilled in the art may modify and alter the above embodiments without violating the spirit and scope of the present invention. Therefore, the scope of protection of the present invention shall be as set forth in the claims.

Claims

1. A text classification method based on heterogeneous neural network, comprising the following steps: Step S0, obtaining a training data set, a verification data set and a test data set to be classified of the text; Step S1, combining the tree structure of the actual classification system in the training data set, constructing an N-layer text classification multi-branch tree corresponding to the tree structure, and according to the structure of the N-layer text classification multi-branch tree, writing the training data in the training data set into the classification files corresponding to each level respectively, naming the files according to the file naming rule R, saving the naming rule of the category number and its corresponding Chinese category name as a classification file for easy reading, selecting the corresponding word segmentation tool to segment the Chinese classification file, and filtering the word segmentation result through the stop word library, and then performing feature selection and saving it in the corresponding feature file; Step S2, constructing the entire document data set into a large text heterogeneous neural network, wherein the nodes in the text heterogeneous neural network include word nodes, document nodes, and category nodes obtained in step S1 after removing stop words; and obtaining an adjacency weight matrix between any two nodes according to the weight calculation formula of the edges between nodes; Step S3, setting parameters of the heterogeneous neural network classification algorithm and adjusting the parameters, importing the feature file data saved in step S2 into the heterogeneous neural network algorithm, iteratively generating and saving each classification and sub-classification model of the training data; Step S4, read the test data sorted in step S0 one by one, and start hierarchical prediction from the first layer t=1 until the classification result of the last layer is obtained. The prediction formula is: Among them, p(.) is the probability predicted by the model, T is the number of levels, t is the tth level, C t is the total number of categories at level t, h d is the vector of document d, h d T is the transpose of the vector of document d, h c t is the vector of category c under the tth layer, u t is the category u, h under the tth layer u t is the vector of category u under the tth layer, is the vector of classification categories at the t-1th layer.

2. A text classification method based on heterogeneous neural network as claimed in claim 1, characterized in that: The step S0 further includes: using D = {d1, d2, ..., d m } represents the sample data set, using (c i 1 ,c i 2 ,...,c i T ) represents the true hierarchical category set of document i; where c i t is the category of document i at level t, c i t ∈(1,2,…,C t ), C t is the total number of categories at the tth level.

3. The text classification method based on heterogeneous neural network according to claim 1 is characterized in that: The step S1 further comprises: Step S100, using a Chinese word segmentation tool to sequentially segment the Chinese text of the classification file; Step S101, constructing a stop word library, and using the constructed stop word library to filter out the word segmentation result of step S100; Step S102, extracting features from the word segmentation results, and using a feature selection tool to select the first m-dimensional features as a feature library.

4. The text classification method based on heterogeneous neural network according to claim 3 is characterized in that: In step S101, the stop word library includes but is not limited to numbers, English characters, adverbs, modal particles and words irrelevant to the business, and the stop word library can be added, deleted or modified according to the needs of the actual work order text data content.

5. The text classification method based on heterogeneous neural network according to claim 1 is characterized in that: In step S2, the weight A of the edge between nodes x and z x,z The calculation formula is: Among them, TF is the word frequency; IDF is the inverse document frequency; the word frequency is the number of times a word appears in a document, and the inverse document frequency is the reciprocal of the logarithm of the number of documents containing the word.

6. A text classification method based on heterogeneous neural network according to claim 5, characterized in that: In step S2, the edge weight PMI(i, j) between word nodes i and j is calculated using the following formula: Where #W is the total number of sliding windows in the corpus, #W(i, j) is the number of sliding windows that contain both words i and j, and #W(i) is the number of sliding windows in the corpus that contain word i.

7. A text classification method based on heterogeneous neural network as claimed in claim 1, characterized in that : In step S3, the verification data set compiled in step S0 is read to determine the accuracy of the classification algorithm result. If it is higher than a given threshold, the model training ends, otherwise it returns to execute step S2.

8. The text classification method based on heterogeneous neural network according to claim 1 is characterized in that: In step S4, first, the vector h of the category c in the tth layer is displayed in the text heterogeneous neural network. c t and the vector h of document d d , then do the inner product of the two vectors and normalize them to get the probability distribution of the category, and then use the category vector obtained by the previous level classification It is added to assist the prediction and classification of this level, and so on until the prediction and classification of the last level, and then the test data and classification results kept in the memory are saved to the result file.

9. The text classification method based on heterogeneous neural network according to claim 2 is characterized in that: In step S4, a multi-level classification loss is used, that is, the classification loss of each level is weighted and calculated into the final loss. The final loss formula is as follows: Among them, L is the final loss, γ t is the weight of the classification loss at the tth level, Y t is the true category matrix of the t-th level, D train is the training set, i is the sample i in the training set, and p(.) is the probability predicted by the model.

10. A text classification device used in the method according to any one of claims 1 to 9, characterized in that: include: A data set generation unit, used to obtain a training data set, a verification data set and a test data set to be classified of the text; The word segmentation and feature processing unit is used to segment the Chinese text of each classification file, filter the word segmentation results through the stop word library, and perform feature selection and save them into the corresponding feature file; A neural network construction unit is used to combine the structure of the actual classification system in the training data set, construct a multi-level text classification heterogeneous neural network corresponding to the structure, and write the training data in the training data set into the classification files corresponding to each level according to the structure of the multi-level text classification heterogeneous neural network; A classification training unit, used to set parameters of the heterogeneous neural network classification algorithm and adjust the parameters, import the feature file data saved by the word segmentation and feature processing unit into the heterogeneous neural network classification algorithm, iteratively generate each classification and sub-classification model of the training data and save them; The prediction and classification unit is used to read the test data set collated by the data set generation unit one by one in sequence, load the model files of each level generated by the classification training unit in sequence, perform prediction and classification, and save the results.

Images