Text information classification method, device, electronic device and storage medium

Abstract

The embodiments of the present invention provide a text information classification method, device, electronic device and storage medium. In the inference process after completing the construction of the label vector matrix and the text vector matrix, and under the premise of using the label observation probability used to express the category label confidence through the existing model as the inference basis, the label transfer probability between different levels is calculated to constrain the classification results of the category labels at each level, thereby avoiding using the wrong label as the label prediction result when the confidence of the output wrong label is higher than the confidence of the correct label, and passing the wrong information to the prediction of the lower layer, thereby improving the accuracy of text information classification.

Description

Technical Field

[0001] The present invention relates to the technical field of text information classification, and in particular to a text information classification method, a text information classification device, an electronic device, and a computer-readable storage medium. Background Art

[0002] Multi-level text classification is a key technology for processing large-scale text information. In many application scenarios for large-scale text information, documents are often organized by multi-level structures. For example, a document (web page or email) can be associated with multiple categories, and these categories can be stored in a file storage tree or a directed acyclic graph.

[0003] In related technologies, due to the imperfect mechanism for multi-level text classification, the accuracy of the algorithm will drop sharply as the levels and text categories increase, resulting in low accuracy for multi-level text classification. Summary of the Invention

[0004] The embodiments of the present invention provide a text information classification method, device, electronic device and computer-readable storage medium to solve the problem of low accuracy in multi-level text classification.

[0005] An embodiment of the present invention discloses a method for classifying text information. The method is applied to a text classification model. The text classification model has multiple levels, each level includes corresponding label nodes, each label node has a corresponding category label, and the category label is used to express the category of the label node. The method may include:

[0006] Generate a label vector for the category label, and use the label vector to generate a label vector matrix;

[0007] Generating a text vector for the text information, and generating a text vector matrix using the text vector;

[0008] generating a label transition probability for the class label;

[0009] Inputting the label vector matrix and the text vector matrix into the text classification model, and generating label observation probabilities for the category labels;

[0010] Based on the label observation probability and the label transition probability, a classification result for the category label is determined, and the classification result is used as a classification result for the text information.

[0011] Optionally, the text classification model is integrated with a word vector model, the label node includes a parent node and a child node directly associated with the parent node, and the steps of generating a label vector for the category label and generating a label vector matrix using the label vector may include:

[0012] Determine a sub-label vector for the sub-category label of the child node using the word vector model;

[0013] Determine the number of the child nodes;

[0014] Generate a parent label vector for the parent category label of the parent node by using the child label vector and the number of child nodes;

[0015] A label vector matrix is ​​constructed using the child label vector and the parent label vector.

[0016] Optionally, the step of generating a label transition probability for the category label includes:

[0017] Determining the number of parent tag categories for the parent category tag in the parent level where the parent node is located;

[0018] Determining the number of sub-tag categories for the sub-category tag in the sub-level where the child node is located;

[0019] The label transition probability for the category label is calculated and generated using the number of parent label categories and the number of child label categories.

[0020] Optionally, the step of determining a classification result for the category label based on the label observation probability and the label transition probability, and using the classification result as the classification result for the text information may include:

[0021] Based on the Viterbi algorithm, the shortest paths between the multiple category labels are calculated using the label observation probability and the label transition probability;

[0022] The shortest path is used as a classification result for the category label, and the classification result is used as a classification result for the text information.

[0023] Optionally, the word vector model is a multilingual model of a multilingual bot model, and the steps of generating a text vector for the text information and generating a text vector matrix using the text vector may include:

[0024] The multilingual bot model is used to obtain text vectors for the text information, and the text vectors are used to generate a text vector matrix.

[0025] An embodiment of the present invention further discloses a text information classification device, which is applied to a text classification model. The text classification model has multiple levels, each level includes corresponding label nodes, each label node has a corresponding category label, and the category label is used to express the category of the label node. The device may include:

[0026] A label vector matrix generation module, configured to generate a label vector for the category label and generate a label vector matrix using the label vector;

[0027] A text vector matrix generation module, configured to generate a text vector for the text information and generate a text vector matrix using the text vector;

[0028] A label transition probability generation module, used to generate a label transition probability for the category label;

[0029] a label observation probability generation module, configured to input the label vector matrix and the text vector matrix into the text classification model and generate a label observation probability for the category label;

[0030] A classification result determination module is used to determine a classification result for the category label based on the label observation probability and the label transition probability, and adopt the classification result as the classification result for the text information.

[0031] Optionally, the text classification model is integrated with a word vector model, the label node includes a parent node and a child node directly associated with the parent node, and the label vector matrix generation module may include:

[0032] a sub-label vector determination sub-module, configured to determine a sub-label vector for the sub-category label of the sub-node using the word vector model;

[0033] A sub-node number determination sub-module, used to determine the number of the sub-nodes;

[0034] A parent label vector calculation submodule, configured to calculate and generate a parent label vector for the parent category label of the parent node by using the child label vector and the number of child nodes;

[0035] The label vector matrix generation submodule is used to construct a label vector matrix using the child label vector and the parent label vector.

[0036] Optionally, the label transition probability generating module may include:

[0037] A parent tag category quantity determination submodule, configured to determine the number of parent tag categories for the parent category tag in the parent level where the parent node is located;

[0038] A sub-label category quantity determination sub-module, configured to determine the number of sub-label categories for the sub-category label in the sub-level where the sub-node is located;

[0039] The label transition probability generating submodule is configured to calculate and generate a label transition probability for the category label by using the number of the parent label category and the number of the child label category.

[0040] Optionally, the classification result determination module may include:

[0041] A shortest path calculation submodule, configured to calculate the shortest paths between each of the plurality of category labels using the label observation probability and the label transition probability based on a Viterbi algorithm;

[0042] The classification result determination submodule is configured to adopt the shortest path as the classification result for the category label, and adopt the classification result as the classification result for the text information.

[0043] Optionally, the word vector model is a multilingual model of a multilingual bot model, and the text vector matrix generation module may include:

[0044] The text vector matrix generation submodule is used to obtain text vectors for the text information using the multilingual bot model, and generate a text vector matrix using the text vectors.

[0045] An embodiment of the present invention further discloses an electronic device, comprising a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other via the communication bus;

[0046] The memory is used to store computer programs;

[0047] The processor is configured to implement the method described in the embodiment of the present invention when executing the program stored in the memory.

[0048] An embodiment of the present invention further discloses a computer-readable storage medium having instructions stored thereon. When executed by one or more processors, the processors are enabled to execute the method according to the embodiment of the present invention.

[0049] The embodiments of the present invention include the following advantages:

[0050] The embodiment of the present invention generates a label vector for a category label and uses the label vector to generate a label vector matrix; generates a text vector for text information and uses the text vector to generate a text vector matrix; generates a label transition probability for the category label; inputs the label vector matrix and the text vector matrix into a text classification model and generates a label observation probability for the category label; determines the classification result for the category label based on the label observation probability and the label transition probability, and uses the classification result as the classification result for the text information, thereby achieving, in the reasoning process after completing the construction of the label vector matrix and the text vector matrix, using the category label observation probability for expressing the category label confidence as the reasoning basis through the existing model, by calculating the label transition probability between different levels, constraining the classification results of the category labels at each level, avoiding using the wrong label as the label prediction result when the confidence of the output wrong label is higher than the confidence of the correct label, and passing the wrong information to the prediction of the lower layer, thereby improving the accuracy of text information classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 is a flowchart of the steps of a text information classification method provided in an embodiment of the present invention;

[0052] Figure 2 This is a schematic diagram of the structural relationship between hierarchical and category labels provided in an embodiment of the present invention;

[0053] Figure 3 This is a schematic diagram of the structural relationship between a parent node and a child node provided in an embodiment of the present invention;

[0054] Figure 4 This is a structural block diagram of a text information classification device provided in an embodiment of the present invention;

[0055] Figure 5 This is a hardware structure block diagram of an electronic device provided in each embodiment of the present invention. DETAILED DESCRIPTION

[0056] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0057] In practical applications, how to implement multi-level label text classification (HMTC) is a basic but challenging task. Related technologies use classifiers to process all label categories at the same time, or directly decompose the original problem into a set of parallel multi-label classification sub-problems without considering the correlation and hierarchical structure between texts, as well as the dependencies between different levels. For example, related technologies classify texts in the following way: split text labels into vector matrices; construct a vector matrix for text content based on text units; use a multi-layer attention model to calculate the category attention matrix for text information and the category representation information for text information; generate a unified text representation and text category prediction; calculate the semantic representation; and finally repeat the above steps until all unified text representation information and text category prediction results for each text are obtained.

[0058] First, in the step of "splitting text labels into a matrix of vectors," related technologies use the word2vec model to embed labels into d-dimensional vectors, splitting text labels into a matrix of vectors to construct a vector matrix. Although word2vec considers contextual information and can better express similarities and analogies between different words, such as labelvec(king) - labelvec(man) + labelvec(woman) = labelvec(queen), in multi-level classification tasks, the vectors obtained using word2vec cannot reflect the relationships between category hierarchies. Taking the tree-like category structure as an example, category labels at different levels reflect different granularity information of text categories. From top to bottom, the labels are refined step by step. The lower the level, the more it reflects the specific content of the text description. Conversely, the upper level is often a low-level summary content with relatively coarse granularity. Therefore, directly embedding the word2vec pair labels fails to consider the hierarchical relationship between parent class labels and child class labels, and cannot distinguish the granularity information of labels at different levels. As a result, the subsequent calculation of the text-category attention matrix and text-category representation lacks information between parent classes and child classes, resulting in low accuracy of multi-layer classification results for text information, which has a negative impact on the classification results.

[0059] Second, in the reasoning process after completing the construction of the label vector matrix and the text vector matrix, the relevant technology only uses the category label observation probability used to express the category label confidence as the only reasoning basis to obtain the predicted label result for the text information. This leads to the predicted label result being unconstrained when determining the label result of each layer layer by layer. When the confidence of the output wrong label is higher than the confidence of the correct label, the wrong label is often used as the label prediction result, and the wrong information is passed to the prediction of the lower layer, thereby affecting the accuracy of the prediction.

[0060] Reference Figure 1, shows a flowchart of a text information classification method provided in an embodiment of the present invention, which may specifically include the following steps:

[0061] Step 101: Generate a label vector for the category label, and use the label vector to generate a label vector matrix;

[0062] Step 102: Generate a text vector for the text information, and use the text vector to generate a text vector matrix;

[0063] Step 103: Generate a label transition probability for the category label;

[0064] Step 104: input the label vector matrix and the text vector matrix into the text classification model, and generate label observation probabilities for the category labels;

[0065] Step 105 : Determine a classification result for the category label based on the label observation probability and the label transition probability, and use the classification result as a classification result for the text information.

[0066] In a specific implementation, the embodiments of the present invention can be applied to text classification models, such as RNN (Recurrent Neural Network), LSTM (Long Short Term Memory) or GRU (Gated Recurrent Unit), etc.

[0067] The text classification model of the embodiment of the present invention can have multiple levels, each level can include corresponding label nodes, the label nodes can have corresponding category labels, and the category labels can be used to express the category of the label nodes. For example, the text classification model can include multiple levels, such as 10 layers, 11 layers, 12 layers... 1N layers. The 10 layer has a root node with a category label of "root user root", the 11 layer has two label nodes with category labels of "artificial intelligence AI" and "system System", and the 12 layer has five category labels of "speech recognition Speech", "natural language processing NLP", "computer vision CV", "network Network", and "database DB". The embodiment of the present invention can integrate a word vector model for the text classification model, such as the multilingual bert model bert, the word vector model word2vec, etc., and use the word vector model to generate a label vector for the category label. The label vector can be a vector used to express the category label. After generating the label vector, the label vector can be used to generate a label vector matrix. The vector matrix can be composed of multiple vectors of equal length, where each column or row expresses a vector. The label vector matrix can be a vector matrix composed of multiple label vectors.

[0068] The embodiment of the present invention can also generate text vectors for text information through a word vector model. The text vector can be a vector used to express text (semantic) information. After generating the text vector, the text vector can be used to generate a text vector matrix. The vector matrix can be composed of multiple vectors of equal length, where each column or row expresses a vector. The text vector matrix can be a vector matrix composed of multiple text vectors.

[0069] After generating the label vector matrix and the text vector matrix, an embodiment of the present invention can input the label vector matrix and the text vector matrix into a text classification model to obtain the label observation probability of the text information at each level for expressing the label confidence of the category label, that is, the label observation probability of the embodiment of the present invention can represent the confidence that the text information is a certain label.

[0070] In a specific implementation, the word vector model of the present invention is distinguished from a text classification model. A word vector model generally refers to a model used to convert natural language words into dense vectors. In the present invention, the word vector model can be used to generate label vectors and text vectors, while the text classification model can be a model obtained by learning text vectors and label vectors. In the present invention, the text classification model can predict text vectors and obtain label observation probabilities.

[0071] In practical applications, during the inference process after the multi-level text classification completes the construction of the label vector matrix and the text vector matrix, if only the label observation probability used to express the confidence of the category label is used as the only reasoning basis, when the confidence of the output wrong label is higher than the confidence of the correct label, the wrong label will be used as the label prediction result, thereby affecting the accuracy of the prediction. Therefore, the embodiment of the present invention can generate a label transfer probability for the category label before determining the classification result for the category label. The label transfer probability can represent the probability of transferring from a label at the previous level to a label at the next level. For example, Figure 2 As shown, Figure 2 A schematic diagram of the structural relationship between hierarchical and category labels provided in an embodiment of the present invention is shown. The 11th layer 201 may have a category label 2011 "artificial intelligence AI" and a category label 2012 "system System", and the 12th layer 202 may have a category label 2021 "speech recognition Speech", a category label 2022 "natural language processing NLP", a category label 2023 "computer vision CV", a category label 2024 "network Network", and a category label 2025 "database DB". Before determining the classification result for the category label, the probability of transferring from a label of the previous layer to a label of the next layer can be determined first, such as from the category label The probability of transferring from the category label 2011 “Artificial Intelligence AI” to the category label 2021 “Speech Recognition Speech”, the category label 2022 “Natural Language Processing NLP”, the category label 2023 “Computer Vision CV”, the category label 2024 “Network Network”, and the category label 2025 “Database DB” respectively; or the probability of transferring from the category label 2012 “System System” to the category label 2021 “Speech Recognition Speech”, the category label 2022 “Natural Language Processing NLP”, the category label 2023 “Computer Vision CV”, the category label 2024 “Network Network”, and the category label 2025 “Database DB” respectively.

[0072] After determining the label transition probability and the label observation probability, the embodiment of the present invention can determine a classification result for the category label based on the label observation probability and the label transition probability, and use the classification result as the classification result for the text information.

[0073] For example, a text classification model may include multiple layers, including layer l0, layer l1, and layer l2. The layer l1 has two label nodes with category labels, namely artificial intelligence AI and system System. Artificial intelligence AI, as the parent node, has three category labels in the layer l2, namely speech recognition Speech, natural language processing NLP, and computer vision CV; the system System, as the parent node, has two category labels, network Network and database DB in the layer l2. Because, if the label of the layer l1 is artificial intelligence AI, then the label of the layer l2 can only be one of natural language processing NLP, speech recognition Speech, or computer vision CV. The label cannot appear in network Network or database DB because they do not have a parent-child node relationship. Therefore, when the artificial intelligence AI label is detected, the transfer probability to the database DB label is 0. Therefore, when the label observation probability of the artificial intelligence AI category label being transferred to the database DB category label is high, and the label transfer probability of the artificial intelligence AI category label being transferred to the database DB category label is 0, the label observation probability of the artificial intelligence AI category label being transferred to the database DB category label is not adopted as the prediction result of the label transfer. For example, the probability of artificial intelligence of the l1 label obtained by the text classification model is relatively high, but in the I2 layer, the observation probability obtained by speech recognition Speech is 0.45, and the observation probability obtained by database DB is 0.55. However, through the label transfer probability, it is known that when l1 is artificial intelligence, the probability of the label database DB appearing in l2 is 0. Then it can be predicted that speech recognition Speech is the label appearing in l2. In this way, all nodes are traversed and classified.

[0074] The embodiment of the present invention generates a label vector for a category label and uses the label vector to generate a label vector matrix; generates a text vector for text information and uses the text vector to generate a text vector matrix; generates a label transition probability for the category label; inputs the label vector matrix and the text vector matrix into a text classification model and generates a label observation probability for the category label; determines the classification result for the category label based on the label observation probability and the label transition probability, and uses the classification result as the classification result for the text information, thereby achieving, in the reasoning process after completing the construction of the label vector matrix and the text vector matrix, using the category label observation probability for expressing the category label confidence as the reasoning basis through the existing model, by calculating the label transition probability between different levels, constraining the classification results of the category labels at each level, avoiding using the wrong label as the label prediction result when the confidence of the output wrong label is higher than the confidence of the correct label, and passing the wrong information to the prediction of the lower layer, thereby improving the accuracy of text information classification.

[0075] Based on the above embodiment, a modified embodiment of the above embodiment is proposed. It should be noted that, in order to simplify the description, only the differences from the above embodiment are described in the modified embodiment.

[0076] In an optional embodiment of the present invention, the text classification model is integrated with a word vector model, the label node includes a parent node and a child node directly associated with the parent node, and the steps of generating a label vector for the category label and generating a label vector matrix using the label vector include:

[0077] Determine a sub-label vector for the sub-category label of the child node using the word vector model;

[0078] Determine the number of the child nodes;

[0079] Generate a parent label vector for the parent category label of the parent node by using the child label vector and the number of child nodes;

[0080] A label vector matrix is ​​constructed using the child label vector and the parent label vector.

[0081] As can be seen from the above, the related technology fails to consider the hierarchical relationship between the parent class label and the child class label, and cannot distinguish the granularity information of labels at different levels, resulting in the subsequent calculation of the text-category attention matrix and the text-category representation lacking the information between the parent class and the child class, thereby resulting in low accuracy of the multi-layer classification results for text information, which has a negative impact on the classification results. Therefore, the embodiment of the present invention can first determine the parent node in the label node and the child node that has a direct association relationship with the parent node, for example, Figure 3 As shown, Figure 3 A schematic diagram of the structural relationship between parent nodes and child nodes provided in an embodiment of the present invention is shown. If the category label 3011 "Artificial Intelligence AI" located in the I1 layer 301 is the parent node, then the child nodes directly associated with the category label 3011 "Artificial Intelligence AI" may be the category label 3021 "Speech Recognition Speech" located in the I2 layer 302, the category label 3022 "Natural Language Processing NLP", and the category label 3023 "Computer Vision CV". If the category label 3012 "System System" located in the I1 layer 301 is the parent node, then the child nodes directly associated with the category label 3012 "System System" may be 3024 "Network Network" and the category label 3025 "Database DB".

[0082] After determining the parent node in the label node and the child nodes that have a direct association with the parent node, an embodiment of the present invention can use a word vector model to determine the child label vector of the subcategory label for the child node, and determine the number of child nodes, and then use the child label vector and the number of child nodes to calculate and generate the parent label vector of the parent category label for the parent node.

[0083] For example, the parent label vector of the parent category label of the parent node can be calculated using the child label vector, and the number of child nodes through the following formula 1. Assume that the vector representation of the mth category label in the nth level is calculated.

[0084] Formula 1:

[0085]

[0086] Vec node_n_m It can be the vectorized representation of the label node_n_m by word2vec. The weight coefficient can take the classic value of 0.5, k is the number of direct subclasses of node_n_m, Vec node_n_m_i A vector representing the labels of the ith direct subclasses of label m, It represents the sum of the vectors of the direct subclasses of the label m. Assuming that the direct subclasses of AI are Speech, NLP, and CV, and the direct subclasses of System are Network and DB, the number of direct subclasses of the label AI is 3. Similarly, the number of direct subclasses of the label System is 2. For the parent label vector of the parent category label of the parent node AI, after determining the sub-label vectors and the number of sub-nodes of the sub-category labels of the child nodes Speech, NLP, and CV through the word vector model, we can use Formula 1 to calculate the parent label vector of the parent category label of the parent node AI.

[0087]

[0088] The category labels in the hierarchy are calculated from bottom to top to obtain the parent label vector for AI as follows.

[0089]

[0090] Of course, those skilled in the art may use other algorithms to calculate and generate a parent label vector for the parent category label of the parent node based on the child label vector and the number of child nodes, and this embodiment of the present invention does not limit this.

[0091] After the parent label vector is determined, the embodiment of the present invention can use the child label vectors and the parent label vector to construct a label vector matrix.

[0092] In an embodiment of the present invention, the word vector model is used to determine the child label vector of the subcategory label for the child node; the number of the child nodes is determined; the child label vector and the number of the child nodes are used to calculate and generate the parent label vector of the parent category label for the parent node; the child label vector and the parent label vector are used to construct a label vector matrix, thereby achieving the simultaneous combination of the semantic information of the label itself and the semantic information of all its subcategory labels when the label is vectorized in the hierarchical category structure. On the one hand, the semantic representation of the parent category label can be enhanced so that it has the implicit information of the subcategory; on the other hand, it also avoids the problem of being unable to capture the category label due to the lack of association information between the parent class and a subclass in the word2vec model, thereby improving the prediction accuracy of the model.

[0093] In an optional embodiment of the present invention, the step of generating the label transition probability for the category label includes:

[0094] Determining the number of parent tag categories for the parent category tag in the parent level where the parent node is located;

[0095] Determining the number of sub-tag categories for the sub-category tag in the sub-level where the child node is located;

[0096] The label transition probability for the category label is calculated and generated using the number of parent label categories and the number of child label categories.

[0097] The parent node and child node in the label node have been determined above. In a specific implementation, an embodiment of the present invention can determine the number of parent label categories for the parent category label in the parent level where the parent node is located, and determine the number of child label categories for the child category label in the child level where the child node is located, and then use the number of parent label categories and the number of child label categories to calculate and generate the label transfer probability for the category label.

[0098] For example, in the embodiment of the present invention, the label transition probability for the category label can be calculated and generated by the following formula 2.

[0099] Formula 2:

[0100]

[0101] Count a，* Indicates the number of category labels a in the text level l = n, without considering the levels after l = n. a，bIt represents the number of texts with the category label a at level l = n and the category label b at level l = n + 1. Assume that the direct subcategories of AI at level I1 are Speech, NLP, and CV at level I2; the direct subcategories of System at level I1 are Network and DB at level I2, the label at level I1 is AI, and the label at level I2 is NLP, then P AI→NLP That is, the transfer probability of the category label from AI to NLP, and the label Network is a child node that has no direct relationship with the parent node AI, that is, there is no transfer relationship between the label Network and the label AI, then the label transfer probability P of AI to Network is AI→Network =0, and similarly the label transfer probability for all category labels can be calculated.

[0102] Of course, those skilled in the art may use other algorithms to calculate and generate the label transfer probability for the category label based on the number of parent label categories and the number of child label categories, and this embodiment of the present invention does not limit this.

[0103] In an optional embodiment of the present invention, the step of determining a classification result for the category label based on the label observation probability and the label transition probability, and using the classification result as a classification result for the text information includes:

[0104] Based on the Viterbi algorithm, the shortest paths between the multiple category labels are calculated using the label observation probability and the label transition probability;

[0105] The shortest path is used as a classification result for the category label, and the classification result is used as a classification result for the text information.

[0106] In practical applications, the number of category labels increases with each level, and text classification models generally have many levels, so the number of category labels will undoubtedly be very large. If the shortest path between multiple category labels is calculated by exhaustive method, it will consume huge computing power. Therefore, the embodiment of the present invention can calculate the shortest path between multiple category labels based on the Viterbi algorithm, using label observation probability and label transition probability. The Viterbi algorithm is a dynamic programming algorithm used to find the Viterbi path-implicit state sequence that is most likely to produce an observation event sequence.

[0107] For example:

[0108] Formula 3:

[0109] δ1(i)=π i b i (c 1i ), i = 1, 2, ..., N

[0110] Formula 4:

[0111] ψ1(i)=0, i=1, 2,...,N

[0112] δ1(i) can be used to express the probability that all different categories of category labels appear in the first layer, where i represents the level in the word vector model hierarchy, b i (c 1i ) represents the category label c 1i The probability of observation occurring at level l = 1, π i is the category label c 1i The initial probability of appearing in the i-th layer can be statistically calculated from the annotated text. Different from the label transfer probability, the initial probability can be the probability of the category label only for the I-th layer, and there is no transfer. Therefore, the initial probability of each category label in the first layer can be calculated based on the number of each category label in the l-th layer divided by the total number of texts for the text information. For example, the two categories of category labels in the first layer are C11 and C12, then P(C11) = the number of texts with the text label belonging to the category label C11 / the total number of texts.

[0113] ψ1(i) can be used to record the previous state j (label) of the maximum probability path for the category label; since l = 1 here, that is, the first layer, the previous state j (label) of the category label with l = 1 is 0 by default.

[0114] For 1=2, 3, ..., L, when l and i remain unchanged, j=1, 2, 3, ..., N.

[0115] Formula 5:

[0116] δ l (i) = max[δ l-1 (j)a ji ]b i (c li ), i=1,2,...,N

[0117] Formula 6:

[0118] ψ l (i) = argmax[δ l-1 (j)a ji ], i = 1, 2, ..., N

[0119] δ l (i) It can be used to express the probability that all possible category labels in the l-1 level (one level lower than l) appear in the state j (label) of the l level, that is, the shortest path when all category labels of different categories are in the 1-1 level and the state is j ，此概率受标签转移概率a ji约束，若通过标签转移概率判定转移的可能性为0，则放弃此概率。

[0120] ψ l(i)可以用于表达在l-1层次的标签最有可能转移到l层次的标签i，并记录在l层所有状态i(标签)的最大概率路径的l-1层状态j(标签)。

[0121] 由于每一层都有多个标签，所以上一层所有类别标签与下一层的所有类别标签相连接路径非常多，而只需求在第l层各个标签时上一层标签的最短状态即最短标签，因此可以过滤掉上一层针对类别标签的非最短路径，降低后续计算量。

[0122] Step3：

[0123] 公式七：

[0124] P *＝maxδ L(i)

[0125] 公式八：

[0126]

[0127] P *可以用于表达出现在L层以及L层状态为i(标签)的最大出现概率， 可以用于表达i在L层以及L往上每一层的最短路径＜math>。

[0128] Step4：针对l＝L-1、L-2、...1，根据l＝L最有可能的类别标签反向推出l＝L-1、l＝L-2、...、2、1层次中最有可能的类别标签，即，可以通过公式 回溯多个类别标签两两之间的最短路径，最终可以求得最短路径

[0129] 本发明实施例基于维特比算法，采用所述标签观测概率和所述标签转移概率计算出多个所述类别标签两两之间的最短路径；采用所述最短路径作为针对所述类别标签的分类结果，并采用所述分类结果作为针对所述文本信息的分类结果，从而实现了通过计算不同层次之间的标签转移概率，结合利用既有模型获取到的标签观测概率，基于维特比的多层次分类方法来寻找符合标签层次结构的最优标签集合，对各层预测标签结果加以约束，避免在输出错误标签的置信度高于正确标签的置信度时，将错误标签作为标签预测结果，并将错误信息传递给下层的预测中，从而提高多层次分类的合理性和准确率。

[0130] 在本发明的一个可选地实施例中，所述词向量模型为多语言博特模型的多语言模型，所述生成针对所述文本信息的文本向量，并采用所述文本向量生成文本向量矩阵的步骤包括：

[0131] 采用所述多语言博特模型获取针对所述文本信息的文本向量，并采用所述文本向量生成文本向量矩阵。

[0132] 在实际应用中，词向量模型通常基于英文数据集进行训练，由于词表中包含中文词汇数量有限，且中英文分词方式存在差异，从而导致模型提取的文本特征向量区分度较低，进而导致针对文本信息的多层分类结果准确率较低，所以，本发明实施例可以采用多语言博特模型bert获取针对文本信息的文本向量，并采用文本向量生成文本向量矩阵，Bert来自谷歌的论文，语言理解深度双向变换器的预训练(Pre-training of DeepBidirectional Transformers for Language Understanding)BERT是"BidirectionalEncoder Representations from Transformers”的首字母缩写，Bert整体是一个自编码语言模型。

[0133] 在相关技术中，提取文本向量的方式为，输入文本序列，则可以对应输出一个文本向量。例如，输入一句话作为文本序列，则可以输出1024*1这一特征，而这一方式目前仅适用于英文文本，这是因为现有的词向量模型都是大多基于英文文本进行训练，用单一向量足以满足对文本信息的表达，但如果是中文文本，各分词的信息已经混淆在单一向量中，从而导致无法对其进行细分。所以，本发明实施例可以不直接采用一个特征向量表达文本序列，而是查找在较浅的层数输出的文本向量，具体地，可以通过多语言文本模型依次对每一条文本信息进行分词，即，通过bert对每一条文本序列中的每个词的词性进行分类，从而提取针对多个分词的多个文本向量，并采用多个文本向量生成文本向量矩阵。

[0134] 本发明实施例，通过采用所述多语言博特模型获取针对所述文本信息的文本向量，并采用所述文本向量生成文本向量矩阵，实现了针对中文文本信息的多层次文本分类，提升了针对中文文本的多层次文本分类效率。

[0135] 可选地，本发明实施例还可以采用所述多语言博特模型获取针对所述类别标签的标签向量，并采用所述标签向量生成标签向量矩阵。

[0136] 需要说明的是，对于方法实施例，为了简单描述，故将其都表述为一系列的动作组合，但是本领域技术人员应该知悉，本发明实施例并不受所描述的动作顺序的限制，因为依据本发明实施例，某些步骤可以采用其他顺序或者同时进行。其次，本领域技术人员也应该知悉，说明书中所描述的实施例均属于优选实施例，所涉及的动作并不一定是本发明实施例所必须的。

[0137] 参照 图4，示出了本发明实施例中提供的一种文本信息分类装置的结构框图，具体可以包括如下模块：

[0138] 标签向量矩阵生成模块401，用于生成针对所述类别标签的标签向量，并采用所述标签向量生成标签向量矩阵；

[0139] 文本向量矩阵生成模块402，用于生成针对所述文本信息的文本向量，并采用所述文本向量生成文本向量矩阵；

[0140] 标签转移概率生成模块403，用于生成针对所述类别标签的标签转移概率；

[0141] 标签观测概率生成模块404，用于将所述标签向量矩阵和所述文本向量矩阵输入至所述文本分类模型，并生成针对所述类别标签的标签观测概率；

[0142] 分类结果确定模块405，用于基于所述标签观测概率和所述标签转移概率，确定针对所述类别标签的分类结果，并采用所述分类结果作为针对所述文本信息的分类结果。

[0143] 可选地，所述文本分类模型集成有词向量模型，所述标签节点包括父节点，以及，与所述父节点具有直接关联关系的子节点，所述标签向量矩阵生成模块可以包括：

[0144] 子标签向量确定子模块，用于采用所述词向量模型确定针对所述子节点的子类别标签的子标签向量；

[0145] 子节点个数确定子模块，用于确定所述子节点的个数；

[0146] 父标签向量计算子模块，用于采用所述子标签向量，和，所述子节点的个数计算生成针对所述父节点的父类别标签的父标签向量；

[0147] 标签向量矩阵生成子模块，用于采用所述子标签向量，和，所述父标签向量构建标签向量矩阵。

[0148] 可选地，所述标签转移概率生成模块可以包括：

[0149] 父标签类别数量确定子模块，用于在所述父节点所在的父层级中确定出针对所述父类别标签的父标签类别数量；

[0150] 子标签类别数量确定子模块，用于在所述子节点所在的子层级中确定出针对所述子类别标签的子标签类别数量；

[0151] 标签转移概率生成子模块，用于采用所述父标签类别数量和所述子标签类别数量计算生成针对所述类别标签的标签转移概率。

[0152] 可选地，所述分类结果确定模块可以包括：

[0153] 最短路径计算子模块，用于基于维特比算法，采用所述标签观测概率和所述标签转移概率计算出多个所述类别标签两两之间的最短路径；

[0154] 分类结果确定子模块，用于采用所述最短路径作为针对所述类别标签的分类结果，并采用所述分类结果作为针对所述文本信息的分类结果。

[0155] 可选地，所述词向量模型为多语言博特模型的多语言模型，所述文本向量矩阵生成模块可以包括：

[0156] 文本向量矩阵生成子模块，用于采用所述多语言博特模型获取针对所述文本信息的文本向量，并采用所述文本向量生成文本向量矩阵。

[0157] 对于装置实施例而言，由于其与方法实施例基本相似，所以描述的比较简单，相关之处参见方法实施例的部分说明即可。

[0158] 另外，本发明实施例还提供了一种电子设备，包括：处理器，存储器，存储在存储器上并可在处理器上运行的计算机程序，该计算机程序被处理器执行时实现上述文本信息分类方法实施例的各个过程，且能达到相同的技术效果，为避免重复，这里不再赘述。

[0159] 本发明实施例还提供了一种计算机可读存储介质，计算机可读存储介质上存储有计算机程序，计算机程序被处理器执行时实现上述文本信息分类方法实施例的各个过程，且能达到相同的技术效果，为避免重复，这里不再赘述。其中，所述的计算机可读存储介质，如只读存储器(Read-Only Memory，简称ROM)、随机存取存储器(Random Access Memory，简称RAM)、磁碟或者光盘等。

[0160] 图5为实现本发明各个实施例的一种电子设备的硬件结构示意图。

[0161] 该电子设备500包括但不限于：射频单元501、网络模块502、音频输出单元503、输入单元504、传感器505、显示单元506、用户输入单元507、接口单元508、存储器509、处理器510、以及电源511等部件。本领域技术人员可以理解， 图5中示出的电子设备结构并不构成对电子设备的限定，电子设备可以包括比图示更多或更少的部件，或者组合某些部件，或者不同的部件布置。在本发明实施例中，电子设备包括但不限于手机、平板电脑、笔记本电脑、掌上电脑、车载终端、可穿戴设备、以及计步器等。

[0162] 应理解的是，本发明实施例中，射频单元501可用于收发信息或通话过程中，信号的接收和发送，具体的，将来自基站的下行数据接收后，给处理器510处理；另外，将上行的数据发送给基站。通常，射频单元501包括但不限于天线、至少一个放大器、收发信机、耦合器、低噪声放大器、双工器等。此外，射频单元501还可以通过无线通信系统与网络和其他设备通信。

[0163] 电子设备通过网络模块502为用户提供了无线的宽带互联网访问，如帮助用户收发电子邮件、浏览网页和访问流式媒体等。

[0164] 音频输出单元503可以将射频单元501或网络模块502接收的或者在存储器509中存储的音频数据转换成音频信号并且输出为声音。而且，音频输出单元503还可以提供与电子设备500执行的特定功能相关的音频输出(例如，呼叫信号接收声音、消息接收声音等等)。音频输出单元503包括扬声器、蜂鸣器以及受话器等。

[0165] 输入单元504用于接收音频或视频信号。输入单元504可以包括图形处理器(Graphics Processing Unit，GPU)5041和麦克风5042，图形处理器5041对在视频捕获模式或图像捕获模式中由图像捕获装置(如摄像头)获得的静态图片或视频的图像数据进行处理。处理后的图像帧可以显示在显示单元506上。经图形处理器5041处理后的图像帧可以存储在存储器509(或其它存储介质)中或者经由射频单元501或网络模块502进行发送。麦克风5042可以接收声音，并且能够将这样的声音处理为音频数据。处理后的音频数据可以在电话通话模式的情况下转换为可经由射频单元501发送到移动通信基站的格式输出。

[0166] 电子设备500还包括至少一种传感器505，比如光传感器、运动传感器以及其他传感器。具体地，光传感器包括环境光传感器及接近传感器，其中，环境光传感器可根据环境光线的明暗来调节显示面板5061的亮度，接近传感器可在电子设备500移动到耳边时，关闭显示面板5061和 / 或背光。作为运动传感器的一种，加速计传感器可检测各个方向上(一般为三轴)加速度的大小，静止时可检测出重力的大小及方向，可用于识别电子设备姿态(比如横竖屏切换、相关游戏、磁力计姿态校准)、振动识别相关功能(比如计步器、敲击)等；传感器505还可以包括指纹传感器、压力传感器、虹膜传感器、分子传感器、陀螺仪、气压计、湿度计、温度计、红外线传感器等，在此不再赘述。

[0167] 显示单元506用于显示由用户输入的信息或提供给用户的信息。显示单元506可包括显示面板5061，可以采用液晶显示器(Liquid Crystal Display，LCD)、有机发光二极管(Organic Light-Emitting Diode，OLED)等形式来配置显示面板5061。

[0168] 用户输入单元507可用于接收输入的数字或字符信息，以及产生与电子设备的用户设置以及功能控制有关的键信号输入。具体地，用户输入单元507包括触控面板5071以及其他输入设备5072。触控面板5071，也称为触摸屏，可收集用户在其上或附近的触摸操作(比如用户使用手指、触笔等任何适合的物体或附件在触控面板5071上或在触控面板5071附近的操作)。触控面板5071可包括触摸检测装置和触摸控制器两个部分。其中，触摸检测装置检测用户的触摸方位，并检测触摸操作带来的信号，将信号传送给触摸控制器；触摸控制器从触摸检测装置上接收触摸信息，并将它转换成触点坐标，再送给处理器510，接收处理器510发来的命令并加以执行。此外，可以采用电阻式、电容式、红外线以及表面声波等多种类型实现触控面板5071。除了触控面板5071，用户输入单元507还可以包括其他输入设备5072。具体地，其他输入设备5072可以包括但不限于物理键盘、功能键(比如音量控制按键、开关按键等)、轨迹球、鼠标、操作杆，在此不再赘述。

[0169] 进一步的，触控面板5071可覆盖在显示面板5061上，当触控面板5071检测到在其上或附近的触摸操作后，传送给处理器510以确定触摸事件的类型，随后处理器510根据触摸事件的类型在显示面板5061上提供相应的视觉输出。虽然在 图5中，触控面板5071与显示面板5061是作为两个独立的部件来实现电子设备的输入和输出功能，但是在某些实施例中，可以将触控面板5071与显示面板5061集成而实现电子设备的输入和输出功能，具体此处不做限定。

[0170] 接口单元508为外部装置与电子设备500连接的接口。例如，外部装置可以包括有线或无线头戴式耳机端口、外部电源(或电池充电器)端口、有线或无线数据端口、存储卡端口、用于连接具有识别模块的装置的端口、音频输入 / 输出(I / O)端口、视频I / O端口、耳机端口等等。接口单元508可以用于接收来自外部装置的输入(例如，数据信息、电力等等)并且将接收到的输入传输到电子设备500内的一个或多个元件或者可以用于在电子设备500和外部装置之间传输数据。

[0171] 存储器509可用于存储软件程序以及各种数据。存储器509可主要包括存储程序区和存储数据区，其中，存储程序区可存储操作系统、至少一个功能所需的应用程序(比如声音播放功能、图像播放功能等)等；存储数据区可存储根据手机的使用所创建的数据(比如音频数据、电话本等)等。此外，存储器509可以包括高速随机存取存储器，还可以包括非易失性存储器，例如至少一个磁盘存储器件、闪存器件、或其他易失性固态存储器件。

[0172] 处理器510是电子设备的控制中心，利用各种接口和线路连接整个电子设备的各个部分，通过运行或执行存储在存储器509内的软件程序和 / 或模块，以及调用存储在存储器509内的数据，执行电子设备的各种功能和处理数据，从而对电子设备进行整体监控。处理器510可包括一个或多个处理单元；优选的，处理器510可集成应用处理器和调制解调处理器，其中，应用处理器主要处理操作系统、用户界面和应用程序等，调制解调处理器主要处理无线通信。可以理解的是，上述调制解调处理器也可以不集成到处理器510中。

[0173] 电子设备500还可以包括给各个部件供电的电源511(比如电池)，优选的，电源511可以通过电源管理系统与处理器510逻辑相连，从而通过电源管理系统实现管理充电、放电、以及功耗管理等功能。

[0174] 另外，电子设备500包括一些未示出的功能模块，在此不再赘述。

[0175] 需要说明的是，在本文中，术语"包括”、"包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者装置不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者装置所固有的要素。在没有更多限制的情况下，由语句"包括一个……”限定的要素，并不排除在包括该要素的过程、方法、物品或者装置中还存在另外的相同要素。

[0176] 通过以上的实施方式的描述，本领域的技术人员可以清楚地了解到上述实施例方法可借助软件加必需的通用硬件平台的方式来实现，当然也可以通过硬件，但很多情况下前者是更佳的实施方式。基于这样的理解，本发明的技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质(如ROM / RAM、磁碟、光盘)中，包括若干指令用以使得一台终端(可以是手机，计算机，服务器，空调器，或者网络设备等)执行本发明各个实施例所述的方法。

[0177] 上面结合附图对本发明的实施例进行了描述，但是本发明并不局限于上述的具体实施方式，上述的具体实施方式仅仅是示意性的，而不是限制性的，本领域的普通技术人员在本发明的启示下，在不脱离本发明宗旨和权利要求所保护的范围情况下，还可做出很多形式，均属于本发明的保护之内。

[0178] 本领域普通技术人员可以意识到，结合本发明实施例中所公开的实施例描述的各示例的单元及算法步骤，能够以电子硬件、或者计算机软件和电子硬件的结合来实现。这些功能究竟以硬件还是软件方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本发明的范围。

[0179] 所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，上述描述的系统、装置和单元的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。

[0180] 在本申请所提供的实施例中，应该理解到，所揭露的装置和方法，可以通过其它的方式实现。例如，以上所描述的装置实施例仅仅是示意性的，例如，所述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，装置或单元的间接耦合或通信连接，可以是电性，机械或其它的形式。

[0181] 所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。

[0182] 另外，在本发明各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。

[0183] 所述功能如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本发明的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本发明各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、ROM、RAM、磁碟或者光盘等各种可以存储程序代码的介质。

[0184] 以上所述，仅为本发明的具体实施方式，但本发明的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本发明揭露的技术范围内，可轻易想到变化或替换，都应涵盖在本发明的保护范围之内。因此，本发明的保护范围应以权利要求的保护范围为准。

Claims

1. A text information classification method, characterized in that: The method is applied to a text classification model, wherein the text classification model has multiple levels, each level includes corresponding label nodes, each label node has a corresponding category label, and the category label is used to express the category of the label node. The method includes: Generate a label vector for the category label, and use the label vector to generate a label vector matrix; Generating a text vector for the text information, and generating a text vector matrix using the text vector; generating a label transition probability for the class label; Inputting the label vector matrix and the text vector matrix into the text classification model, and generating label observation probabilities for the category labels; Based on the label observation probability and the label transition probability, a classification result for the category label is determined, and the classification result is used as a classification result for the text information.

2. The method according to claim 1, characterized in that The text classification model is integrated with a word vector model, the label node includes a parent node and a child node directly associated with the parent node, and the steps of generating a label vector for the category label and using the label vector to generate a label vector matrix include: Determine a sub-label vector for the sub-category label of the child node using the word vector model; Determine the number of the child nodes; Generate a parent label vector for the parent category label of the parent node by using the child label vector and the number of child nodes; A label vector matrix is ​​constructed using the child label vector and the parent label vector.

3. The method according to claim 2, characterized in that The step of generating the label transition probability for the category label comprises: Determining the number of parent tag categories for the parent category tag in the parent level where the parent node is located; Determining the number of sub-tag categories for the sub-category tag in the sub-level where the child node is located; The label transition probability for the category label is calculated and generated using the number of parent label categories and the number of child label categories.

4. The method according to claim 3, characterized in that The step of determining a classification result for the category label based on the label observation probability and the label transition probability, and using the classification result as a classification result for the text information includes: Based on the Viterbi algorithm, the shortest paths between the multiple category labels are calculated using the label observation probability and the label transition probability; The shortest path is used as a classification result for the category label, and the classification result is used as a classification result for the text information.

5. The method according to claim 2, characterized in that The word vector model is a multilingual model of a multilingual bot model, and the steps of generating a text vector for the text information and generating a text vector matrix using the text vector include: The multilingual bot model is used to obtain text vectors for the text information, and the text vectors are used to generate a text vector matrix.

6. A text information classification device, characterized in that: The device is applied to a text classification model, wherein the text classification model has multiple levels, each level includes corresponding label nodes, each label node has a corresponding category label, and the category label is used to express the category of the label node. The device includes: A label vector matrix generation module, configured to generate a label vector for the category label and generate a label vector matrix using the label vector; A text vector matrix generation module, configured to generate a text vector for the text information and generate a text vector matrix using the text vector; A label transition probability generation module, used to generate a label transition probability for the category label; a label observation probability generation module, configured to input the label vector matrix and the text vector matrix into the text classification model and generate a label observation probability for the category label; A classification result determination module is used to determine a classification result for the category label based on the label observation probability and the label transition probability, and adopt the classification result as the classification result for the text information.

7. The device according to claim 6, characterized in that The text classification model is integrated with a word vector model, the label node includes a parent node and a child node directly associated with the parent node, and the label vector matrix generation module includes: a sub-label vector determination sub-module, configured to determine a sub-label vector for the sub-category label of the sub-node using the word vector model; A sub-node number determination sub-module, used to determine the number of the sub-nodes; A parent label vector calculation submodule, configured to calculate and generate a parent label vector for the parent category label of the parent node by using the child label vector and the number of child nodes; The label vector matrix generation submodule is used to construct a label vector matrix using the child label vector and the parent label vector.

8. The device according to claim 7, characterized in that The label transfer probability generation module includes: A parent tag category quantity determination submodule, configured to determine the number of parent tag categories for the parent category tag in the parent level where the parent node is located; A sub-label category quantity determination sub-module, configured to determine the number of sub-label categories for the sub-category label in the sub-level where the sub-node is located; The label transition probability generating submodule is configured to calculate and generate a label transition probability for the category label by using the number of the parent label category and the number of the child label category.

9. The device according to claim 8, characterized in that The classification result determination module includes: A shortest path calculation submodule, configured to calculate the shortest paths between each of the plurality of category labels using the label observation probability and the label transition probability based on a Viterbi algorithm; The classification result determination submodule is configured to adopt the shortest path as the classification result for the category label, and adopt the classification result as the classification result for the text information.

10. The device according to claim 7, characterized in that The word vector model is a multilingual model of the multilingual Bot model, and the text vector matrix generation module includes: The text vector matrix generation submodule is used to obtain text vectors for the text information using the multilingual bot model, and generate a text vector matrix using the text vectors.

11. An electronic device, characterized in that: comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; The memory is used to store computer programs; The processor is configured to implement the method according to any one of claims 1 to 5 when executing a program stored in the memory.

12. A computer-readable storage medium having instructions stored thereon, which, when executed by one or more processors, cause the processors to perform the method according to any one of claims 1 to 5.

Images