Text classification method, classification device and electronic equipment

By extracting word segmentation and word frequency characteristics of the target text and combining entity and entity type characteristics for fusion classification, the problem of low classification accuracy in the multi-classification problems in the prior art is solved, and efficient text classification results are achieved.

CN118820478BActive Publication Date: 2024-11-29CHENGDU CELIS TECH CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202411303961.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-09-19
Publication Date
2024-11-29
Estimated Expiration
2044-09-19

AI Technical Summary

Technical Problem

When existing long text classification technology deals with multi-classification problems with close boundaries between classes, the classification accuracy is low and it is prone to misrecall problems.

Method used

By extracting the first feature of the target text based on word segmentation and word frequency, and matching it with the pre-constructed target double array to obtain the second feature of the entity feature and the entity type feature, perform feature fusion, and use a classifier to classify the fused feature vectors .

Benefits of technology

It improves the accuracy of text classification, especially in multi-classification problems, effectively avoids misrecalls, and compared with deep learning methods, it shorter training time and less computing resource consumption, making it suitable for application scenarios with limited resources.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118820478B_ABST
    Figure CN118820478B_ABST
Patent Text Reader

Abstract

The present application relates to the field of artificial intelligence technology, and provides a text classification method, a classification device, and an electronic device. The method extracts a first feature from a target text according to the word segmentation quantity and the word frequency, and obtains a second feature including a target text entity feature and an entity type feature by matching with a pre-constructed first target double array, and then fuses the first feature and the second feature, and uses a classifier and the fused first target feature vector to classify the target text, thereby obtaining a classification result with high accuracy.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a text classification method, a classification device and an electronic device. Background Art

[0002] Natural Language Processing (NLP) is an important branch of artificial intelligence that studies how to make computers understand and generate human language. Text classification is an important task in natural language processing, and its goal is to classify text documents into predefined categories.

[0003] Existing long text classification technologies may have low classification accuracy when dealing with classification problems, especially multi-classification problems where the boundaries between classes are close. Summary of the invention

[0004] In view of this, the embodiments of the present application provide a text classification method, a classification device and an electronic device to solve the problem of low classification accuracy in the prior art.

[0005] A first aspect of an embodiment of the present application provides a text classification method, comprising:

[0006] Get the target text;

[0007] According to the word count and word frequency, the first feature of the target text is obtained;

[0008] Matching the target text with a pre-constructed first target double array to obtain a second feature of the target text, the second feature including an entity feature and an entity type feature of the target text;

[0009] Perform feature fusion based on the first feature and the second feature to obtain a first target feature vector;

[0010] The first target feature vector is classified using a classifier to obtain a classification result of the target text, where the classification result includes a category of the target text.

[0011] A second aspect of the embodiments of the present application provides a text classification device, including:

[0012] An acquisition module, configured to acquire a target text;

[0013] A feature extraction module is configured to obtain a first feature of the target text according to the word segmentation quantity and word frequency;

[0014] The feature extraction module is further configured to match the target text with a pre-constructed first target double array to obtain a second feature of the target text, the second feature comprising an entity feature and an entity type feature of the target text;

[0015] A vector concatenation module is configured to perform feature fusion based on the first feature and the second feature to obtain a first target feature vector;

[0016] The classification module is configured to use a classifier to classify the first target feature vector to obtain a classification result of the target text, wherein the classification result includes a category of the target text.

[0017] According to a third aspect of an embodiment of the present application, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.

[0018] The beneficial effects of the embodiments of the present application compared with the prior art are as follows: the embodiments of the present application extract the first feature of the target text according to the word segmentation quantity and the word frequency respectively, and obtain the second feature including the target text entity feature and the entity type feature by matching with the pre-constructed first target dual array, and then fuse the first feature and the second feature, and use the classifier and the fused first target feature vector to classify the target text, so as to obtain a classification result with higher accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0020] Figure 1 It is a flowchart of a text classification method provided in an embodiment of the present application.

[0021] Figure 2 It is a flowchart of a method for constructing a first target dual array provided in an embodiment of the present application.

[0022] Figure 3 It is a flowchart of a method for constructing a Trie tree according to M entity libraries and then constructing a first target double array provided in an embodiment of the present application.

[0023] Figure 4 It is a flowchart of a method for calculating the basic array value corresponding to each child node and checking the array value according to the encoding value of each entity provided in an embodiment of the present application.

[0024] Figure 5 It is a schematic diagram of the Trie tree constructed according to the embodiment of the present application.

[0025] Figure 6It is a flowchart of a method for obtaining a first target feature vector by performing feature fusion based on a first feature and a second feature provided in an embodiment of the present application.

[0026] Figure 7 It is a schematic diagram of an implementation method provided in an embodiment of the present application of adding a first eigenvector and a second eigenvector and then averaging them to obtain a first target eigenvector.

[0027] Figure 8 It is a schematic diagram of an implementation method for concatenating a first feature vector and a second feature vector to obtain a first target feature vector provided in an embodiment of the present application.

[0028] Fig. 9 It is a flowchart of another text classification method provided in an embodiment of the present application.

[0029] Fig.10 It is a schematic diagram of a text classification device provided in an embodiment of the present application.

[0030] Fig.11 It is a schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0031] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0032] A text classification method and device according to an embodiment of the present application will be described in detail below with reference to the accompanying drawings.

[0033] Existing long text classification technologies may have low classification accuracy when dealing with classification problems, especially multi-classification problems where the boundaries between classes are close.

[0034] In view of this, an embodiment of the present application provides a text classification method, which extracts a first feature from a target text based on word segmentation quantity and word frequency, and obtains a second feature including a target text entity feature and an entity type feature by matching with a pre-constructed first target dual array, and then fuses the first feature and the second feature, and uses a classifier and the fused first target feature vector to classify the target text, thereby obtaining a classification result with higher accuracy.

[0035] Figure 1 Schematic diagram of a text classification method provided in an embodiment of the present application. Figure 1 As shown, the method comprises the following steps:

[0036] In step S101, the target text is obtained.

[0037] In step S102, the first feature of the target text is obtained according to the word segmentation quantity and word frequency.

[0038] In step S103, the target text is matched with the pre-constructed first target double array to obtain the second feature of the target text.

[0039] The second feature includes entity features and entity type features of the target text.

[0040] In step S104, feature fusion is performed based on the first feature and the second feature to obtain a first target feature vector.

[0041] In step S105, a classifier is used to classify the first target feature vector to obtain a classification result of the target text, where the classification result includes a category of the target text.

[0042] In certain embodiments of the present application, the method may be executed by a server, or by a terminal with certain processing capabilities. Among them, the server may be a single server, or a server cluster consisting of several servers, or a cloud computing service center, which is not limited in the embodiments of the present application. The terminal may be a terminal equipped with a processing unit or a computing unit with certain processing capabilities, such as a terminal equipped with an edge computing unit, or others, which is not limited in the embodiments of the present application.

[0043] In some embodiments, the server or the terminal may first obtain a target text, which is a text to be classified.

[0044] In some implementations, the first feature of the target text may be obtained. For example, the first feature of the target text may be obtained based on the word segmentation quantity and word frequency. That is, feature extraction may be performed on the target text based on the word segmentation quantity and word frequency to obtain the first feature of the target text.

[0045] In some embodiments, the second feature of the target text may also be obtained. For example, the target text may be matched with a pre-constructed first target double array to obtain the second feature of the target text. The second feature of the target text may include an entity feature and an entity type feature of the target text.

[0046] In some embodiments of the present application, an entity refers to a thing or concept that can be defined in the system, such as a person / role (such as a student), an object (such as a ticket), a concept (such as a profile), or an event (such as a transaction). Entities can be divided into different types according to actual needs.

[0047] After obtaining the first feature and the second feature of the target text, the first feature and the second feature can be fused to obtain a first target feature vector. Feature fusion refers to extracting different feature vectors from the same pattern in a serial or parallel manner to optimize the combination.

[0048] In some implementations, a classifier may be obtained, which may be an untrained classifier or a classifier obtained by training an initialization classifier. Further, the classifier may be used to classify the first target feature vector obtained by the above fusion to obtain a classification result of the target text.

[0049] For example, the first target feature vector can be input into a classifier to obtain an output vector, which can include one or more categories and prediction weights for each category. The target category in the output vector is determined as the classification result of the target text, so that the category of the target text can be obtained. Among them, the category can be the class to which the text belongs, such as a health category, an entertainment category, etc. The category can use an existing category or can be defined according to actual needs, which is not limited here.

[0050] In some embodiments of the present application, the classifier may be, for example, a fully connected layer in a classification model, or other classifiers. The classification model may be, for example, a fast text classification algorithm fasttext classification model, or a BERT model, or other classification models, which are not limited here.

[0051] According to the technical solution provided in the embodiment of the present application, by extracting the first feature of the target text according to the word segmentation quantity and the word frequency, and obtaining the second feature including the target text entity feature and the entity type feature by matching with the pre-constructed first target dual array, and then fusing the first feature and the second feature, the category of the target text is obtained by using a classifier and the fused first target feature vector, and a higher precision classification result can be obtained.

[0052] In related technologies, long text classification is usually performed using methods based on the text classification model TextCNN or the Transformer-based bidirectional encoder representation model (Bidirectional Encoder Representations from Transformers, BERT). The TextCNN-based method mainly extracts the features of words in the text through different convolution kernels, and then concatenates the convolutional features of different sizes together as the features of the text after passing through the pooling layer. The BERT-based method mainly divides the long text into sentences, extracts features using BERT respectively, passes through the LSTM output layer and then concatenates the Attention layer to obtain the features of the text, and then performs classification.

[0053] However, the TextCNN-based method is suitable for short text classification and cannot handle unregistered words (OOV), and the pooling operation will lose some useful features, resulting in low model classification accuracy. At the same time, although the BERT-based method has a certain improvement in feature representation capabilities, it needs to segment sentences and extract features for fusion when processing long texts, and the training process consumes a lot of computing resources and takes a long time to train. In addition, when these methods deal with classification problems, especially multi-classification problems with close boundaries between classes, the classification accuracy may be low, and false recall problems are prone to occur between classes.

[0054] Therefore, an embodiment of the present application provides an implementation scheme for extracting a first feature of a target text based on word segmentation quantity and word frequency, respectively, and obtaining a second feature of the target text using a pre-built dual array matching, and then fusing the first feature and the second feature to obtain a first target feature vector of text features including multiple dimensions such as entities and entity types, and then using a classifier to classify the first target feature vector to obtain a category of the target text, thereby solving the above-mentioned problem that when processing multi-classification problems with close boundaries between classes, the classification accuracy may be low and false recall is prone to occur between classes.

[0055] In certain embodiments of the present application, obtaining the first feature of the target text according to the word segmentation quantity and word frequency may be to extract features of the target text using the N-Gram algorithm to obtain the first feature of the target text. The N-Gram algorithm is an algorithm based on a statistical language model, and its basic idea is to slide a text sequence according to a window of size n, and take adjacent co-occurring words of length n, where n is a positive integer.

[0056] Taking "I am Chinese" as an example, when n=1, the features extracted using the N-Gram algorithm are: [I, am, Chinese, person]; when n=2, the features extracted using the N-Gram algorithm are: [I am, is Chinese, Chinese, person]; when n=3, the features extracted using the N-Gram algorithm are: [I am Chinese, is Chinese, Chinese], and so on.

[0057] When using N-Gram to extract the first feature of the target text, you can use n=1, 2, 3, etc. to solve the word order problem to a certain extent. After using the N-Gram algorithm to extract features, the vocabulary will become larger. At this time, in order to save space, the features extracted by the N-Gram algorithm can be hashed first during implementation. Multiple N-Gram features hashed to the same position will share an embedding expression.

[0058] Figure 2 is a flow chart of a method for constructing a first target dual array provided in an embodiment of the present application. Figure 2 As shown, the method comprises the following steps:

[0059] In step S201, a first text sample is obtained.

[0060] In step S202, M entity libraries are constructed according to the first text sample.

[0061] Each entity library is configured with an entity type, and each entity library includes at least one entity in the first text sample.

[0062] Furthermore, M is a positive integer.

[0063] In step S203, a query Trie tree is constructed according to the M entity libraries.

[0064] In step S204, a first target double array is constructed according to the Trie tree.

[0065] The first target dual array at least includes a basic array, and each array element of the basic array may include a physical feature, or include a physical feature and a physical type feature.

[0066] When constructing the first target double array, a first text sample may be first obtained. The first text sample may include multiple texts, and each text in the first text sample is marked with a text type. The marked text type may be set according to actual needs. In one example, the text type may include, but is not limited to, types under multiple categories such as entertainment, geography, health care, technology, sports, food and beverages, etc.

[0067] Furthermore, M entity libraries can be constructed according to the text types annotated by each text in the first text sample, wherein each entity library corresponds to a text type, and multiple entity libraries can correspond to the same text type. In other words, each entity library can be configured with an entity type, and the entity type is strongly related to the text type.

[0068] Taking the health and medical text type as an example, it can include entity libraries of diseases, drug ingredients, symptoms, diets, etc. Taking the entertainment text type as an example, it can include entity libraries of singers, song titles, actors, film and television works, operas, etc. These entity libraries remain independent, with occasional overlaps. For example, in the singer and actor entity libraries, there may be an entity that is both a singer and an actor. It should be noted that these entities have no logical dependencies.

[0069] Furthermore, each entity library may be configured with at least one entity in the first text sample, and the text type of the text in which the entity is located corresponds to the type of the entity library. For example, a text is marked as an entertainment text type, and the entities therein may be saved to a singer entity library, a song title entity library, an actor entity library, a film and television work entity library, a drama entity library, etc. according to actual conditions.

[0070] After the entity library is constructed, a query (Trie) tree can be constructed using the entity library. That is, the constructed M entity libraries can be used to construct a Trie tree. Furthermore, the constructed Trie tree can be used to further construct a first target double array. The first target double array can include a base array and check data, and each array element in the base array includes at least entity features, and in some cases can include entity features and entity type features.

[0071] In certain embodiments of the present application, when the target text is matched with a pre-constructed first target double array to obtain the second feature of the target text, the target text can be matched with the first target double array to obtain multiple matching results, wherein each matching result corresponds to a set of values ​​in the first target double array. The target matching result can be obtained by averaging the multiple matching results, and the value in the first target double array corresponding to the target matching result can constitute the second feature.

[0072] In order to enrich the semantics of the text, the dual array string search multi-pattern matching (DAT-AC) algorithm can be used to extract features of the target text to obtain the second feature of the target text. The DAT-AC algorithm is an improved DAT-AC algorithm. The extracted second feature includes not only the entity features of the target text, but also the entity type features of the target text. That is, the second feature can include one or more entity features and entity type feature pairs.

[0073] Figure 3 1 is a flow chart of a method for constructing a Trie tree and then constructing a first target double array according to M entity libraries provided in an embodiment of the present application. Figure 3 As shown, the method comprises the following steps:

[0074] In step S301, all entities in M ​​entity repositories are obtained.

[0075] In step S302, a root node of the Trie tree is created, and the basic array value and the check array value corresponding to the root node are initialized.

[0076] In step S303, in the Trie tree, child nodes of each level are inserted in sequence starting from the root node to obtain a constructed Trie tree.

[0077] Among them, each child node corresponds to an entity.

[0078] In step S304, the basic array value and the check array value corresponding to each child node are calculated according to the encoding value of each entity to obtain a first target double array.

[0079] In some embodiments of the present application, a Trie tree may be first constructed based on the created entity library, and then a first target double array may be constructed using the Trie tree, and then the first target double array may be used to match the target text to obtain a second feature of the target text.

[0080] When constructing the Trie tree and the first target double array according to the created entity library, all entities in the M entity libraries can be first obtained, and further, each entity in the entire entity library can be encoded to obtain the encoding value of each entity. Different entities have different encoding values, and the same entity in different entity libraries has the same encoding value.

[0081] In one example, an 8-bit (Universal Character Set / Unicode Transformation Format, UTF-8) character encoding method may be used to encode each entity to obtain the encoding value of each entity. Alternatively, other encoding methods may be used, which are not limited here.

[0082] On the other hand, a root node of the Trie tree can be created, and the basic array value and the check array value corresponding to the root node can be initialized. The root node corresponds to the first array element in the basic array and the check array, and the basic array value and the check array value corresponding to the root node can be initialized by setting the entity feature value of the first array element in the basic array to 1, the entity type feature value to null, and the value of the first array element in the check array to 0.

[0083] Next, you can insert child nodes of each level in the Trie tree starting from the root node to obtain a constructed Trie tree. Each child node corresponds to an entity, and the parent-child relationship between nodes is determined by the order of entities. For example, in a certain entity library, there are entities s, h, and e, and the entity is extracted from the content "she" in the text, so its order is s, h, and e, and the node corresponding to entity h is the child node of the node corresponding to entity s, and the node corresponding to entity e is the child node of the node corresponding to entity h.

[0084] When inserting a child node, the basic array value and the check array value corresponding to each child node can be calculated according to the encoding value of the entity corresponding to each child node, so as to obtain the first target double array.

[0085] It should be noted that the array sizes of the base array and the check array in the first target double array are determined according to the entity library and the entity encoding. For different entity libraries or entity library combinations, or when the number of entities used for encoding in the same entity library is different, the sizes of the base array and the check array in the first target double array may be different.

[0086] Figure 4 1 is a flow chart of a method for calculating the basic array value corresponding to each child node and checking the array value according to the encoding value of each entity provided in an embodiment of the present application. Figure 4 As shown, the method comprises the following steps:

[0087] In step S401, the current node and all child nodes of the current node are obtained.

[0088] In step S402, a preset value is determined according to entity code values ​​of all child nodes of the current node.

[0089] Among them, the preset value meets the following conditions: in the check array, the check array element value whose serial number is the sum of the preset value and the entity code value of any child node of the current node is 0, and the serial numbers of each array element in the basic array and the check array are numbered sequentially starting from 0.

[0090] In step S403, the basic array value of the current node is determined according to a preset value, and the basic array entity feature value of the parent node of the current node is determined as the check array value of the current node.

[0091] In some embodiments of the present application, when the basic array value and check array value corresponding to each child node are calculated based on the coding value of each entity, the current node and all child nodes of the current node can be obtained first, and then the preset value can be determined based on the coding values ​​of all child nodes of the current node.

[0092] Among them, the preset value needs to meet the following conditions: in the check array, the check array element value whose serial number is the sum of the preset value and the entity code value of any child node of the current node is 0, and the serial numbers of each array element in the basic array and the check array are sequentially numbered from 0. Furthermore, when there are multiple values ​​that meet the above conditions at the same time, the smallest value can be selected as the preset value. Since the serial numbers of each element in the basic array and the check array are sequentially numbered from 0, the serial number of the element in the check array can be determined according to the position of the element in the array. For example, the serial number of the first element in the check array is 0, the serial number of the second element is 1, the serial number of the third element is 2, and so on.

[0093] After the preset value is determined, the basic array value of the current node can be determined according to the preset value, and the basic array entity feature value of the parent node of the current node is used as the check array value of the current node.

[0094] In some implementations, determining the base array value of the current node according to the determined preset value may be, if the current node is a node corresponding to a non-end entity in the target entity library, then setting the base array entity feature value of the current node to the preset value, and setting the entity type feature value of the base array of the current node to null. On the other hand, if the current node is a node corresponding to an end entity in the target entity library, then setting the base array entity feature value of the current node to the negative of the preset value, and setting the entity type feature value of the base array of the current node to the entity type of the target entity library.

[0095] The ending entity may be the last entity of one or more words, sentences, etc. in the text. For example, for "she" in the text, the entity "e" may be the ending entity. For another example, for "I love China" in the text, the entity "country" may also be the ending entity. The ending entity may be marked in each entity library. Alternatively, the last entity in a set of sequentially arranged entities may be directly determined as the ending entity.

[0096] The following takes the example of constructing an entity library with a first entity type et1 and an entity library with a second entity type et2 based on a first text sample, and the entity library with entity type et1 includes entities {he, hers, she}, and the entity library with entity type et2 includes entities {his, she}, and describes in detail the method of constructing the Trie tree and the first target double array.

[0097] First, we can construct the encoding of all characters included in each entity. The constructed encoding is shown in Table 1:

[0098]

[0099] Then create the root node root. Let the base array be represented by base and the check array be represented by check. To initialize the root node root, you can set base[root].first=1, that is, set the first value of the root node's base array to 1.

[0100] Next, find the child node set {root.children} (i=1…n) of the root node root, so that check[root.children] = base[root].first = 1, that is, the values ​​of the check array of all child nodes of the root node root are the same as the first value of the base array of the root node root.

[0101] Then for each child node root.children of the root node, find its child node element.children, and then select a preset value begin, so that the check array of each child node element.children satisfies check[begin + element.children.code] = 0, where element.children.code is the encoding of the character corresponding to the child node element.children. Then set base[element.children].first to begin. Recursively execute the above method of selecting begin for each child node until all child nodes are traversed. If a child node has no child nodes when traversed, that is, the child node is a leaf node, then the first value of its base array can be set to a negative value (usually obtained by negatively indexing its sequence number in the dictionary).

[0102] Assume that the current state of a child node is s, and the characters of the successor node of s are {A, B, C, ...}, then base[s] needs to satisfy:

[0103] base[s].first + code(A)=base[s].first + code(B)=base[s].first + code(C)=...=0; and

[0104] check[base[s].first + code(A)]= check[base[s].first + code(B)]= check[...] = 0. Here, code() is the character encoding, and check[...] = 0 means that the corresponding position of ... is empty.

[0105] Furthermore, after determining the value of base[s].first, the check value of the child node position can be set to s, that is, check[base[s].first + code(A)] = check[base[s].first + code(B)] = ... = s.

[0106] When base[s].first represents the end entity, base[s].second can be set to the entity type. If the entity exists in multiple libraries, special symbols such as # are used to separate the entity library types. The Trie tree thus constructed is as follows Figure 5As shown. Among them, the root node is node 0, and the node tree includes four branches: 1-2-3, 4-5, 4-5-6-7 and 4-8-9. Nodes 3, 5, 7 and 9 are nodes corresponding to the ending entities, which have the entity type identification of the entity library where the node is located. For example, node 3 corresponds to the entity she, which exists in the entity libraries corresponding to et1 and et2, so it can be identified as {she, et1#2}. Node 5 corresponds to the entity he, which exists in the entity library corresponding to et1, so it can be identified as {he, et1}. Node 7 corresponds to the entity hers, which exists in the entity library corresponding to et1, so it can be identified as {hers, et1}, and node 9 corresponds to the entity his, which exists in the entity library corresponding to et2, so it can be identified as {his, et2}.

[0107] Next, we need to initialize the double array of the trie tree. First, we can write the state of the root node to 1, check to 0, and initialize the values ​​of the corresponding positions of the first-level nodes h and s to 1. Since there is no prefix, check is set to -1, and the double array is shown in Table 2:

[0108]

[0109] For the character s of the first-level node, since code(s)=3, we can determine the value of base[3].first. The successor node character of the character s of the first-level node is h, and code(h)=1. When begin=0, check[0+code(h)]=check[1]!=0, which is not true; when begin=1, check[1+code(h)]=check[2]=0, which is true. Therefore, the value of begin is 1, that is, base[3].first=1.

[0110] At the same time, the state transition needs to satisfy check[t]=base[s], that is, check[begin+base[s].first]=base[s], so check[2]=3. The arrays of base and check after assignment are shown in Table 3:

[0111]

[0112] For the character h of the first-level node, since code(h)=1, the value of base[1].first can be determined. The successor nodes of the character h of the first-level node are e and i, and code(e)=2, code(i)=4. Recursive calculation: when begin = 0, since check[begin+code(e)]=check[2] != 0, it does not hold; when begin = 1, since check[begin+code(e)]=base[3].first != 0, it does not hold; when begin = 2, check[begin+code(e)]=base[4].first == check[begin+code(i)]=base[6].first == 0&&check[4] == check[6]== 0, it holds. Therefore, the value of begin is 2, that is, base[1].first = 2.

[0113] At the same time, the check array of the child node corresponding to the first-level character h needs to be assigned: check[base[1].first + code(e)]= check[base[1].first + code(i)]= 1. The assigned base and check arrays are shown in Table 4:

[0114]

[0115] For the character h of the second-level node, its corresponding state is base[2].first, and it has only one child node e, code(e)=2. Recursive calculation shows that when begin = 0, check[begin+code(e)]=check[2] != 0, which is not true; when begin = 1, base[3].first != 0, which is not true; when begin = 2, check[4] != 0, which is not true; when begin = 3, base[5].first == 0&check[5] == 0, which is true. Therefore, the value of begin is 3, so base[2]=3, check[5] = 2.

[0116] The base and check arrays after assignment are shown in Table 5:

[0117]

[0118] For the character e of the second-layer node, its corresponding state is base[4].first, and it has only one child node r, code(r) = 5. Recursive calculation shows that when begin = 0, check[begin+code(r)]=check[5]!= 0, which is not true; when begin = 1, base[6].first != 0, which is not true; when begin = 2, base[7].first == 0&check[7]== 0, which is true, so base[4].first = 2, check[7] = 4. Because he forms an entity, the node corresponding to the second-layer character e is the end node, so its base state is set to a negative value, that is, base[4].first = -2, and the entity type is added to the base array, that is, base[4].second = et1. The assigned base and check arrays are shown in Table 6:

[0119]

[0120] For the character i of the second-level node, its corresponding state is base[6].first, and it has only one child node s, code(s) = 3. Recursive calculation shows that when begin = 0, check[begin+code(s)]=check[3]!= 0, which is not true; when begin = 1, check[4]!= 0, which is not true; when begin = 2, check[5]!= 0, which is not true; when begin = 3, check[6]!= 0, which is not true; when begin = 4, check[7]!= 0, which is not true; when begin = 5, check[8]! == 0, which is true, so begin = 5, that is, base[6].first = 5, check[8] = 6. The base and check arrays after assignment are shown in Table 7:

[0121]

[0122] The same rules are used to process the characters e, r, s of the third-layer nodes and the character s of the fourth-layer nodes, and the constructed base and check arrays are shown in Table 8:

[0123]

[0124] So far, the initialization of the double array of the trie tree has been completed. It should be noted that in the specific implementation, when defining the array, the base array is usually directly defined as an integer (int type), that is, through the statement std::vector<int>Create an integer container (i.e., an array). Since the base array needs to store not only the current state value, but also the entity type corresponding to the entity for the array element at the end node, the array type needs to be converted. For example, through the statement std::vector <std::pair<int, std::string> >Increase the type of each node and convert the integer container into a pair of elements in each node of the container, where the first element is of int type and the second element is of string type.

[0125] The technical solution provided in the embodiment of the present application performs feature matching on the target text through an improved DAT-AC algorithm, and the matched entity results are in the format of entity-entity type (ent-type) pairs, thereby achieving the integration of features of entities and entity types without adding additional overhead.

[0126] Furthermore, the improved DAT-AC multi-pattern matching algorithm can be implemented by a C++ algorithm, all of which are defined in the header file. The compiled functions are called using Python's external function library ctypes, and can be used in Python training and inference.

[0127] Figure 6 is a flow chart of a method for obtaining a first target feature vector by performing feature fusion based on a first feature and a second feature provided in an embodiment of the present application. Figure 6 As shown, the method comprises the following steps:

[0128] In step S601, a first feature vector is obtained according to the first feature, and a second feature vector is obtained according to the second feature.

[0129] In step S602, the first eigenvector and the second eigenvector are added and then averaged to obtain a first target eigenvector.

[0130] In step S603, the first feature vector and the second feature vector are concatenated to obtain a first target feature vector.

[0131] In some embodiments of the present application, the extracted first feature can be converted into a first feature vector, and the second feature can be converted into a second feature vector, and then the first target feature vector is obtained according to the first feature vector and the second feature vector. The dimensions of the first feature vector and the second feature vector can be set according to actual needs, for example, to 256.

[0132] For example, the first eigenvector and the second eigenvector may be added and then averaged to obtain the first target eigenvector. Alternatively, the first eigenvector and the second eigenvector may be concatenated to obtain the first target eigenvector.

[0133] That is to say, there are two ways to fuse N-Gram features and entity features. One is add, that is, the two features are directly added, and the other is concat, that is, the two feature vectors are concatenated.

[0134] Figure 7 Schematic diagram of an implementation method of adding the first eigenvector and the second eigenvector and then calculating the average to obtain the first target eigenvector provided by an embodiment of the present application. Figure 7 As shown in the figure, after inputting the target text, the features x1 to x1 are obtained by encoding with the N-Gram algorithm. n , the feature x1 to x n Represents the words or N-Gram features in the text. After the N-Gram embedding matrix, the vectors of each N-Gram feature are obtained, and then the first feature vector of the target text is obtained by averaging these vectors.

[0135] At the same time, entities e1 to e in the target text are extracted n , e1 to e n It means that n entities are extracted, and the vectors corresponding to each entity are obtained through an entity EntEmbedding matrix, and each entity vector is averaged to the second feature vector of the target text. It should be noted that if no entity is matched in the target text, the entity vector in the text is all 0. Then the first feature vector and the second feature vector are added and averaged to get the first target feature vector.

[0136] The first target feature vector obtained by fusion can be used to input a classifier to obtain a classification result of the target text. The classifier can be an untrained classifier or a classifier obtained by training an initialization classifier.

[0137] If the classifier is a trained classifier, the classifier can be trained in the following way:

[0138] A second text sample is obtained, and the second text sample is configured with a text category. It can be understood that the second text sample used to train the classifier can be the same sample as the first text sample, or can be a sample different from the first text sample.

[0139] The third feature of the second text sample can be obtained according to the word segmentation quantity and word frequency, and the second text sample is matched with the pre-constructed second target double array to obtain the fourth feature of the second text sample. When the second text sample is the same as the first text sample, the second target double array can be the same as the first target double array constructed using the first text sample. Conversely, when the second text sample is different from the first text sample, the second target double array can also be different from the first target double array constructed using the first text sample.

[0140] Next, the third feature and the fourth feature may be fused to obtain a second target feature vector. For example, a third feature vector may be obtained based on the third feature, a fourth feature vector may be obtained based on the fourth feature, and then the third feature vector and the fourth feature vector may be added together, or the third feature vector and the fourth feature vector may be concatenated to obtain the second target feature vector.

[0141] Finally, the second target feature vector can be input into the initialized classifier, and the trained classifier can be obtained according to the text category of the second text sample. During training, the output of the classifier with the second target feature vector as input can be first obtained, and then the output can be compared with the text category of the second text sample. When the difference between the classifier output and the text category of the second text sample is greater than a preset threshold, the classifier parameters can be adjusted, and then the second target feature vector can be input into the classifier with adjusted parameters to obtain the classifier output again, until the difference between the classifier output and the text category of the second text sample is less than or equal to the preset threshold. The classifier at this time is the trained classifier.

[0142] Figure 8 is a schematic diagram of an implementation method of concatenating a first feature vector and a second feature vector to obtain a first target feature vector provided by an embodiment of the present application. Figure 8 The method for obtaining the first eigenvector and the second eigenvector in the embodiment shown is similar to that in Figure 7 The embodiments shown are basically the same and will not be described in detail here. Figure 8 As shown, after the first eigenvector and the second eigenvector are obtained, the first eigenvector and the second eigenvector can be directly concatenated to obtain the first target eigenvector.

[0143] Fig. 9 FIG. 1 is a flow chart of another text classification method provided in an embodiment of the present application. Fig. 9 As shown in the figure, when performing text classification, you can first obtain the text_ids, text_lens, ent_ids, and ent_lens of the target text. Among them, text_ids represents the encoding identification (identification, id) of the text N-Gram, text_lens represents the length of the N-Grams id list, ent_ids represents the entity-entity type id in the matching (empty if no match is successful), and ent_lens represents the length of the entity list in the matching.

[0144] Text_ids can obtain the feature vector corresponding to each N-Gram id through table lookup, for example, 256 dimensions. Similarly, ent_ids can obtain the feature vector corresponding to each entity id through the entity embedding table, for example, also 256 dimensions.

[0145] According to the length of text_lens and ent_lens, we add and average them to get N-Gram features and entity-entity type features. Then we add and average the obtained N-Gram features and entity-entity type features to get the whole text feature representation. Finally, we map it to the corresponding classification quantity dimension through a fully connected layer.

[0146] The technical solution provided in the embodiment of the present application can enrich the semantic information of the text by fusing entity and entity type features, thereby improving the accuracy of classification. In particular, when dealing with multi-classification problems, it can effectively avoid the problems of low precision and easy misrecall in multi-classification in traditional methods.

[0147] Furthermore, the technical solution provided in the embodiment of the present application has a shorter training time than the method based on deep learning, and the training process does not require a large amount of computing resources, which is more suitable for application scenarios with limited computing resources. At the same time, this method has good scalability and is not only applicable to fasttext, but also to other classification models, such as BERT.

[0148] All the above optional technical solutions can be arbitrarily combined to form optional embodiments of the present application, which will not be described one by one here.

[0149] The following is an embodiment of the device of the present application, which can be used to execute the embodiment of the method of the present application. For details not disclosed in the embodiment of the device of the present application, please refer to the embodiment of the method of the present application.

[0150] Fig.10 Schematic diagram of a text classification device provided in an embodiment of the present application. Fig.10 As shown, the device comprises:

[0151] The acquisition module 1001 is configured to acquire the target text.

[0152] The feature extraction module 1002 is configured to obtain a first feature of the target text according to the word segmentation quantity and word frequency.

[0153] The feature extraction module 1002 is further configured to match the target text with a pre-constructed first target double array to obtain a second feature of the target text, where the second feature includes an entity feature and an entity type feature of the target text.

[0154] The fusion module 1003 is configured to perform feature fusion based on the first feature and the second feature to obtain a first target feature vector.

[0155] The classification module 1004 is configured to use a classifier to classify the first target feature vector to obtain a classification result of the target text, where the classification result includes a category of the target text.

[0156] According to the technical solution provided in the embodiment of the present application, by extracting the first feature of the target text according to the word segmentation quantity and the word frequency, and obtaining the second feature including the target text entity feature and the entity type feature by matching with the pre-constructed first target dual array, and then fusing the first feature and the second feature, the category of the target text is obtained by using a classifier and the fused first target feature vector, and a higher precision classification result can be obtained.

[0157] In some embodiments of the present application, a first target dual array is constructed in the following manner: a first text sample is obtained; M entity libraries are constructed based on the first text sample, each entity library is configured with an entity type, and each entity library includes at least one entity in the first text sample, where M is a positive integer; a query Trie tree is constructed based on the M entity libraries; a first target dual array is obtained based on the Trie tree construction, the first target dual array includes at least a basic array, and each array element of the basic array includes an entity feature, or includes an entity feature and an entity type feature.

[0158] In some embodiments of the present application, the first target double array also includes a check array; constructing a query Trie tree based on the M entity libraries, and constructing a first target double array based on the Trie tree, including: obtaining all entities in the M entity libraries; creating a root node of the Trie tree, and initializing the basic array value and the check array value corresponding to the root node; in the Trie tree, inserting child nodes of each level in sequence starting from the root node to obtain a constructed Trie tree, in which each child node corresponds to an entity; calculating the basic array value and the check array value corresponding to each child node according to the encoding value of each entity, and obtaining the first target double array.

[0159] In some embodiments of the present application, the root node of the Trie tree corresponds to the base array of the first target double array and the first array element in the check array; the base array value and the check array value corresponding to the root node are initialized, including: setting the entity feature value of the first array element in the base array to 1, and setting the entity type feature value to empty; setting the value of the first array element in the check array to 0.

[0160] In some embodiments of the present application, the basic array value and the check array value corresponding to each child node are calculated according to the coding value of each entity, including: obtaining the current node and all the child nodes of the current node; determining a preset value according to the entity coding values ​​of all the child nodes of the current node, and the preset value satisfies the following conditions: in the check array, the check array element value whose serial number is the sum of the preset value and the entity coding value of any child node of the current node is 0, and the serial numbers of each array element in the basic array and the check array are sequentially numbered starting from 0; determining the basic array value of the current node according to the preset value, and determining the basic array entity feature value of the parent node of the current node as the check array value of the current node.

[0161] In some embodiments of the present application, in response to determining that the current node is a node corresponding to a non-end entity in the target entity library, the base array entity feature value of the current node is determined to be a preset value, and the entity type feature value is empty; in response to determining that the current node is a node corresponding to an end entity in the target entity library, the base array entity feature value of the current node is determined to be a negative number of the preset value, and the entity type feature value of the base array of the current node is the entity type of the target entity library.

[0162] In some embodiments of the present application, feature fusion is performed based on the first feature and the second feature to obtain a first target feature vector, including: obtaining a first feature vector based on the first feature, and obtaining a second feature vector based on the second feature; adding the first feature vector and the second feature vector and calculating the average to obtain the first target feature vector; or concatenating the first feature vector and the second feature vector to obtain the first target feature vector.

[0163] In some embodiments of the present application, the classifier is an initialized classifier, or a classifier obtained by training the initialized classifier; wherein the trained classifier is trained in the following manner: a second text sample is obtained, and the second text sample is configured with a text category; a third feature of the second text sample is obtained based on the word segmentation quantity and the word frequency; the second text sample is matched with a pre-constructed second target double array to obtain a fourth feature of the second text sample; the third feature and the fourth feature are feature-fused to obtain a second target feature vector; the second target feature vector is input into the initialized classifier, and the trained classifier is obtained based on the text category of the second text sample.

[0164] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0165] Fig.11 Schematic diagram of an electronic device provided in an embodiment of the present application. Fig.11 As shown, the electronic device 11 of this embodiment includes: a processor 1101, a memory 1102, and a computer program 1103 stored in the memory 1102 and executable on the processor 1101. When the processor 1101 executes the computer program 1103, the steps in the above-mentioned method embodiments are implemented. Alternatively, when the processor 1101 executes the computer program 1103, the functions of the modules / units in the above-mentioned device embodiments are implemented.

[0166] The electronic device 11 may be a desktop computer, a notebook, a PDA, a cloud server, or other electronic device. The electronic device 11 may include, but is not limited to, a processor 1101 and a memory 1102. Those skilled in the art will appreciate that Fig.11 The electronic device 11 is merely an example and does not limit the electronic device 11 , and may include more or less components than those shown in the figure, or different components.

[0167] The processor 1101 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0168] The memory 1102 may be an internal storage unit of the electronic device 11, for example, a hard disk or memory of the electronic device 11. The memory 1102 may also be an external storage device of the electronic device 11, for example, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card (FlashCard), etc. equipped on the electronic device 11. The memory 1102 may also include both an internal storage unit of the electronic device 11 and an external storage device. The memory 1102 is used to store computer programs and other programs and data required by the electronic device.

[0169] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units.

[0170] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.< / int>

Claims

1. A text classification method, It is characterized in that include: Get the target text; According to the word segmentation quantity and word frequency, a first feature of the target text is obtained; Matching the target text with a pre-constructed first target double array to obtain a second feature of the target text, wherein the second feature includes an entity feature of a non-ending entity in the target text, and an entity feature and an entity type feature of an ending entity in the target text, wherein the ending entity is the last entity of one or more words or sentences in the target text, or is the last entity in a group of entities arranged in sequence in the target text; Perform feature fusion based on the first feature and the second feature to obtain a first target feature vector; The first target feature vector is classified using a classifier to obtain a classification result of the target text, wherein the classification result includes a category of the target text.

2. The method according to claim 1, It is characterized in that The first target double array is constructed in the following manner: Obtaining a first text sample; Construct M entity libraries according to the first text sample, each entity library is configured with an entity type, and each entity library includes at least one entity in the first text sample, and M is a positive integer; Construct a query Trie tree according to the M entity libraries; The first target double array is constructed according to the Trie tree, and the first target double array includes at least a basic array, and each array element of the basic array includes an entity feature, or includes an entity feature and an entity type feature.

3. The method according to claim 2, It is characterized in that The first target dual array also includes a check array; Constructing a query Trie tree according to the M entity libraries, and obtaining the first target double array according to the Trie tree, including: Obtain all entities in the M entity libraries; Create a root node of the Trie tree, and initialize the basic array value and check array value corresponding to the root node; In the Trie tree, starting from the root node, child nodes of each level are inserted in sequence to obtain a constructed Trie tree, in which each child node corresponds to an entity; The basic array value and the check array value corresponding to each child node are calculated according to the encoding value of each entity to obtain the first target double array.

4. The method according to claim 3, It is characterized in that The root node of the Trie tree corresponds to the first array element in the base array and the check array of the first target double array; The initializing the basic array value and checking the array value corresponding to the root node includes: The entity feature value of the first array element in the basic array is set to 1, and the entity type feature value is set to null; The value of the first array element in the check array is set to 0.

5. The method according to claim 3, It is characterized in that The step of calculating the basic array value and the check array value corresponding to each child node according to the encoding value of each entity includes: Get the current node and all its child nodes; Determine a preset value according to the entity code values ​​of all child nodes of the current node, and the preset value satisfies the following conditions: in the check array, the check array element value whose serial number is the sum of the preset value and the entity code value of any child node of the current node is 0, and the serial numbers of each array element in the basic array and the check array are sequentially numbered starting from 0; The basic array value of the current node is determined according to the preset value, and the basic array entity feature value of the parent node of the current node is determined as the check array value of the current node.

6. The method according to claim 5, It is characterized in that In response to determining that the current node is a node corresponding to a non-end entity in the target entity library, determining that a basic array entity feature value of the current node is the preset value, and an entity type feature value is empty; In response to determining that the current node is a node corresponding to an end entity in a target entity library, determining that a base array entity feature value of the current node is a negative number of the preset value, and an entity type feature value of the base array of the current node is an entity type of the target entity library.

7. The method according to claim 1, It is characterized in that The step of performing feature fusion based on the first feature and the second feature to obtain a first target feature vector includes: Obtain a first feature vector according to the first feature, and obtain the second feature vector according to the second feature; Add the first eigenvector and the second eigenvector and calculate the average to obtain the first target eigenvector; or The first feature vector and the second feature vector are concatenated to obtain the first target feature vector.

8. The method according to claim 1, It is characterized in that The classifier is an initialized classifier, or a classifier obtained by training the initialized classifier; The trained classifier is trained in the following way: Acquire a second text sample, where the second text sample is configured with a text category; Obtaining a third feature of the second text sample according to the word segmentation quantity and word frequency; Matching the second text sample with a pre-constructed second target double array to obtain a fourth feature of the second text sample; Performing feature fusion on the third feature and the fourth feature to obtain a second target feature vector; The second target feature vector is input into an initialized classifier, and the trained classifier is obtained according to the text category of the second text sample.

9. A text classification device, It is characterized in that include: An acquisition module, configured to acquire a target text; A feature extraction module is configured to obtain a first feature of the target text according to the word segmentation quantity and word frequency; The feature extraction module is further configured to match the target text with a pre-constructed first target double array to obtain a second feature of the target text, wherein the second feature includes an entity feature of a non-ending entity in the target text, and an entity feature and an entity type feature of an ending entity in the target text, wherein the ending entity is the last entity of one or more words or sentences in the target text, or is the last entity in a group of entities arranged in sequence in the target text; a fusion module, configured to perform feature fusion based on the first feature and the second feature to obtain a first target feature vector; The classification module is configured to use a classifier to classify the first target feature vector to obtain a classification result of the target text, wherein the classification result includes a category of the target text.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, It is characterized in that When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

Citation Information

Patent Citations

  • Chinese news classification method based on Transformer

    CN111666373A

  • Named entity identification method and device, computer equipment and storage medium

    CN112800769A