Text classification method and device, electronic equipment and storage medium

Abstract

The application discloses a text classification method and device, electronic equipment and a storage medium. The method comprises the following steps: obtaining a first text; preprocessing the first text to obtain a word vector included in the first text and at least one word vector corresponding to the word vector; performing weighted fusion processing on each word vector in the word vector and the at least one word vector based on a lattice-LSTM model to obtain a word lattice vector of each position corresponding to each word vector; performing semantic aggregation on the word lattice vector based on a capsule network model to obtain a first semantic vector of each position; determining a category label corresponding to the first semantic vector of each position, and taking the category label as the classification of the first text.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a text classification method, apparatus, electronic device, and storage medium. Background Technology

[0002] Text classification, a fundamental task in natural language processing, involves assigning text to specified categories within a given classification system. The basic idea is to extract features from the text and then select the optimal match to classify it. Text classification primarily involves the field of natural language processing, using machine learning or deep learning methods to learn text features for classification. However, these techniques suffer from inaccurate classification. Summary of the Invention

[0003] To address the related technical issues, embodiments of this application provide a text classification method, apparatus, electronic device, and storage medium.

[0004] The technical solution of this application embodiment is implemented as follows:

[0005] This application provides a text classification method, including:

[0006] Get the first text;

[0007] The first text is preprocessed to obtain character vectors included in the first text and at least one word vector corresponding to the character vectors;

[0008] The word vector and each word vector in the at least one word vector are weighted and fused based on the Lattice-LSTM model to obtain the word vector at each position corresponding to each word vector;

[0009] Based on the capsule network model, semantic aggregation is performed on the word lattice vectors to obtain the first semantic vector for each position;

[0010] Determine the category label corresponding to the first semantic vector at each position, and use the category label as the classification of the first text.

[0011] In the above scheme, the weighted fusion processing of each word vector among the character vectors and the at least one word vector based on the Lattice-LSTM model to obtain the word lattice vector for each position corresponding to each word vector includes:

[0012] The Lattice-LSTM model is used to encode the character vector and each word vector to obtain the first encoding information of the character vector and the second encoding information of each word vector.

[0013] A first weight coefficient for the first encoded information and a second weight coefficient for the second encoded information are determined; the first weight coefficient represents the proportion of the character vector in the at least one word vector; the second weight coefficient represents the proportion of each word vector in the at least one word vector.

[0014] The word lattice vector is determined based on the first weight coefficient, the second weight coefficient, the first encoding information, and the second encoding information.

[0015] In the above scheme, the step of encoding the character vector and each word vector based on the Lattice-LSTM model to obtain the first encoding information of the character vector and the second encoding information of each word vector includes:

[0016] The word vector is encoded based on the first word lattice structure in the Lattice-LSTM model to obtain the first encoded information; the first word lattice structure includes a first input gate, a first forget gate and a first output gate.

[0017] Each word vector is encoded based on the second lattice structure in the Lattice-LSTM model to obtain the second encoded information; the second lattice structure includes a second input gate and a second forget gate.

[0018] In the above scheme, determining the first weight coefficient of the first encoded information and the second weight coefficient of the second encoded information includes:

[0019] The word vector is input into the first input gate for transformation to obtain the first state information of the word vector;

[0020] Each word vector is input into the second input gate for transformation to obtain the second state information of each word vector;

[0021] The first weight coefficient and the second weight coefficient are determined based on the first state information and the second state information.

[0022] In the above scheme, the semantic aggregation of the word lattice vectors based on the capsule network model to obtain the first semantic vector for each position includes:

[0023] The first coupling coefficient corresponding to the word lattice vector is determined based on the capsule network model; the first coupling coefficient characterizes the degree to which the semantic features corresponding to each position are dynamically acquired in the capsule network model.

[0024] The first influence weight of the semantic features corresponding to each position on the first text is determined based on the first coupling coefficient.

[0025] The semantic features corresponding to each position are aggregated based on the first influence weight to obtain the first semantic vector.

[0026] In the above scheme, the step of aggregating the semantic features corresponding to each position based on the first influence weight to obtain the first semantic vector includes:

[0027] Based on the first influence weight, the semantic features corresponding to each position are aggregated to obtain the second semantic vector;

[0028] The second semantic vector is compressed to obtain the first semantic vector.

[0029] In the above scheme, the step of compressing the second semantic vector to obtain the first semantic vector includes:

[0030] Determine whether the modulus of the second semantic vector satisfies a preset condition;

[0031] If the modulus of the second semantic vector satisfies the preset condition, the second semantic vector is compressed using the first compression function to obtain the first semantic vector;

[0032] If the modulus of the second semantic vector does not meet the preset condition, the second semantic vector is compressed using a second compression function to obtain the first semantic vector. The compression coefficient of the first compression function is different from that of the second compression function, so that when the modulus of the second semantic vector approaches zero, the compressed second semantic vector will not lose some of the semantic vector information.

[0033] The method in the above scheme further includes:

[0034] The second coupling coefficient corresponding to the first semantic vector is determined based on the capsule network model; the second coupling coefficient is greater than the first coupling coefficient.

[0035] The first coupling coefficient is updated based on the second coupling coefficient to obtain the updated first coupling coefficient;

[0036] The first influence weight is re-determined based on the updated coupling coefficient;

[0037] Based on the re-determined first influence weight, the semantic features are re-aggregated to obtain the first semantic vector again, until the re-obtained first semantic vector converges, at which point the determination of the second coupling coefficient corresponding to the re-obtained first semantic vector based on the capsule network model is stopped.

[0038] In the above scheme, the preprocessing of the first text to obtain the character vectors included in the first text and at least one word vector corresponding to the character vectors includes:

[0039] Obtain the first character in the first text and at least one word that ends with the first character;

[0040] The character vector corresponding to the first character is matched according to the preset character vector dictionary;

[0041] The at least one word vector corresponding to the at least one word is matched according to the preset word vector dictionary.

[0042] This application also provides a text classification device, including:

[0043] The acquisition unit is used to acquire the first text;

[0044] A preprocessing unit is used to preprocess the first text to obtain character vectors included in the first text and at least one word vector corresponding to the character vectors;

[0045] The weighted fusion processing unit is used to perform weighted fusion processing on each word vector among the character vector and the at least one word vector based on the Lattice-LSTM model to obtain the word lattice vector corresponding to each position of each word vector;

[0046] A semantic aggregation unit is used to perform semantic aggregation on the word lattice vectors based on a capsule network model to obtain a first semantic vector for each position.

[0047] A determining unit is used to determine the category label corresponding to the first semantic vector of each position, and to use the category label as the classification of the first text.

[0048] This application also provides an electronic device, including:

[0049] Memory, used to store executable instructions;

[0050] A processor, when executing executable instructions stored in the memory, implements any step of the method described above.

[0051] This application also provides a computer-readable storage medium storing executable instructions for implementing any step of the method described above when executed by a processor.

[0052] The text classification method, apparatus, electronic device, and storage medium provided in this application include: acquiring a first text; preprocessing the first text to obtain character vectors and at least one word vector corresponding to each character vector; performing weighted fusion processing on each word vector of the character vectors and the at least one word vector based on a Lattice-LSTM model to obtain a word lattice vector at each position corresponding to each word vector; performing semantic aggregation on the word lattice vectors based on a capsule network model to obtain a first semantic vector at each position; determining a category label corresponding to the first semantic vector at each position, and using the category label as the classification of the first text. The scheme in this application embodiment uses a Lattice-LSTM model to perform weighted fusion processing on each word vector among the character vectors and at least one word vector, dynamically fusing the character information and word information of the text. It fully considers the different weights of character information and a character in different words, and performs weighted fusion of characters and multiple words, thereby enhancing the vocabulary of the characters in the text, better carrying rich word information, realizing the full utilization of the semantics of the text and richer semantic information learning; by performing semantic aggregation on the word lattice vectors based on a capsule network model, it fully considers the spatial information of the text, and performs semantic aggregation on the text features at each position to form a better text representation, so as to achieve more accurate text classification. Attached Figure Description

[0053] Figure 1 This is a schematic diagram illustrating a text classification method flow provided in an embodiment of this application;

[0054] Figure 2 This is a schematic diagram of the first word lattice structure in the Lattice-LSTM model of the text classification method in the embodiments of this application;

[0055] Figure 3 This is a schematic diagram of the second word lattice structure in the Lattice-LSTM model of the text classification method in the embodiments of this application;

[0056] Figure 4 This is a schematic diagram showing the compression coefficient curves of the improved compression function and the unimproved compression function, assuming that the modulus of the semantic vector in the embodiments of this application meets the preset conditions.

[0057] Figure 5 This is a schematic diagram showing the compression coefficient curves of the entire improved compression function and the entire unimproved compression function in the embodiments of this application.

[0058] Figure 6 This is a schematic diagram of the text classification model based on lexical enhancement-semantic aggregation in this application;

[0059] Figure 7 This is a schematic diagram of a text classification device according to an embodiment of this application;

[0060] Figure 8 This is a schematic diagram of the hardware entity structure of an electronic device in an embodiment of this application. Detailed Implementation

[0061] The present application will now be described in further detail with reference to the accompanying drawings and embodiments.

[0062] In related technologies, traditional feature extraction-based text classification methods require manual extraction of text features. The extracted text features are mainly at the character and word level, which cannot well represent the deep semantics contained in the text. At the same time, manual representation is time-consuming and labor-intensive, and it is a feature template extracted for a specific dataset. When the dataset or the text to be classified changes, it cannot be reused and lacks transferability.

[0063] On the other hand, deep learning text classification methods in related technologies mainly suffer from two problems. One problem is that the text encoding process is mostly based on characters or words. These two methods have the potential problems of weak semantic extraction capabilities and word segmentation errors leading to encoding errors, which can cause error propagation and affect model performance when the model learns semantics. The other problem is that current common deep learning methods, such as CNNs, use low-level features to construct high-order features to represent higher-level meanings. However, CNN models reuse many feature detectors and cannot effectively model spatial information while maintaining feature representation.

[0064] Based on this, embodiments of this application provide a text classification method applied to an electronic device. The function implemented by this method can be achieved by a processor in the electronic device calling program code. Of course, the program code can be stored in a computer storage medium. Therefore, the electronic device includes at least a processor and a storage medium. As an example, the electronic device can be a mobile phone, computer, terminal, information transceiver, tablet device, personal digital assistant, etc.

[0065] Figure 1 This application provides a schematic diagram of a text classification method flow; as shown in the embodiments. Figure 1 As shown, the method includes:

[0066] Step 101: Obtain the first text;

[0067] Step 102: Preprocess the first text to obtain the character vectors included in the first text and at least one word vector corresponding to the character vectors;

[0068] Step 103: Based on the Lattice-LSTM model, perform weighted fusion processing on each word vector among the character vectors and the at least one word vector to obtain the word lattice vector corresponding to each position of each word vector;

[0069] Step 104: Perform semantic aggregation on the word lattice vectors based on the capsule network model to obtain the first semantic vector for each position;

[0070] Step 105: Determine the category label corresponding to the first semantic vector of each position, and use the category label as the classification of the first text.

[0071] In practical applications, the specific content of the first text in step 101 can be determined according to the actual situation, and is not limited here. The first text can be denoted as c = {c1, c2, ..., c...} N}, where N is the character granularity length of the text.

[0072] In step 102, the preprocessing can be determined according to the actual situation, and is not limited here. As an example, the preprocessing can be a transformation process, specifically a vector transformation process.

[0073] In one embodiment, the preprocessing of the first text to obtain character vectors included in the first text and at least one word vector corresponding to the character vectors includes:

[0074] Obtain the first character in the first text and at least one word that ends with the first character;

[0075] The character vector corresponding to the first character is matched according to the preset character vector dictionary;

[0076] The at least one word vector corresponding to the at least one word is matched according to the preset word vector dictionary.

[0077] The first character can be any character or symbol in the first text; the preset character vector dictionary and the preset word vector dictionary can be determined according to the actual situation, and are not limited here. As an example, the preset character vector dictionary can be composed of Chinese characters and other commonly used symbols, and is pre-trained by the model; the preset word vector dictionary can be composed of preset words, and is pre-trained by the model; the preset words have been verified and are all correct words, excluding words caused by word segmentation errors.

[0078] Obtaining the first character and at least one word ending with the first character in the first text can be understood as representing the first text as a character and a word.

[0079] Matching the character vector corresponding to the first character according to the preset character vector dictionary can be understood as converting the character representation into a character vector; the character vector can also be called character-granular representation or character-granular representation, and can be denoted as N×V; where N is the character-granularity length and V is the dimension of the character vector.

[0080] Matching at least one word vector corresponding to at least one word according to a preset word vector dictionary can be understood as converting word representation into word vector; the word vector can also be called word granular representation, which can be denoted as M×V; where M is the word granularity length and V is the dimension of the word vector.

[0081] In practical applications, the natural language information in the first text is converted into a vector representation. First, each character is represented as a character vector through random initialization. Then, the text words appearing in the dictionary are represented as word vectors, and the word vector e c It is a distributed vector obtained from a dictionary composed of Chinese characters and other commonly used symbols, the word vector e w The text representation is based on a pre-trained dictionary of word vectors, and can be represented by the following formula (1):

[0082]

[0083] In equation (1), c j It is the character input at time j, e c Embed the character vectors at the character level into a dictionary. c represents the vector information obtained from the current transformation. b,e It is a word composed of characters input between time b and e, where e is the input character. w Embed the word vectors into a dictionary. This refers to the word vector information obtained at the current moment. This process can be understood as obtaining the text representation based on the character vector dictionary and the word vector dictionary. In other words, it involves extracting the granularity from the original text features.

[0084] This embodiment addresses the shortcomings of traditional encoding models that rely solely on character (e.g., missing word information) or word encoding (e.g., word segmentation error propagation), which hinders the semantic representation of text during the encoding stage. It proposes a method to construct a large dictionary, avoiding the problems of traditional encoding models that rely solely on character (e.g., missing word information) or word encoding (e.g., word segmentation error propagation).

[0085] Since text features are mainly expressed at the character and word level and cannot well represent the deep semantics contained in the text, this application adopts a Lattice LSTM model to encode and enhance the lexical information of the text, so as to make full use of the semantics of natural text and learn richer semantic information.

[0086] In step 103, the word lattice vector can be denoted as a Lattice vector; the weighted fusion process can be understood as enabling the dynamic fusion of character information and word information in the text, fully considering the different weights of character information and a character in different words, considering the weighted fusion of characters and multiple words, and better carrying rich word information.

[0087] In one embodiment, the weighted fusion processing of each word vector among the character vectors and the at least one word vector based on the Lattice-LSTM model to obtain the word lattice vector for each position corresponding to each word vector includes:

[0088] The Lattice-LSTM model is used to encode the character vector and each word vector to obtain the first encoding information of the character vector and the second encoding information of each word vector.

[0089] A first weight coefficient for the first encoded information and a second weight coefficient for the second encoded information are determined; the first weight coefficient represents the proportion of the character vector in the at least one word vector; the second weight coefficient represents the proportion of each word vector in the at least one word vector.

[0090] The word lattice vector is determined based on the first weight coefficient, the second weight coefficient, the first encoding information, and the second encoding information.

[0091] Wherein, the first encoding information is the encoding information of the word vector, which can be denoted as: This can be understood as the temporary unit state at position j; the second encoding information is the encoding information of each word vector, which can be denoted as... This can be understood as the unit state of words that start with b and end with j; the first weight coefficient can be denoted as... The second weighting coefficient can be denoted as The word criterion vector can be denoted as: The word lattice vector can be understood as the hidden state at time j obtained by using the hidden state in LSTM.

[0092] In practical applications, the process of determining the word case vector can refer to the following formulas (2) and (3):

[0093]

[0094]

[0095] In equations (2) and (3), This is the first weighting coefficient; This is the first encoded information; This is the second weighting coefficient; This is the second encoded information; Let represent the output gate at time j, and tanh be the hyperbolic tangent function.

[0096] This embodiment uses a new encoding method that enables the dynamic fusion of character and word information in text. It fully considers the different weights of character information and a character in different words, and considers the weighted fusion of characters and multiple words to better carry rich word information.

[0097] In one embodiment, the step of encoding the character vectors and each word vector based on the Lattice-LSTM model to obtain first encoding information for the character vectors and second encoding information for each word vector includes:

[0098] The word vector is encoded based on the first word lattice structure in the Lattice-LSTM model to obtain the first encoded information; the first word lattice structure includes a first input gate, a first forget gate and a first output gate.

[0099] Each word vector is encoded based on the second lattice structure in the Lattice-LSTM model to obtain the second encoded information; the second lattice structure includes a second input gate and a second forget gate.

[0100] In this embodiment, for ease of understanding, a schematic diagram of the first word structure can be referred to. Figure 2 To understand, Figure 2 This is a schematic diagram of the first word lattice structure in the Lattice-LSTM model of the text classification method in the embodiments of this application; 201 represents the first input gate in the first word lattice structure; 202 represents the first forget gate in the first word lattice structure; 203 represents the first output gate in the first word lattice structure.

[0101] The diagram of the second grammatical structure can be found by referring to Figure 3 To understand, Figure 3 This is a schematic diagram of the second word lattice structure in the Lattice-LSTM model of the text classification method in the embodiments of this application; 301 represents the second input gate in the second word lattice structure; 302 represents the second forget gate in the second word lattice structure.

[0102] In practical applications, the determination process of the first and second encoded information requires the use of an LSTM structure; among them, the character vector-based structure is the original structure of LSTM, and the word vector-based LSTM is based on the original structure with the output gate subtracted. That is to say, the character-based LSTM propagation process in the word lattice structure is the same as the original LSTM, controlled by three gate units: input gate, output gate, and forget gate, while the word granularity calculation is compared with it without the output gate. The hidden state is the hidden state of the character position at the beginning of the word, and the text is a word fragment from the beginning position b to the end position w. The input gate, forget gate and the temporary state of the word are obtained by calculation, and the unit state of the word is obtained by dot product summation operation. The calculation process is as follows: formula (4):

[0103]

[0104] In equation (4), This represents the input and output gates for words that begin at position b and end at position w. This is the temporary unit state for words that begin at position b and end at position w. The calculation method is the same as the original LSTM. The cell state of the character at position b can be calculated using a character-based LSTM structure.

[0105] In one embodiment, determining the first weight coefficient of the first encoded information and the second weight coefficient of the second encoded information includes:

[0106] The word vector is input into the first input gate for transformation to obtain the first state information of the word vector;

[0107] Each word vector is input into the second input gate for transformation to obtain the second state information of each word vector;

[0108] The first weight coefficient and the second weight coefficient are determined based on the first state information and the second state information.

[0109] The first state information can be understood as the state information of the word vector, and can be denoted as: The second state information can be understood as the state information of each word vector, and can be denoted as:

[0110] In practical applications, in order to integrate the character information and word information at the current moment, the word vector of each word matched in the dictionary for the text segment ending at the current moment is obtained by performing a linear transformation and nonlinear calculation to obtain the input gate. The calculation process is as follows: Formula (5):

[0111]

[0112] In equation (5), b l For linearly varying parameters, Let σ represent the unit state of the word that starts at position b and ends at position w, and let σ be the sigmoid function.

[0113] The input gate is obtained using the character LSTM encoding at the current time step. Input gate for all words ending with the current time j Perform normalization to calculate the corresponding weights The weighted normalized representation of the two input gates is calculated using the following formula (6):

[0114]

[0115] In equation (6), This is the first state information; The state information for each word vector; exp represents the input gate. This represents the state information of the word vector after it passes through the first input gate. This represents the state information of the word vector after it passes through the second input gate.

[0116] This embodiment addresses the shortcomings of traditional encoding models that only use character encoding (e.g., missing word information) or word encoding (e.g., word segmentation error propagation), which leads to deficiencies in the semantic representation stage of text encoding. It proposes a method to construct a large dictionary and use a Lattice LSTM model to encode and enhance the lexical information of text. This key point enables full utilization of the semantics of natural text and richer semantic information learning.

[0117] In neural network methods, the concentration of spatial patterns at lower levels helps represent higher-level concepts. For example, CNNs construct convolutional feature detectors to extract local patterns from windows of vector sequences and use max pooling to select the most salient features. They then hierarchically construct and aggregate these patterns across multiple levels. As a spatially sensitive model, CNNs pay the price of inefficiency in replicating feature detectors across a grid, having to choose between replicating detectors whose size grows exponentially with dimensionality and increasing the volume of the labeled training set in a similarly exponential manner. On the other hand, spatially insensitive methods are effective in judgment scenarios regardless of the order of any words or local patterns; however, they are inevitably limited by the rich structure presented in the feature sequences, affecting the final results. Therefore, improving the efficiency of the spatial order of the encoded sequences while maintaining flexibility in representativeness is a major challenge.

[0118] This application then proposes a semantic aggregation model based on capsule networks to capture text semantic features. Compared with traditional CNN models, this model can better extract positional and global features from the text, resulting in more spatialized semantic features, and fully considers the semantic features of the text and the relationships in the spatial mapping. The text is encoded using word lattice to obtain the hidden state vector h = {h1, h2, ..., h} corresponding to each position. N Then, through dynamic route learning aggregation, the degree of influence of each position on the whole text is obtained, and finally the aggregated semantic representation of the text is obtained.

[0119] In one embodiment, the semantic aggregation of the word lattice vectors based on the capsule network model to obtain a first semantic vector for each position includes:

[0120] The first coupling coefficient corresponding to the word lattice vector is determined based on the capsule network model; the first coupling coefficient characterizes the degree to which the semantic features corresponding to each position are dynamically acquired in the capsule network model.

[0121] The first influence weight of the semantic features corresponding to each position on the first text is determined based on the first coupling coefficient.

[0122] The semantic features corresponding to each position are aggregated based on the first influence weight to obtain the first semantic vector.

[0123] The first coupling coefficient can be determined by the initial coupling coefficient; the initial coupling coefficient can be denoted as b. ij The first coupling coefficient can be denoted as b. i The semantic features corresponding to each position can be denoted as: The first influence weight of the first text can be denoted as e. i The first semantic vector can be denoted as s. j '.

[0124] In one embodiment, the step of aggregating the semantic features corresponding to each position based on the first influence weight to obtain the first semantic vector includes:

[0125] Based on the first influence weight, the semantic features corresponding to each position are aggregated to obtain the second semantic vector;

[0126] The second semantic vector is compressed to obtain the first semantic vector.

[0127] Wherein, the first influence weight can be denoted as e i The second semantic vector can be denoted as s. j The first semantic vector can be denoted as t. j .

[0128] In one embodiment, the step of compressing the second semantic vector to obtain the first semantic vector includes:

[0129] Determine whether the modulus of the second semantic vector satisfies a preset condition;

[0130] If the modulus of the second semantic vector satisfies the preset condition, the second semantic vector is compressed using the first compression function to obtain the first semantic vector;

[0131] If the modulus of the second semantic vector does not meet the preset condition, the second semantic vector is compressed using a second compression function to obtain the first semantic vector. The compression coefficient of the first compression function is different from that of the second compression function, so that when the modulus of the second semantic vector approaches zero, the compressed second semantic vector will not lose some of the semantic vector information.

[0132] The preset conditions can be determined according to the actual situation and are not limited here. As an example, the preset conditions can be |s j || < 1. The first compression function can be The second compression function can be

[0133] In this embodiment, the compression function in the original capsule network is mainly considered. The compression function in the formula mainly consists of two terms. The first term is the compression coefficient, which is the compression factor when s j When s is very large, the compression factor is approximately equal to 1. j When it is very small, the coefficient approaches 0. The second term is s. j A normalized vector of length 1 is made so that the vector s is made so by the squash function. j The length is between 0 and 1. By observing the compression coefficient, it can be found that when ||s j When || < 1, the compression coefficient is less than 1 / 2, and it increases with ||s j The decreasing || value gradually approaches 0, which makes the compressed vector too small when the vector length is small, easily resulting in the loss of information when passing it to the next capsule unit. An improved compression function is proposed to comprehensively consider the impact of each feature on the overall semantics, solving the problem of features being easily ignored when they are small. That is, the magnitude of the second semantic vector satisfies |s|. j When || < 1, the second semantic vector is applied... The first semantic vector is obtained by compression using an improved compression function in capsule networks; the second semantic vector is obtained when the modulus satisfies ||s||. jWhen ||≥1, the second semantic vector is applied... The first semantic vector is obtained by compressing the original compression function in the capsule network.

[0134] In one embodiment, the method further includes:

[0135] The second coupling coefficient corresponding to the first semantic vector is determined based on the capsule network model; the second coupling coefficient is greater than the first coupling coefficient.

[0136] The first coupling coefficient is updated based on the second coupling coefficient to obtain the updated first coupling coefficient;

[0137] The first influence weight is re-determined based on the updated coupling coefficient;

[0138] Based on the re-determined first influence weight, the semantic features are re-aggregated to obtain the first semantic vector again, until the re-obtained first semantic vector converges, at which point the determination of the second coupling coefficient corresponding to the re-obtained first semantic vector based on the capsule network model is stopped.

[0139] In this embodiment, the capsule aggregation in the capsule network is divided into the first L-1 layers and the Lth layer, where L is the total number of layers in the capsule network. An improved activation function is applied to calculate the compressed vector t. j This keeps the direction of the vector unchanged and normalizes its length. The normalization operation allows more important features to have a higher weight in the subsequent coupling coefficient update. Therefore, the coupling coefficient update can take into account the various features of the text to a greater extent and perform semantic aggregation more accurately. This can better improve the ability of semantic features to represent text in multidimensional space and improve the performance of text classification.

[0140] As an example, in practical applications, the semantic aggregation task of text aggregation is defined as follows:

[0141] {h j ∈R d} j＝1,2...,N →{t∈R d} (7)

[0142] The overall computation process is divided into the first L-1 layers and the Lth layer, where L is the total number of layers in the capsule network. The Lth layer contains 1 capsule, which represents the aggregated semantic representation of the text. The propagation process of the overall network is as follows: first, the coupling coefficient b is initialized for all capsules i in the lth layer and all capsules j in the l+1th layer. ij =0. The input latent vector h of layer l undergoes a vector transformation. After entering the capsule layer, it is first mapped to the same number of neurons in the higher capsule layers through a linear transformation of the shared weight matrix. The calculation formula is as follows:

[0143]

[0144] This approach encodes the semantic relationships between low-level and high-level features, and then uses a softmax operation on the coupling coefficients to obtain the influence weight e of the feature at position i on the entire text. i The weighted summation yields the current representation s of the capsule in the (l+1)th layer. j The calculation is as follows:

[0145] e i =softmax(b i (9)

[0146]

[0147] Then, a compression function is used to ensure that the text representation does not exceed 1, while maintaining the same direction as the original s. j Similarly, the compression function in the original capsule network is as follows:

[0148]

[0149] The compression function mainly consists of two terms. The first term in the formula is the compression coefficient, when s j When s is very large, the compression factor is approximately equal to 1. j When it is very small, the coefficient approaches 0. The second term is s. j A normalized vector of length 1 is made so that the vector s is made so by the squash function. j The length is between 0 and 1. By observing the compression coefficient, it can be found that when ||s j When || < 1, the compression coefficient is less than 1 / 2, and it increases with ||s j The decrease of || gradually approaches 0, which makes the compressed vector too small when the vector length is small, and this part of the vector information is easily lost when it is passed to the next layer of capsule unit.

[0150]

[0151] To facilitate understanding, a schematic diagram of the compression coefficient curve of the first compression function is shown below, such as... Figure 4 As shown, Figure 4 This is a schematic diagram of the compression coefficient curves of the improved compression function and the unimproved compression function, assuming that the modulus of the semantic vector in this embodiment meets the preset conditions; 401 represents the compression coefficient curve of the improved compression function, which can be denoted as cut_squash; 402 represents the compression coefficient curve of the unimproved compression function, that is, the compression coefficient curve of the original compression function, which can be denoted as the original squash. Figure 5This is a schematic diagram of the compression coefficient curves of the entire improved compression function and the entire unimproved compression function in an embodiment of this application; 501 represents the compression coefficient curve of the entire improved compression function; 502 represents the compression coefficient curve of the entire unimproved compression function, that is, the compression coefficient curve of the entire original compression function.

[0152] Therefore, this application improves the compression function as shown in the above equation, when ||s j When || < 1, the compression factor will be modified to . Figure 4 For function comparison, 401 represents the improved `cut_squash`, and 402 represents the original `squash`. It can be seen that the improvement slows down the approach to 0 for smaller vectors, thus ensuring the preservation of information for smaller vectors without affecting the overall ||s|. j The larger the value of ||, the higher its proportion in the next layer of capsule aggregation. The compressed vector t is calculated using an improved activation function. j This keeps the vector direction unchanged and normalizes its length. Normalization ensures that more important features have a higher weight in subsequent coupling coefficient updates. The coupling coefficient update is determined by the degree of aggregation between the features at each position and the text, and is calculated as follows:

[0153]

[0154] The text aggregation method proposed in this application has stronger feature extraction capabilities than traditional CNN models, mainly in capturing directional and structural information in the feature space. It obtains the aggregated semantic representation of the text through spatial information encoding and dynamic semantic relevance weights. Then, the softmax activation function is used to calculate the specific category label corresponding to the text, completing the text classification process.

[0155] This embodiment addresses the insufficient utilization of spatial location information in current deep learning models by proposing a text aggregation method based on an improved capsule network. This method fully considers the spatial information of the text and can better aggregate the feature representations of various positional vectors through dynamic weights, resulting in a better text representation. Simultaneously, the improved capsule network solves the problem of small feature vectors being easily ignored and losing semantic information, thus maximizing the consideration of various text features.

[0156] To better understand, this example illustrates a text classification method specifically based on lexical enhancement and semantic aggregation. Figure 6 This diagram illustrates the text classification model based on lexical enhancement and semantic aggregation proposed in this application. The model mainly consists of a lexical enhancement module and a semantic aggregation module. First, the text c is defined as {c1, c2, ..., c...}. N}, where N is the character granularity length of the text. The vocabulary enhancement module obtains the current character and word granularity representation vectors through dictionary vectors and lexical vectors respectively. Then, it encodes the character and word granularity information separately through Lattice-LSTM. Finally, the two types of information are weighted and fused to obtain the hidden state h = {h1, h2, ..., h} of the text. N}. Subsequently, this application proposes an improved text semantic aggregation algorithm based on the dynamic routing idea of ​​capsule networks in image algorithms. The specific process of the semantic aggregation module is as follows: by mapping the Lattice vectors at each position of the text from low level to high level of capsule units, the text vector containing text semantic information and spatial structure information is extracted. The number of capsules is set to 1 in the last layer of the capsule to obtain the semantic representation of the text. Then, the text classification is completed through an activation function.

[0157] This embodiment proposes a lexical enhancement-semantic aggregation model for text classification, addressing issues such as feature extraction units, semantic feature learning, and semantic space feature representation. This model can better enhance the ability of semantic features to represent text in a multi-dimensional space, thereby improving the performance of text classification.

[0158] Compared with existing technologies, the advantages of this application are mainly reflected in the following aspects:

[0159] First, the text classification method based on lexical enhancement and semantic aggregation proposed in this application extracts original text features at a higher granularity, represents positional text information according to weights, and uses an improved capsule method on the semantic aggregation of features at each position to dynamically obtain improved text feature representations, which is a text feature learning model with better performance.

[0160] Second: This application constructs a large dictionary in the vocabulary enhancement stage and uses a new encoding method that enables the dynamic fusion of character information and word information in the text. It fully considers the different weights of character information and a character in different words, and considers the weighted fusion of characters and multiple words to better carry rich word information.

[0161] Third: In the semantic aggregation module, this application fully considers the spatial features of the text in addition to the original features and the relationships between the features. At the same time, it proposes an improved compression function that can comprehensively consider the impact of each feature on the overall semantics, and solves the problem that features are easily ignored when they are small.

[0162] To implement the method of this application embodiment, this application embodiment also provides a text classification device 300, which is installed on an electronic device. Figure 7 This is a schematic diagram of a text classification device according to an embodiment of this application; as shown below. Figure 7 As shown, it includes:

[0163] Acquisition unit 701 is used to acquire the first text;

[0164] The preprocessing unit 702 is used to preprocess the first text to obtain character vectors included in the first text and at least one word vector corresponding to the character vectors;

[0165] The weighted fusion processing unit 703 is used to perform weighted fusion processing on each word vector among the character vector and the at least one word vector based on the Lattice-LSTM model to obtain the word lattice vector corresponding to each position of each word vector;

[0166] Semantic aggregation unit 704 is used to perform semantic aggregation on the word lattice vectors based on the capsule network model to obtain a first semantic vector for each position;

[0167] The determining unit 705 is used to determine the category label corresponding to the first semantic vector of each position, and use the category label as the classification of the first text.

[0168] Here, in one embodiment, the weighted fusion processing unit 703 includes an encoding module and a determination module; wherein,

[0169] The encoding module is used to encode the character vector and each word vector based on the Lattice-LSTM model to obtain the first encoding information of the character vector and the second encoding information of each word vector.

[0170] The determining module is used to determine a first weight coefficient of the first encoding information and a second weight coefficient of the second encoding information; the first weight coefficient represents the proportion of the character vector in the at least one word vector; the second weight coefficient represents the proportion of each word vector in the at least one word vector; and the word vector is determined based on the first weight coefficient, the second weight coefficient, the first encoding information, and the second encoding information.

[0171] Here, in one embodiment, the encoding module is further configured to encode the word vectors based on the first word lattice structure in the Lattice-LSTM model to obtain the first encoding information; the first word lattice structure includes a first input gate, a first forget gate, and a first output gate; and to encode each word vector based on the second word lattice structure in the Lattice-LSTM model to obtain the second encoding information; the second word lattice structure includes a second input gate and a second forget gate.

[0172] Here, in one embodiment, the determining module is further configured to input the character vector into the first input gate for transformation to obtain the first state information of the character vector; input each word vector into the second input gate for transformation to obtain the second state information of each word vector; and determine the first weight coefficient and the second weight coefficient based on the first state information and the second state information.

[0173] Here, in one embodiment, the semantic aggregation unit 704 is further configured to determine a first coupling coefficient corresponding to the word lattice vector based on the capsule network model; the first coupling coefficient characterizes the degree to which the semantic features corresponding to each position are dynamically acquired in the capsule network model; determine a first influence weight of the semantic features corresponding to each position on the first text based on the first coupling coefficient; and aggregate the semantic features corresponding to each position based on the first influence weight to obtain the first semantic vector.

[0174] Here, in one embodiment, the semantic aggregation unit 704 is further configured to aggregate the semantic features corresponding to each position based on the first influence weight to obtain a second semantic vector; and to compress the second semantic vector to obtain the first semantic vector.

[0175] Here, in one embodiment, the semantic aggregation unit 704 is further configured to determine whether the modulus of the second semantic vector satisfies a preset condition; if the modulus of the second semantic vector satisfies the preset condition, the second semantic vector is compressed using a first compression function to obtain the first semantic vector; if the modulus of the second semantic vector does not satisfy the preset condition, the second semantic vector is compressed using a second compression function to obtain the first semantic vector; the compression coefficient of the first compression function is different from the compression coefficient of the second compression function, so that when the modulus of the second semantic vector approaches zero, the compressed second semantic vector will not lose some information of the semantic vector.

[0176] Here, in one embodiment, the semantic aggregation unit 704 is further configured to determine a second coupling coefficient corresponding to the first semantic vector based on the capsule network model; the second coupling coefficient is greater than the first coupling coefficient; update the first coupling coefficient according to the second coupling coefficient to obtain an updated first coupling coefficient; redetermine the first influence weight according to the updated coupling coefficient; re-aggregate the semantic features based on the redetermined first influence weight to obtain the first semantic vector again, until the first semantic vector obtained again converges, and stop determining the second coupling coefficient corresponding to the first semantic vector obtained again based on the capsule network model.

[0177] Here, in one embodiment, the preprocessing unit 702 is further configured to obtain a first character in the first text and at least one word corresponding to the first character; match the character vector corresponding to the first character according to a preset character vector dictionary; and match the at least one word vector corresponding to the at least one word according to a preset word vector dictionary.

[0178] It should be noted that the text classification device provided in the above embodiments is only illustrated by the division of the above program modules when performing text classification. In actual applications, the above processing can be assigned to different program modules as needed, that is, the internal structure of the device can be divided into different program modules to complete all or part of the processing described above. In addition, the text classification device and the text classification method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0179] Based on the hardware implementation of the above program modules, this application embodiment also provides an electronic device, including a memory and a processor. The memory stores a computer program that can run on the processor. When the processor executes the program, it implements the steps in the text classification method provided in the above embodiments.

[0180] Correspondingly, embodiments of this application provide a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps in the text classification method provided in the above embodiments.

[0181] It should be noted that the descriptions of the storage medium and device embodiments above are similar to the descriptions of the method embodiments above, and have similar beneficial effects. For technical details not disclosed in the storage medium and device embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0182] It should be noted that, Figure 8 This is a schematic diagram of a hardware entity structure of an electronic device in an embodiment of this application, such as... Figure 8 As shown, the hardware entity of the electronic device 800 includes a processor 801 and a memory 803. Optionally, the electronic device 800 may also include a communication interface 802.

[0183] It is understood that memory 803 can be volatile memory or non-volatile memory, or both. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), ferromagnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disc, or compact disc read-only memory (CD-ROM); magnetic surface memory can be disk storage or magnetic tape storage. Volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Synchronous Static Random Access Memory (SSRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM), Enhanced Synchronous Dynamic Random Access Memory (ESDRAM), SyncLink Dynamic Random Access Memory (SLDRAM), and Direct Rambus Random Access Memory (DRRAM).The memory 803 described in the embodiments of this application is intended to include, but is not limited to, these and any other suitable types of memory.

[0184] The methods disclosed in the embodiments of this application can be applied to or implemented by processor 801. Processor 801 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuit of the hardware in processor 801 or by instructions in software form. The processor 801 may be a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Processor 801 can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. A general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the methods disclosed in the embodiments of this application can be directly manifested as being executed by a hardware decoding processor, or being executed by a combination of hardware and software modules in the decoding processor. The software modules may be located in a storage medium, which is located in memory 803. Processor 801 reads the information in memory 803 and combines it with its hardware to complete the steps of the aforementioned method.

[0185] In an exemplary embodiment, the device may be implemented by one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers (MCUs), microprocessors, or other electronic components to perform the aforementioned method.

[0186] It should be understood that the phrase "one embodiment" or "an embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that in the various embodiments of this application, the sequence numbers of the above-described processes do not imply a sequential order of execution; the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. The sequence numbers of the above-described embodiments are merely descriptive and do not represent the superiority or inferiority of the embodiments.

[0187] It should be noted that, in this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0188] The methods disclosed in the several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0189] The features disclosed in the several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0190] The features disclosed in the several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method or device embodiments.

[0191] The above description is merely an embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A text classification method, characterized in that, include: Get the first text; The first text is preprocessed to obtain character vectors included in the first text and at least one word vector corresponding to the character vectors; The word vector and each word vector in the at least one word vector are weighted and fused based on the Lattice-LSTM model to obtain the word vector at each position corresponding to each word vector; The Lattice-LSTM model, which performs weighted fusion processing on each word vector among the character vectors and at least one word vector to obtain the word lattice vector for each position corresponding to each word vector, includes: The Lattice-LSTM model is used to encode the character vector and each word vector to obtain the first encoding information of the character vector and the second encoding information of each word vector. A first weight coefficient for the first encoded information and a second weight coefficient for the second encoded information are determined; the first weight coefficient represents the proportion of the character vector in the at least one word vector; the second weight coefficient represents the proportion of each word vector in the at least one word vector. The word lattice vector is determined based on the first weight coefficient, the second weight coefficient, the first encoding information, and the second encoding information. Based on the capsule network model, semantic aggregation is performed on the word lattice vectors to obtain the first semantic vector for each position; Determine the category label corresponding to the first semantic vector at each position, and use the category label as the classification of the first text.

2. The method according to claim 1, characterized in that, The process of encoding the character vectors and each word vector based on the Lattice-LSTM model to obtain the first encoding information of the character vectors and the second encoding information of each word vector includes: The word vector is encoded based on the first word lattice structure in the Lattice-LSTM model to obtain the first encoded information; the first word lattice structure includes a first input gate, a first forget gate and a first output gate. Each word vector is encoded based on the second lattice structure in the Lattice-LSTM model to obtain the second encoded information; the second lattice structure includes a second input gate and a second forget gate.

3. The method according to claim 2, characterized in that, Determining the first weight coefficient of the first encoded information and the second weight coefficient of the second encoded information includes: The word vector is input into the first input gate for transformation to obtain the first state information of the word vector; Each word vector is input into the second input gate for transformation to obtain the second state information of each word vector; The first weight coefficient and the second weight coefficient are determined based on the first state information and the second state information.

4. The method according to claim 1, characterized in that, The semantic aggregation of the word lattice vectors based on the capsule network model to obtain the first semantic vector for each position includes: The first coupling coefficient corresponding to the word lattice vector is determined based on the capsule network model; the first coupling coefficient characterizes the degree to which the semantic features corresponding to each position are dynamically acquired in the capsule network model. The first influence weight of the semantic features corresponding to each position on the first text is determined based on the first coupling coefficient. The semantic features corresponding to each position are aggregated based on the first influence weight to obtain the first semantic vector.

5. The method according to claim 4, characterized in that, The step of aggregating the semantic features corresponding to each position based on the first influence weight to obtain the first semantic vector includes: Based on the first influence weight, the semantic features corresponding to each position are aggregated to obtain the second semantic vector; The second semantic vector is compressed to obtain the first semantic vector.

6. The method according to claim 5, characterized in that, The compression process of the second semantic vector to obtain the first semantic vector includes: Determine whether the modulus of the second semantic vector satisfies a preset condition; If the modulus of the second semantic vector satisfies the preset condition, the second semantic vector is compressed using the first compression function to obtain the first semantic vector; If the modulus of the second semantic vector does not meet the preset condition, the second semantic vector is compressed using a second compression function to obtain the first semantic vector. The compression coefficient of the first compression function is different from that of the second compression function, so that when the modulus of the second semantic vector approaches zero, the compressed second semantic vector will not lose some of the semantic vector information.

7. The method according to claim 4, characterized in that, The method further includes: The second coupling coefficient corresponding to the first semantic vector is determined based on the capsule network model; the second coupling coefficient is greater than the first coupling coefficient. The first coupling coefficient is updated based on the second coupling coefficient to obtain the updated first coupling coefficient; The first influence weight is re-determined based on the updated first coupling coefficient; Based on the re-determined first influence weight, the semantic features are re-aggregated to obtain the first semantic vector again, until the re-obtained first semantic vector converges, at which point the determination of the second coupling coefficient corresponding to the re-obtained first semantic vector based on the capsule network model is stopped.

8. The method according to claim 1, characterized in that, The preprocessing of the first text to obtain character vectors and at least one word vector corresponding to the character vectors includes: Obtain the first character in the first text and at least one word that ends with the first character; The character vector corresponding to the first character is matched according to the preset character vector dictionary; The at least one word vector corresponding to the at least one word is matched according to the preset word vector dictionary.

9. A text classification device, characterized in that, include: The acquisition unit is used to acquire the first text; A preprocessing unit is used to preprocess the first text to obtain character vectors included in the first text and at least one word vector corresponding to the character vectors; A weighted fusion processing unit is used to perform weighted fusion processing on the character vector and each word vector of the at least one word vector based on a Lattice-LSTM recurrent neural network model to obtain a word vector at each position corresponding to each word vector; wherein, the weighted fusion processing on the character vector and each word vector of the at least one word vector based on the Lattice-LSTM recurrent neural network model to obtain a word vector at each position corresponding to each word vector includes: encoding the character vector and each word vector respectively based on the Lattice-LSTM recurrent neural network model to obtain first encoding information of the character vector and second encoding information of each word vector; determining a first weight coefficient of the first encoding information and a second weight coefficient of the second encoding information; the first weight coefficient represents the proportion of the character vector in the at least one word vector; the second weight coefficient represents the proportion of each word vector in the at least one word vector; and determining the word vector based on the first weight coefficient, the second weight coefficient, the first encoding information, and the second encoding information; A semantic aggregation unit is used to perform semantic aggregation on the word lattice vectors based on a capsule network model to obtain a first semantic vector for each position. A determining unit is used to determine the category label corresponding to the first semantic vector of each position, and to use the category label as the classification of the first text.

10. An electronic device, characterized in that, include: Memory, used to store executable instructions; A processor, when executing executable instructions stored in the memory, implements the text classification method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that, It stores executable instructions for implementing the text classification method according to any one of claims 1 to 8 when executed by a processor.

Images