Text classification method and device, computer device and storage medium

Abstract

The application relates to a text classification method and device, computer equipment, a storage medium and a computer program product. The method comprises the following steps: obtaining a target text, inputting the target text into a target text classification model; performing text feature extraction on the target text by using the target text classification model to obtain initial text features; constructing the correlation degree between each word of the target text based on the initial text features by using the target text classification model to obtain initial structure features; performing feature aggregation on the initial text features and the initial structure features by using the target text classification model to obtain initial aggregated features; and outputting a text classification result corresponding to the target text based on the initial aggregated features. The method can improve the accuracy of text classification.

Description

Technical Field

[0001] This application relates to the field of natural language processing technology, and in particular to a text classification method, apparatus, computer device, storage medium, and computer program product. Background Technology

[0002] Text classification is an important research area in natural language processing, with a wide range of applications, such as sentiment classification, question answering, and public opinion monitoring. It can also be used to process complex text information, aiding in text mining.

[0003] Traditional text classification methods focus solely on the semantic features of words within the text, determining the meaning of the text based on these semantic features, and then classifying the text based on that meaning. This approach yields text classification results that are not very accurate. Summary of the Invention

[0004] Therefore, it is necessary to provide a text classification method, apparatus, computer device, computer-readable storage medium, and computer program product that can improve the accuracy of text classification in response to the above-mentioned technical problems.

[0005] This application provides a text classification method. The method includes:

[0006] Obtain the target text and input it into the target text classification model;

[0007] The target text is classified using a target text feature extraction model to obtain initial text features.

[0008] By using a target text classification model, based on the initial text features, the correlation between words in the target text is constructed to obtain the initial structural features;

[0009] The initial aggregated features are obtained by aggregating the initial text features and initial structural features using a target text classification model.

[0010] The text classification result corresponding to the target text is output based on the initial aggregation features.

[0011] This application also provides a text classification device. The device includes:

[0012] The target text acquisition module is used to acquire the target text and input it into the target text classification model;

[0013] The text feature extraction module is used to extract text features from the target text using the target text classification model to obtain initial text features;

[0014] The structural feature extraction module is used to construct the correlation between words in the target text based on the initial text features through the target text classification model, thereby obtaining the initial structural features;

[0015] The aggregation feature determination module is used to aggregate the initial text features and initial structural features through the target text classification model to obtain the initial aggregated features;

[0016] The classification result determination module is used to output the text classification result corresponding to the target text based on the initial aggregation features.

[0017] A computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the above-described text classification method.

[0018] A computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the above-described text classification method.

[0019] A computer program product includes a computer program that, when executed by a processor, implements the steps of the above-described text classification method.

[0020] The aforementioned text classification methods, devices, computer equipment, storage media, and computer program products acquire target text and input it into a target text classification model. The target text classification model then extracts text features from the target text to obtain initial text features. Based on these initial text features, the target text classification model constructs the correlation between words in the target text to obtain initial structural features. The target text classification model then aggregates the initial text features and initial structural features to obtain initial aggregated features. Finally, based on these initial aggregated features, the text classification result corresponding to the target text is output. In this way, the initial text features reflect the most basic semantic information of the target text, and the initial structural features, constructed from the correlation between words in the target text, can characterize the structural information between words in the target text, excluding semantic information. Aggregating the initial text features and initial structural features corresponding to the target text yields the initial aggregated features, which are high-order feature information that aggregates the semantic and structural information of the target text. Because the initial aggregated features fully integrate the semantic and structural information of the target text, they can more effectively and accurately reflect the feature information of the target text, making the text classification result based on the initial aggregated features more accurate. Attached Figure Description

[0021] Figure 1 This is a diagram illustrating the application environment of a text classification method in one embodiment.

[0022] Figure 2This is a flowchart illustrating a text classification method in one embodiment;

[0023] Figure 3 This is a flowchart illustrating the process of determining initial structural features in one embodiment;

[0024] Figure 4 This is a flowchart illustrating the process of constructing structural features of target text in one embodiment;

[0025] Figure 5 This is a framework diagram of a target text classification model in one embodiment;

[0026] Figure 6 This is a structural block diagram of a text classification device in one embodiment;

[0027] Figure 7 This is an internal structural diagram of a computer device in one embodiment;

[0028] Figure 8 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0029] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0030] The text classification method provided in this application embodiment can be applied to, for example, Figure 1 In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated onto server 104 or located on the cloud or other network servers. Terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can be smart TVs, smart in-vehicle devices, etc. Portable wearable devices can be smartwatches, smart bracelets, head-mounted devices, etc. Server 104 can be implemented using a standalone server or a server cluster consisting of multiple servers. Terminal 102 and server 104 can be directly or indirectly connected via wired or wireless communication, which is not limited herein.

[0031] Both the terminal and the server can be used independently to execute the text classification method provided in the embodiments of this application.

[0032] For example, the terminal acquires target text and inputs it into a target text classification model. The terminal extracts text features from the target text using the target text classification model, obtaining initial text features. Based on the initial text features, the terminal constructs the correlation between words in the target text using the target text classification model, obtaining initial structural features. The terminal then aggregates the initial text features and initial structural features using the target text classification model, obtaining initial aggregated features. Finally, the terminal outputs the text classification result corresponding to the target text based on the initial aggregated features.

[0033] Terminals and servers can also work together to execute the text classification method provided in the embodiments of this application.

[0034] For example, a terminal sends a text classification request to a server, carrying the text identifier of the target text. The server obtains the target text based on the text identifier and inputs it into a target text classification model. The server extracts text features from the target text using the target text classification model, obtaining initial text features. Based on the initial text features, the server constructs the correlation between words in the target text using the target text classification model, obtaining initial structural features. The server then aggregates the initial text features and initial structural features using the target text classification model, obtaining initial aggregated features. Based on the initial aggregated features, the server outputs the text classification result corresponding to the target text. The server can then send the text classification result to the terminal, which can then display the text classification result.

[0035] In one embodiment, such as Figure 2 As shown, a text classification method is provided. Taking the application of this method to a computer device as an example, the computer device can be a terminal or a server. The method can be executed independently by the terminal or server, or it can be implemented through interaction between the terminal and the server. The text classification method includes the following steps:

[0036] Step S202: Obtain the target text and input the target text into the target text classification model.

[0037] In this context, target text refers to text that requires classification to determine its category. Examples of target text include news articles, push notifications, course materials, and proverbs. A target text classification model is a model used to classify text. Its input is at least one target text, and its output is the text category corresponding to each target text. The target text classification model is a trained text classification model.

[0038] For example, a computer device acquires at least one target text locally or from another device, and uses the target text as input data for a target text classification model.

[0039] Step S204: Extract text features from the target text using the target text classification model to obtain initial text features.

[0040] Text features are used to represent the semantic information of the text, and include the semantic features corresponding to each word in the text. Initial text features refer to the text features obtained by extracting text features from the target text, which are used to represent the semantic information of the target text.

[0041] For example, by using a target text classification model, text feature extraction is performed on the target text input to the target text classification model to obtain the initial text features corresponding to the target text.

[0042] Step S206: Using the target text classification model, based on the initial text features, construct the correlation between each word in the target text to obtain the initial structural features.

[0043] The correlation between words refers to the degree of association between two words, used to construct the structural features corresponding to the target text. Structural features are the structural information of the target text constituted by the correlation between the words in the text, representing the relationships between words in the target text. The initial structural features are structural features obtained based on the initial text features.

[0044] For example, using a target text classification model, the correlation between words in the target text is calculated based on the initial text features corresponding to the target text. Based on the correlation between words, the initial structural features corresponding to the target text are obtained.

[0045] In one embodiment, feature enhancement is performed on the initial text features to obtain at least one target text feature, which includes the semantic features of each word in the target text. Feature enhancement improves the semantic representation capability of the target text features. Reference text features are determined from the target text features, and the semantic features of each word in the reference text features are interacted with the semantic features of each word in the other target text features to obtain the correlation between the words in the target text. Based on the correlation between the words, the initial structural features of the target text are obtained.

[0046] Step S208: Using the target text classification model, feature aggregation is performed on the initial text features and initial structural features to obtain the initial aggregated features.

[0047] Aggregated features refer to features obtained by aggregating textual and structural features, used to represent high-order feature information that aggregates the semantic and structural information of the target text. Initial aggregated features are aggregated features obtained by aggregating initial textual and initial structural features.

[0048] For example, by using a target text classification model, feature aggregation operations are performed on the initial text features and initial structural features corresponding to the target text to obtain the initial aggregated features corresponding to the target text.

[0049] Step S210: Output the text classification result corresponding to the target text based on the initial aggregation features.

[0050] The text classification result refers to the text category corresponding to the target text.

[0051] For example, the target text classification model processes the initial aggregated features corresponding to the target text to obtain the text classification result corresponding to the target text, and outputs the text classification result.

[0052] In the above text classification method, the target text is acquired and input into a target text classification model. The target text classification model extracts text features to obtain initial text features. Based on these initial text features, the model constructs the correlation between words in the target text to obtain initial structural features. The model then aggregates these initial text features and initial structural features to obtain initial aggregated features. Finally, the text classification result corresponding to the target text is output based on these initial aggregated features. In this way, the initial text features reflect the most basic semantic information of the target text, while the initial structural features, constructed from the correlation between words in the target text, represent the structural information between words in the target text, excluding semantic information. Aggregating the initial text features and initial structural features yields the initial aggregated features, which are high-order feature information that integrates the semantic and structural information of the target text. Because the initial aggregated features fully integrate the semantic and structural information of the target text, they can more effectively and accurately reflect the feature information of the target text, resulting in more accurate text classification results.

[0053] In one embodiment, the target text is used to extract text features from the target text using a target text classification model to obtain initial text features, including:

[0054] The target text is transformed to obtain word embedding features, position embedding features and sentence embedding features corresponding to each word of the target text; based on the word embedding features, position embedding features and sentence embedding features corresponding to the same word, the text embedding features corresponding to the target text are obtained; attention processing is applied to the text embedding features to obtain the initial text features.

[0055] Feature transformation refers to converting each word in the target text into three different vector representations: word embedding features, positional embedding features, and sentence embedding features. Word embedding features are fixed-dimensional vector representations of each word in the target text. Positional embedding features are vector representations used to characterize the input order of words in the target text. Sentence embedding features are vector representations that help distinguish between two sentences in a sentence pair. Attention processing refers to processing the text embedding features using a multi-head attention mechanism, extracting text feature information from multiple dimensions, and strengthening the more important features.

[0056] For example, feature transformation is performed on the target text to obtain word embedding features, positional embedding features, and sentence embedding features for each word. These features are then summed and concatenated to obtain the text embedding features for the target text. Attention processing is then applied to these text embedding features to obtain the initial text features for the target text.

[0057] In the above embodiments, the target text classification model performs feature transformation on the target text to obtain word embedding features, positional embedding features, and sentence embedding features corresponding to each word in the target text. Word embedding features give different words different vector representations; positional embedding features indicate the input order of words in the target text, allowing for word differentiation based on input order when the same word appears; and sentence embedding features distinguish different sentences in the target text. The text embedding features obtained by aggregating word embedding features, positional embedding features, and sentence embedding features comprehensively reflect the feature information of each word in the target text, effectively improving the accuracy of text classification. The initial text features obtained by processing the text embedding features based on a multi-head attention mechanism can more accurately reflect the semantic information of the target text, thereby improving the accuracy of text classification.

[0058] In one embodiment, such as Figure 3 As shown, the target text classification model includes a structural feature extraction layer, which includes at least one text feature processing channel. Based on the initial text features, the model constructs the correlation between words in the target text to obtain initial structural features, including:

[0059] Step S302: Input the initial text features into the structural feature extraction layer, and process the initial text features through each text feature processing channel to obtain at least one target text feature; the target text feature includes the semantic features of each word in the target text.

[0060] Step S304: Determine reference text features from each target text feature.

[0061] Step S306: Perform feature interaction between the semantic features of each word in the reference text features and the semantic features of each word in the other target text features to obtain the correlation between the words in the target text.

[0062] Step S308: Based on the correlation between words in the target text, the initial structural features are obtained.

[0063] In this context, the structural feature extraction layer refers to the model layer in the target text classification model used to extract the structural features of the target text. The text feature processing channel refers to the channel capable of processing text features, used to further extract different semantic information corresponding to each word in the target text based on the initial text features. In one embodiment, the text feature processing channel is a module capable of feature extraction, such as an MLP network (Multi-Layer Perceptron) or a BP network (Back Propagation neural network). The target text feature refers to the text features obtained after the text feature processing channel processes the initial text features, used to represent the semantic information corresponding to each word in the target text. For example, if the structural feature extraction layer has two text feature processing channels, M1 and M2, the initial text feature X is processed by text feature processing channel M1 to obtain target text feature X1, and the initial text feature X is processed by text feature processing channel M2 to obtain target text feature X2. X1 and X2 represent two different sets of semantic information corresponding to each word in the target text.

[0064] Reference text features refer to any target text features arbitrarily selected from among the various target text features. Word semantic features refer to the feature vectors corresponding to each word in the target text, used to represent the semantic information of the words. Initial structural features refer to the structural features of the target text obtained based on the initial text features.

[0065] For example, the computer device inputs the initial text features corresponding to the target text into the text feature processing channels in the structural feature extraction layer to process the initial text features and obtain at least one target text feature. At least one target text feature is selected as a reference text feature from among the target text features. The semantic features of each word in the reference text feature are then used for feature interaction with the semantic features of each word in other target text features to obtain the correlation between the words in the target text. Based on the correlation between the words in the target text, the initial structural features corresponding to the target text are obtained.

[0066] In one embodiment, the structural feature extraction layer has only one text feature processing channel. Initial text features are input into this channel to obtain a target text feature. The correlation between words in the target text feature is obtained by performing feature interactions between the semantic features of each word and the semantic features of other words in the target text feature. Based on these correlations, the initial structural feature of the target text is derived. For example, if the target text feature is X, the transpose of the target text feature can be calculated first, then multiplied by the transpose, and the result of the multiplication is activated to obtain the initial structural feature of the target text.

[0067] In one embodiment, the structural feature extraction layer includes two text feature processing channels. For example... Figure 4 As shown, if the structural feature extraction layer uses an MLP network as the text feature processing channel, and there are only two text feature processing channels in the structural feature extraction layer, namely MLP_1 and MLP_2, then the two MLP networks have the same parameter size and the same function. The difference is that the parameters of the two MLP networks are optimized independently during training. If the input data of the target text classification model is a target text of length n, where n is the number of words in the target text, then the initial text features input to the structural feature extraction layer are matrix X. n×C C represents the length of the semantic features corresponding to each word in the target text. The initial text features X are input into MLP_1 and MLP_2 respectively for feature extraction to obtain the target text features X1. n×C′ and target text features X2 n×C′ C′ represents the length of the semantic features of each word in the target text after processing by the text feature processing layer. Using the target text feature X1 as the reference text feature, the semantic features of each word in the reference text feature and the semantic features of each word in the target text feature X2 are interacted to obtain the correlation between words in the target text. Based on the correlation between words, the initial structural features of the target text are obtained. For example, the transpose of the target text feature X2 can be multiplied by the reference text feature X1, and the softmax activation function can be used to activate the product of the transpose of X2 and X1 to obtain the initial structural feature A. n×n Alternatively, the semantic features of each word in the reference text feature X1 can be calculated, and the semantic features of each word in the target text feature X2 can be compared. These similarities are then used as the correlation between words in the target text. Based on this correlation, the initial structural feature A is obtained. n×nThe element in the i-th row and j-th column of the matrix represents the correlation between the i-th and j-th words in the target text; and so on. Compared to using only one MLP network to construct the structural features corresponding to the target text, using two MLP networks allows X1 and X2 to be optimized independently during backpropagation during model training, making the optimization process of structural features easier. After the model is trained, the two MLP networks can effectively capture the different semantic information corresponding to each word, realizing high-order information interaction between different words. Using MLP networks can complete the transformation from low-level features to high-level features, and MLP networks can complete effective feature transformation with low space consumption and low time complexity.

[0068] In the above embodiments, using a single text feature processing channel to process the initial text features yields a target text feature. Based on this target text feature, the corresponding initial structural features of the target text are then obtained. This improves the efficiency of calculating the initial structural features and saves computer memory resources, thereby improving the classification efficiency of the target text classification model. Using multiple text feature processing channels to process the initial text features separately effectively captures the different semantic information of each word in the target text, allowing the target text features obtained from the initial text features to more comprehensively and fully reflect the semantic information of each word in the target text. Based on each target text feature, the correlation between words in the target text is calculated, thus obtaining the corresponding initial structural features of the target text. It is evident that the initial structural features integrate the different semantic information corresponding to each word in the target text and uncover the connections between different words. In other words, obtaining the initial structural features based on the initial text features achieves the transformation from low-order features to high-order features. The initial structural features can reflect the high-order feature information of the target text. Classifying the target text based on the initial structural features can effectively improve the accuracy of text classification.

[0069] In one embodiment, the initial text features and initial structural features are aggregated using a target text classification model to obtain initial aggregated features, including:

[0070] The first aggregated feature is obtained by fusing the initial text features and the initial structural features; the second aggregated feature is obtained by activating the first aggregated feature using the first activation function; the third aggregated feature is obtained by fusing the second aggregated feature and the initial text features; the initial aggregated feature is obtained by activating the third aggregated feature using the second activation function.

[0071] Here, the first activation function refers to the activation function used to activate the first aggregated feature. The second activation function refers to the activation function used to activate the third aggregated feature. Aggregated features refer to the aggregated features obtained by aggregating the textual and structural features corresponding to the target text, and can represent the feature information extracted from the textual and structural features corresponding to the target text. Initial aggregated features refer to the aggregated features obtained by aggregating the initial textual and structural features corresponding to the target text.

[0072] For example, the computer device aggregates initial text features and initial structural features to obtain a first aggregated feature. For instance, the product of the initial text features, initial structural features, and a parameter matrix can be used as the first aggregated feature; the product of the initial text features, initial structural features, and a parameter matrix can be multiplied by a preset value to obtain the first aggregated feature; and so on. A first activation function is applied to the first aggregated feature to obtain a second aggregated feature. To make the initial aggregated features extracted by the target text classification model more complete and to ensure that the target text classification model does not degenerate, residual connections are performed on the second aggregated feature, that is, a fusion operation is performed between the second aggregated feature and the initial text features to obtain a third aggregated feature. For instance, the sum of the second aggregated feature and the initial text features can be used as the third aggregated feature; the sum of the second aggregated feature and the initial text features can be calculated, and then multiplied by a preset value to obtain the third aggregated feature; and so on. A second activation function is applied to the third aggregated feature to obtain the initial aggregated feature corresponding to the target text.

[0073] In the above embodiments, the computer device aggregates initial text features and initial structural features to obtain a first aggregated feature. This first aggregated feature characterizes the features extracted by the target text classification model based on the corresponding text and structural features of the target text. It can be understood that the calculation of the first aggregated feature considers not only the semantic information of the target text but also its structural information, comprehensively and fully reflecting the feature information of the target text. The first aggregated feature is activated using a first activation function to obtain a second aggregated feature. Residual connections are then performed on the second aggregated feature, that is, the second aggregated feature is fused with the initial text features to obtain a third aggregated feature. This makes the initial aggregated features extracted by the target text classification model more complete and ensures that the target text classification model does not degenerate, thereby effectively improving the accuracy of text classification.

[0074] In one embodiment, the second aggregation feature of the target text can be calculated using the following formula:

[0075] f(X,A1)=σ(A1XW1)

[0076] Where f(X,A1) is the second aggregated feature, X is the initial text feature corresponding to the target text, A1 is the initial structural feature corresponding to the target text, W1 is the parameter matrix, σ is the first activation function, and A1XW1 is the first aggregated feature.

[0077] In one embodiment, the third aggregation feature of the target text can be calculated using the following formula:

[0078] Y = f(X, A1) + X

[0079] Where Y is the third aggregation feature.

[0080] In one embodiment, the initial aggregate features of the target text can be calculated using the following formula:

[0081] Y1 = Relu(f(X,A1) + X)

[0082] Where Y1 is the initial aggregated feature and ReLU is the second activation function.

[0083] In one embodiment, the text classification result corresponding to the target text is output based on the initial aggregated features, including:

[0084] Based on the initial aggregated features, the correlation between words in the target text is constructed to obtain the target structural features; the initial aggregated features and the target structural features are aggregated to obtain the target aggregated features; and the text classification result corresponding to the target text is output based on the target aggregated features.

[0085] The target structural feature refers to the structure obtained based on the initial aggregated features. The target aggregated feature refers to the aggregated feature obtained by aggregating the initial aggregated features and the target structural feature.

[0086] For example, when obtaining the text classification result corresponding to the target text based on the initial aggregated features, the semantic and structural information in the initial aggregated features can be further fully integrated by constructing structural features and aggregating features to obtain target aggregated features with stronger semantic and structural representation capabilities. Finally, the text classification result is obtained based on the target aggregated features.

[0087] Specifically, computer equipment can first construct the correlation between words in the target text based on the initial aggregated features to obtain the target structural features, and then perform feature aggregation on the initial aggregated features and the target structural features to obtain the target aggregated features.

[0088] In one embodiment, the target text classification model includes two structural feature extraction layers. Initial text features are input into the first structural feature extraction layer, which constructs the correlation between words in the target text, resulting in initial structural features. The initial text features and initial structural features are then aggregated to obtain initial aggregated features. These initial aggregated features are input into the second structural feature extraction layer, which constructs the correlation between words in the target text, resulting in target structural features. The initial aggregated features and initial structural features are then aggregated to obtain target aggregated features. Finally, the text classification result corresponding to the target text is obtained based on the target aggregated features.

[0089] In one embodiment, initial aggregated features are input into a second structural feature extraction layer. The initial aggregated features are processed through each text feature processing channel in the second structural feature extraction layer to obtain at least one intermediate aggregated feature. New reference aggregated features are determined from these intermediate aggregated features. The semantic features of each word in the new reference aggregated feature are interacted with the semantic features of each word in the other intermediate aggregated features to obtain the correlation between words. Based on these correlations, target structural features are obtained. The initial aggregated features and target structural features are aggregated to obtain a first aggregated feature. A first activation function is applied to the first aggregated feature to obtain a second aggregated feature. The second and initial aggregated features are fused to obtain a third aggregated feature. A second activation function is applied to the third aggregated feature to obtain the target aggregated feature corresponding to the target text. The target aggregated feature is input into a classification layer to obtain the text classification result corresponding to the target text.

[0090] In the above embodiments, structural features corresponding to the target text are reconstructed based on the initial aggregated features, enabling the target text classification model to obtain aggregated features based on different structural features, thereby enhancing the flexibility of the target text classification model. The target aggregated features obtained by aggregating the initial aggregated features and the target structural features can more comprehensively and fully reflect the feature information of the target text. Inputting the target aggregated features into the classification layer to obtain the text classification result corresponding to the target text can effectively improve the accuracy of text and classification.

[0091] In one embodiment, the target text classification model includes a text feature extraction layer, a structural feature extraction layer, a feature aggregation layer, and a classification layer. The text feature extraction layer is used to extract the text features corresponding to the text, the structural feature extraction layer is used to extract the structural features corresponding to the text, the feature aggregation layer is used to aggregate the text features and structural features to obtain the aggregated features corresponding to the text, and the classification layer is used to output the text classification result corresponding to the text.

[0092] For example, when target text is input into a target text classification model, the target text is first fed into the text feature extraction layer, which extracts initial text features corresponding to the target text. These initial text features are then input into the structure feature extraction layer. The structure feature extraction layer receives the initial text features and extracts the initial structure features corresponding to the target text, which are then input into the feature aggregation layer. The feature aggregation layer acquires the initial text features and the initial structure features, aggregating them to obtain the initial aggregated features corresponding to the target text. These initial aggregated features are then input into the classification layer. The classification layer receives the initial aggregated features, obtains the text classification result corresponding to the target text based on these features, and then outputs the text classification result.

[0093] In one embodiment, the target text classification model may include at least one combined layer connected in sequence. Each combined layer includes a structural feature extraction layer and a feature aggregation layer connected in sequence. The target text is input to the text feature extraction layer, which outputs initial text features. These initial text features are then input to the first combined layer, which outputs initial aggregated features. The initial aggregated features are then input to the second combined layer, which outputs new aggregated features, and so on, until the last combined layer outputs the final aggregated features. The final aggregated features are then input to a classifier to obtain the text classification result corresponding to the target text. The target text classification model outputs the text classification result.

[0094] In one embodiment, the feature aggregation layer is a graph convolutional neural network. Graph convolutional neural networks are a very powerful type of graph neural network architecture for graph data.

[0095] In the above embodiments, the target text is processed layer by layer through the text feature extraction layer, structural feature extraction layer, feature aggregation layer, and classification layer in the target text classification model to obtain the corresponding text features, structural features, aggregated features, and text classification results. Text features can represent the semantic information of the target text, structural features can represent the structural information between words in the target text, and aggregated features can represent higher-order feature information that aggregates semantic and structural information. Therefore, the aggregated features obtained based on text features and structural features can more completely and comprehensively reflect the feature information of the target text. The classifier obtains the text classification result corresponding to the target text based on the aggregated feature information, which can effectively improve the accuracy of text classification.

[0096] In one specific embodiment, the user data processing method of this application can be applied to text classification of news reports in different fields. For example... Figure 5 As shown, the text classification method includes the following steps:

[0097] 1. Extract initial text features

[0098] B text input sequences of length n are input into the target text classification model. The input embedding module first performs feature transformation on each text in the input sequence to obtain word embedding features, position embedding features, and sentence embedding features for each text. The word embedding features, position embedding features, and sentence embedding features are tensors of shape (1, n, C). First, the word embedding features, position embedding features, and sentence embedding features corresponding to each word are summed and concatenated. Then, the summed and concatenated results are subjected to layer normalization and dropout (random deactivation) regularization to obtain the text embedding features corresponding to each text. Layer normalization ensures that the data follows a normal distribution, thus ensuring that the gradients of each parameter do not differ too much in subsequent processing, improving the convergence of the model. Regularization can alleviate the overfitting problem in model training. The text embedding features are tensors of shape (1, n, C). Since the number of texts in the input sequence is B, a tensor Z of shape (B, n, C) is obtained. Z is fed into the text feature extraction layer as input, and processed using a multi-head attention mechanism to obtain a tensor Z1 composed of new vector codes corresponding to each word. Each computation in a single self-attention mechanism involves three weight matrices W. q W k W v The three weight matrices are linearly multiplied by Z to obtain three matrices Q, K, and V of size (B, n, C).

[0099] Computer devices can calculate Z1 using the following formula:

[0100]

[0101] The tensor Z1 is subjected to layer normalization and regularization, and then input into the MLP for data processing. The output of the MLP is subjected to layer normalization and regularization to obtain the initial text feature X, which is a tensor with shape (B,n,C).

[0102] 2. Construct the graph structure and calculate the initial aggregation features.

[0103] The initial text features X are input into the first ensemble layer. In this layer, two MLP networks, MLP_1 and MLP_2, are used as text feature extraction channels to perform a mapping operation (structural feature extraction) on each text element. MLP_1 and MLP_2 process the initial text features, and the results are standardized and activated (using ReLU activation function) to obtain target text features X1 and X2. Both X1 and X2 are tensors of shape (B, n, C'), where C' is the length of the semantic features of each word in the text after processing by the text feature processing layer. Based on X1 and X2, the initial structural feature A1 corresponding to the target text is obtained, which is also a tensor of shape (B, n, n). The initial text features X and initial structural feature A1 are aggregated to obtain aggregated feature Y, which is a tensor of shape (B, n, C). A residual connection is performed on the aggregated feature Y, which is the sum of the aggregated feature and the initial text features, followed by activation processing to obtain the initial aggregated feature Y1.

[0104] Computer devices can calculate the initial structural feature A1 using the following formula:

[0105] A1 = softmax(X1 * X2^T)

[0106] Computer devices can calculate aggregate feature Y using the following formula:

[0107] Y = f(X, A1)

[0108] Computer devices can calculate the initial aggregate feature Y1 using the following formula:

[0109] Y1 = Relu(f(X,A1) + X)

[0110] 3. Construct target structural features and calculate target aggregation features.

[0111] The initial aggregated feature Y1 is input into the second combined layer to obtain the target structural feature A2 and the target aggregated feature Y2 corresponding to the text.

[0112] Computer equipment can calculate the target structural feature A2 using the following formula:

[0113] A2 = softmax(Y1*Y2^T)

[0114] Where Y1 is the output obtained by inputting the initial aggregated feature Y1 into MLP_1 for data processing, and then performing standardization and activation processing (ReLU activation function) on the processing result. Standardization refers to scaling the feature information to make different features have the same numerical value. Y2 is the output obtained by inputting the initial aggregated feature Y1 into MLP_2 for data processing, and then performing standardization and activation processing (ReLU activation function) on the processing result.

[0115] Computer devices can calculate the target aggregate feature Y2 using the following formula:

[0116] Y2 = Relu(f(Y1,A2) + Y1)

[0117] 4. Output text classification results based on target aggregation features.

[0118] The target aggregated features are input into the classification layer. These features are flattened to obtain a tensor of shape (B, N×C). This tensor is then input into MLP_3 for data processing, and the result is standardized and activated (using the ReLU activation function) to obtain another tensor of shape (B, C). This tensor is then input into MLP_4 for data processing, and the result is standardized and activated (using the Softmax activation function) to obtain the text classification result. Finally, the text classification result is output as a tensor of shape (B, k), where k is the number of text categories. For example, if the target text classification model is a machine learning model used to determine whether the input text belongs to the category "finance" or "sports," the text classification result for each text in the input sequence can be labeled "finance," "sports," or "none." The "none" label indicates that the text does not belong to either the "finance" or "sports" category.

[0119] In the above embodiments, the target text classification model not only extracts textual feature information from the input text sequence but also constructs a graph structure for each text (extracting structural features). The graph structure can effectively reflect the structural relationships between words in the text. Thus, the target text classification model can learn not only the semantic information of the text but also its structural information, effectively improving the learning effect of text representation in target text classification. Simultaneously, using an MLP network as the text feature extraction channel to process the initial text features enables the transformation from low-level features to high-level features. Furthermore, the MLP network can perform effective feature transformation with low space consumption and low time complexity. Aggregating textual features and graph structures to obtain aggregated features allows the semantic and structural information of the text to interact and influence each other. Introducing a residual network further refines the features extracted by the model and prevents model degradation. Based on the target aggregated features corresponding to each text, the text classification results for each text are obtained, effectively improving the accuracy of text classification.

[0120] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0121] Based on the same inventive concept, this application also provides a text classification apparatus for implementing the text classification method described above. The solution provided by this apparatus is similar to the implementation scheme described in the above method; therefore, the specific limitations in one or more text classification apparatus embodiments provided below can be found in the limitations of the text classification method described above, and will not be repeated here.

[0122] In one embodiment, such as Figure 6 As shown, a text classification device is provided, including: a target text acquisition module 602, a text feature extraction module 604, a structural feature extraction module 606, an aggregation feature determination module 608, and a classification result determination module 610, wherein:

[0123] The target text acquisition module 602 is used to acquire target text and input the target text into the target text classification model.

[0124] The text feature extraction module 604 is used to extract text features from the target text using a target text classification model to obtain initial text features.

[0125] The structural feature extraction module 606 is used to construct the correlation between words in the target text based on the initial text features through the target text classification model, thereby obtaining the initial structural features.

[0126] The aggregation feature determination module 608 is used to aggregate the initial text features and initial structural features through the target text classification model to obtain the initial aggregated features.

[0127] The classification result determination module 610 is used to output the text classification result corresponding to the target text based on the initial aggregation features.

[0128] The aforementioned text classification device acquires target text and inputs it into a target text classification model. The model extracts text features to obtain initial text features. Based on these initial features, the model constructs the relationships between words in the target text to obtain initial structural features. The model then aggregates these initial text features and structural features to obtain initial aggregated features. Finally, it outputs the text classification result corresponding to the target text. In this way, the initial text features reflect the most basic semantic information of the target text, while the initial structural features, constructed from the relationships between words, represent the structural information between words in the target text, excluding semantic information. Aggregating the initial text features and initial structural features yields the initial aggregated features, which are high-order features that combine the semantic and structural information of the target text. Because the initial aggregated features fully integrate the semantic and structural information of the target text, they can more effectively and accurately reflect the feature information of the target text, resulting in more accurate text classification results.

[0129] In one embodiment, the text feature extraction module 604 is further configured to:

[0130] The target text is transformed to obtain word embedding features, position embedding features and sentence embedding features corresponding to each word of the target text; based on the word embedding features, position embedding features and sentence embedding features corresponding to the same word, the text embedding features corresponding to the target text are obtained; attention processing is applied to the text embedding features to obtain the initial text features.

[0131] In one embodiment, the structural feature extraction module 606 is further configured to:

[0132] Initial text features are input into the structural feature extraction layer. The initial text features are processed through each text feature processing channel to obtain at least one target text feature. The target text feature includes the semantic features of each word in the target text. Reference text features are determined from the target text features. The semantic features of each word in the reference text feature are interacted with the semantic features of each word in the other target text features to obtain the correlation between the words in the target text. Based on the correlation between the words in the target text, the initial structural features are obtained.

[0133] In one embodiment, the aggregation feature determination module 608 is further configured to:

[0134] The first aggregated feature is obtained by fusing the initial text features and the initial structural features; the second aggregated feature is obtained by activating the first aggregated feature using the first activation function; the third aggregated feature is obtained by fusing the second aggregated feature and the initial text features; the initial aggregated feature is obtained by activating the third aggregated feature using the second activation function.

[0135] In one embodiment, the classification result determination module 610 is further configured to:

[0136] Based on the initial aggregated features, the correlation between words in the target text is constructed to obtain the target structural features; the initial aggregated features and the target structural features are aggregated to obtain the target aggregated features; and the text classification result corresponding to the target text is output based on the target aggregated features.

[0137] In one embodiment, the target text classification model includes a text feature extraction layer, a structural feature extraction layer, a feature aggregation layer, and a classification layer. The text feature extraction layer is used to extract the text features corresponding to the text, the structural feature extraction layer is used to extract the structural features corresponding to the text, the feature aggregation layer is used to aggregate the text features and structural features to obtain the aggregated features corresponding to the text, and the classification layer is used to output the text classification result corresponding to the text.

[0138] Each module in the aforementioned text classification device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0139] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 7As shown, this computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media. The database stores data such as initial text features, initial structural features, initial aggregation features, and text classification results. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When executed by the processor, the computer program implements a text classification method.

[0140] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 8 As shown, the computer device includes a processor, memory, input / output interfaces, a communication interface, a display unit, and an input device. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When executed by the processor, the computer program implements a text classification method. The display unit is used to form a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0141] Those skilled in the art will understand that Figure 7 , 8The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0142] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0143] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above method embodiments.

[0144] In one embodiment, a computer program product or computer program is provided, the computer product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and executes the computer instructions, causing the computer device to perform the steps in the above-described method embodiments.

[0145] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data shall comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0146] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0147] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0148] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A text classification method, characterized in that, The method includes: Obtain the target text and input the target text into a target text classification model. The target text classification model includes a structural feature extraction layer, and the structural feature extraction layer includes at least one text feature processing channel. The target text is used to extract text features from the target text using the target text classification model to obtain initial text features; Using the target text classification model, based on the initial text features, the correlation between words in the target text is constructed to obtain initial structural features. This includes: inputting the initial text features into the structural feature extraction layer; processing the initial text features through various text feature processing channels to obtain at least one target text feature; the target text feature includes the semantic features of each word in the target text; determining reference text features from the target text features; performing feature interaction between the semantic features of each word in the reference text features and the semantic features of each word in other target text features to obtain the correlation between words in the target text; and obtaining the initial structural features based on the correlation between words in the target text. The initial aggregated features are obtained by aggregating the initial text features and the initial structural features using the target text classification model. Based on the initial aggregation features, the text classification result corresponding to the target text is output.

2. The method according to claim 1, characterized in that, The step of extracting text features from the target text using the target text classification model to obtain initial text features includes: The target text is subjected to feature transformation to obtain word embedding features, position embedding features and sentence embedding features corresponding to each word of the target text. Based on the word embedding features, position embedding features, and sentence embedding features corresponding to the same word, the text embedding features corresponding to the target text are obtained. Attention processing is applied to the text embedding features to obtain the initial text features.

3. The method according to claim 1, characterized in that, The target text includes one of the following: news text, push notification text, course text, and proverb text.

4. The method according to claim 1, characterized in that, The step of aggregating the initial text features and the initial structural features using the target text classification model to obtain initial aggregated features includes: The initial text features and the initial structural features are fused to obtain the first aggregated feature; The first aggregated feature is activated by the first activation function to obtain the second aggregated feature; The second aggregated feature and the initial text feature are combined to obtain the third aggregated feature; The third aggregated feature is activated by a second activation function to obtain the initial aggregated feature.

5. The method according to claim 1, characterized in that, The step of outputting the text classification result corresponding to the target text based on the initial aggregation features includes: Based on the initial aggregation features, the correlation between words in the target text is constructed to obtain the target structural features; The initial aggregated features and the target structural features are aggregated to obtain the target aggregated features; Based on the target aggregation features, the text classification result corresponding to the target text is output.

6. The method according to any one of claims 1 to 5, characterized in that, The target text classification model includes a text feature extraction layer, a structural feature extraction layer, a feature aggregation layer, and a classification layer. The text feature extraction layer is used to extract the text features corresponding to the text. The structural feature extraction layer is used to extract the structural features corresponding to the text. The feature aggregation layer is used to aggregate the text features and the structural features to obtain the aggregated features corresponding to the text. The classification layer is used to output the text classification result corresponding to the text.

7. A text classification device, characterized in that, The device includes: The target text acquisition module is used to acquire target text and input the target text into a target text classification model. The target text classification model includes a structural feature extraction layer, and the structural feature extraction layer includes at least one text feature processing channel. The text feature extraction module is used to extract text features from the target text using the target text classification model to obtain initial text features; The structural feature extraction module is used to construct the correlation between words in the target text based on the initial text features using the target text classification model, thereby obtaining initial structural features. The module includes: inputting the initial text features into the structural feature extraction layer; processing the initial text features through various text feature processing channels to obtain at least one target text feature; the target text feature includes the semantic features of each word in the target text; determining reference text features from the target text features; performing feature interaction between the semantic features of each word in the reference text features and the semantic features of each word in other target text features to obtain the correlation between words in the target text; and obtaining the initial structural features based on the correlation between words in the target text. The aggregation feature determination module is used to aggregate the initial text features and the initial structural features through the target text classification model to obtain initial aggregate features; The classification result determination module is used to output the text classification result corresponding to the target text based on the initial aggregation features.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images