Text classification method, apparatus, device, and storage medium

Abstract

This invention relates to the field of artificial intelligence, and more particularly to a text classification method, apparatus, device, and storage medium. The method includes: acquiring a text sequence to be classified; extracting local features from the text sequence to be classified through convolutions of different scales to obtain multiple different feature subspaces; performing global attention calculation on each different feature subspace through an attention mechanism to obtain the features output by each feature subspace; fusing and classifying the features output by each feature subspace to obtain the final vector representation of the sentence sequence; and performing a linear transformation on the final vector representation of the sentence sequence to obtain the text classification result, thereby achieving accurate and simple text classification.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence, and in particular to a text classification method, apparatus, device, and storage medium. Background Technology

[0002] Currently, mainstream algorithms for text classification tasks are primarily based on CNNs, RNNs, and attention mechanisms, with a series of combined models derived from these, such as models based on CNNs and RNNs, models based on RNNs and attention mechanisms, and models based on CNNs and attention mechanisms. However, these models suffer from structural defects, high complexity, and shortcomings in performance and accuracy. Therefore, how to accurately and simply perform text classification has become a pressing technical problem that needs to be solved.

[0003] However, some researchers have attempted to indirectly extract long-range features by increasing the depth of the network, proposing models such as VDCNN, DPCNN, and SVDCNN. This has alleviated the shortcomings of CNN to some extent, but these designs have also made the model more complex, resulting in a large number of parameters, greater performance overhead, and making the model more prone to overfitting during training.

[0004] The above content is only used to help understand the technical solution of the present invention and does not represent an admission that the above content is prior art. Summary of the Invention

[0005] The main objective of this invention is to provide a text classification method, apparatus, device, and storage medium, aiming to solve the technical problem that it is difficult to accurately and simply complete text classification in the prior art.

[0006] To achieve the above objectives, the present invention provides a text classification method, which includes the following steps:

[0007] A text sequence to be classified is obtained, and local features are extracted from the text sequence to be classified through convolution at different scales to obtain multiple different feature subspaces;

[0008] Global attention is calculated for each different feature subspace through the attention mechanism to obtain the features output by each feature subspace.

[0009] The features output from each feature subspace are fused and classified to obtain the final vector representation of the sentence sequence;

[0010] A linear transformation is performed on the final vector representation of the sentence sequence to obtain the text classification result.

[0011] Optionally, before obtaining the text sequence to be classified and extracting local features from the text sequence to be classified through convolutions of different scales to obtain multiple different feature subspaces, the method further includes:

[0012] The initial word vector weights are scaled according to a preset rule to obtain scaled weights, and the original sentence sequence is summed according to the scaled weights to obtain the summed embedding vector.

[0013] The summed embedding vector is processed using Dropout and LayerNorm to obtain the text sequence to be classified.

[0014] Optionally, the step of extracting row-local features from the text sequence to be classified through convolutions of different scales yields multiple different feature subspaces, including:

[0015] Based on the text sequence to be classified, a sentence word vector matrix parallel to the layer is obtained;

[0016] The sentence word vector matrix parallel to the layer is subjected to n-gram convolution with convolution kernels of different scales to obtain different feature subspaces.

[0017] Optionally, the step of performing global attention calculation on each different feature subspace through an attention mechanism to obtain the features output by each feature subspace includes:

[0018] Multiple Self-Attention processes are performed on the different feature subspaces to capture important features in the different feature subspaces;

[0019] The important features in the different feature subspaces are concatenated using Concat, and the features output by each subspace are obtained by multiplying them with a mapping matrix.

[0020] Optionally, performing multiple Self-Attention processes on the different feature subspaces to capture important features in the different feature subspaces includes:

[0021] The weight coefficients of keywords in the classification process are adjusted based on the contribution of each word to the text features in different feature subspaces.

[0022] Based on the weight coefficients, secondary feature calculations are performed on the local features extracted by convolution in a global manner to obtain important features in different feature subspaces.

[0023] Optionally, the step of fusing and classifying the features output from each feature subspace to obtain the final vector representation of the sentence sequence further includes:

[0024] The features output from each subspace and the local features obtained through text convolution are residually concatenated to obtain the word vectors after residual concatenation.

[0025] The word vectors after residual concatenation are averaged to obtain the final vector representation of the sentence sequence.

[0026] Optionally, the linear transformation of the final vector representation of the sentence sequence to obtain the text classification result includes:

[0027] The final vector representation of the sentence sequence is transformed into a linearly transformed sentence sequence vector through a series of linear transformations.

[0028] The linearly transformed sentence sequence vector is then used for classification prediction through a Softmax layer to obtain the text classification result.

[0029] Furthermore, to achieve the above objectives, the present invention also proposes a text classification device, the text classification device comprising:

[0030] The acquisition module is used to acquire the text sequence to be classified, and to extract local features from the text sequence to be classified by convolution at different scales to obtain multiple different feature subspaces;

[0031] The processing module is used to perform global attention calculation on each different feature subspace through the Attention mechanism to obtain the output features of each feature subspace.

[0032] The fusion module is used to fuse and classify the features output from each feature subspace to obtain the final vector representation of the sentence sequence;

[0033] The classification module is used to perform a linear transformation on the final vector representation of the sentence sequence to obtain the text classification result.

[0034] Furthermore, to achieve the above objectives, the present invention also proposes a text classification device, the device comprising: a memory, a processor, and a text classification program stored in the memory and running on the processor, the text classification program being configured to implement the text classification method as described above.

[0035] In addition, to achieve the above objectives, the present invention also proposes a storage medium storing a text classification program, which, when executed by a processor, implements the text classification method as described above.

[0036] This invention discloses a text classification method, apparatus, device, and storage medium. The method includes: acquiring a text sequence to be classified; extracting local features from the text sequence to be classified through convolutions of different scales to obtain multiple different feature subspaces; performing global attention calculation on each different feature subspace through an attention mechanism to obtain the features output by each feature subspace; fusing and classifying the features output by each feature subspace to obtain the final vector representation of the sentence sequence; and performing a linear transformation on the final vector representation of the sentence sequence to obtain the text classification result. This method combines CNN and attention mechanisms, fully utilizing the advantages of CNN in local feature extraction while the addition of the attention mechanism completely avoids the shortcomings of CNN in global dependency modeling, perfectly combining the characteristics of both to achieve accurate and simple text classification. Attached Figure Description

[0037] Figure 1 This is a schematic diagram of the hardware operating environment of the text classification device involved in the embodiments of the present invention;

[0038] Figure 2 This is a flowchart illustrating the first embodiment of the text classification method of the present invention;

[0039] Figure 3 This is a diagram illustrating the overall framework of a model for an embodiment of the text classification method of the present invention.

[0040] Figure 4 This is a schematic diagram of the input embedding in an embodiment of the text classification method of the present invention;

[0041] Figure 5 This is a flowchart illustrating the second embodiment of the text classification method of the present invention;

[0042] Figure 6 This is a diagram of the convolutional layer framework of an embodiment of the text classification method of the present invention;

[0043] Figure 7 This is a flowchart illustrating the third embodiment of the text classification method of the present invention;

[0044] Figure 8 This is a schematic diagram of the Attention layer in an embodiment of the text classification method of the present invention;

[0045] Figure 9 This is a schematic diagram of the functional modules of the first embodiment of the text classification device of the present invention.

[0046] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0047] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0048] Reference Figure 1 , Figure 1 This is a schematic diagram of the structure of a text classification device in the hardware operating environment involved in the embodiments of the present invention.

[0049] like Figure 1 As shown, the text classification device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen, and optionally, it may also include a standard wired interface or a wireless interface. In this invention, the wired interface of the user interface 1003 may be a USB interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wireless-Fidelity (Wi-Fi) interface). The memory 1005 may be a high-speed random access memory (RAM) or a non-volatile memory (NVM), such as a disk storage device. The memory 1005 may also optionally be a storage device independent of the aforementioned processor 1001.

[0050] Those skilled in the art will understand that Figure 1 The structure shown does not constitute a limitation on the text classification device and may include more or fewer components than illustrated, or combine certain components, or have different component arrangements.

[0051] like Figure 1 As shown, the memory 1005, which is identified as a computer storage medium, may include an operating system, a network communication module, a user interface module, and a text classification program.

[0052] exist Figure 1In the text classification device 4 shown, the network interface 1004 is mainly used to connect to the backend server and communicate data with the backend server; the user interface 1003 is mainly used to connect to the user equipment; the text classification device calls the text classification program stored in the memory 1005 through the processor 1001 and executes the text classification method provided in this embodiment of the invention.

[0053] Based on the above hardware structure, an embodiment of the text classification method of the present invention is proposed.

[0054] Reference Figure 2 , Figure 2 This is a flowchart illustrating the first embodiment of the text classification method of the present invention, which presents the first embodiment of the text classification method of the present invention.

[0055] In the first embodiment, the text classification method includes the following steps:

[0056] Step S10: Obtain the text sequence to be classified, and extract local features from the text sequence to be classified by convolution at different scales to obtain multiple different feature subspaces.

[0057] It should be understood that the execution subject of this embodiment is a text classification device, which has functions such as data processing, data communication and program execution.

[0058] In the specific implementation, before obtaining the text sequence to be classified and extracting local features from the text sequence to be classified through convolutions of different scales to obtain multiple different feature subspaces, the process further includes: scaling the initialized word vector weights according to a preset rule to obtain scaled weights, and summing the original sentence sequence according to the scaled weights to obtain a summed embedding vector; performing Dropout and LayerNorm processing on the summed embedding vector to obtain the text sequence to be classified. Extracting local features from the text sequence to be classified through convolutions of different scales to obtain multiple different feature subspaces includes: obtaining a sentence word vector matrix parallel to the layer layer based on the text sequence to be classified; and performing n-gram convolutions on the sentence word vector matrix parallel to the layer layer using convolution kernels of different scales to obtain different feature subspaces.

[0059] It should be noted that, as Figure 3The diagram shows the overall framework of a text classification method according to an embodiment of the present invention. This novel text classification model overcomes the shortcomings of recurrent neural networks (RNNs) in processing long sequences by combining CNNs and attention mechanisms. It fully leverages the advantages of CNNs in local feature extraction, while the addition of the attention mechanism completely avoids the shortcomings of CNNs in modeling global dependencies, perfectly combining the characteristics of both. This paper optimizes the feature extraction structure of the model. First, the model uses convolutional kernels of different scales to extract local features from the input sequence, transforming the original semantics into different feature subspaces. Then, the attention mechanism is applied in different feature subspaces to perform global attention calculation. Finally, the features output from each subspace are fused and classified.

[0060] This structure significantly enhances the model's feature extraction capabilities, effectively improving its performance and accuracy. Based on the idea of ​​local-to-global feature extraction, the model leverages the strengths of CNNs in local feature extraction and the attention mechanism in global dependency modeling, perfectly combining their advantages. The proposed new model is structurally simpler and has lower complexity. Experiments on various text classification datasets also demonstrate significant improvements in both convergence speed and accuracy.

[0061] Step S20: Perform global attention calculation on each different feature subspace using the Attention mechanism to obtain the features output by each feature subspace.

[0062] It should be understood that the weight coefficients of keywords in the classification process are adjusted according to the contribution of each word to the text features in different feature subspaces. Based on these weight coefficients, secondary feature calculations are performed on the local features extracted by convolution in a global manner to obtain the important features in different feature subspaces. The important features in the different feature subspaces are concatenated using Concat and multiplied by a mapping matrix to obtain the output features of each subspace.

[0063] In practice, the overall model mainly consists of an input layer for word vector embedding, a feature extraction layer composed of text convolution and attention calculations, a feature fusion layer for text representation, and a linear classification layer for output prediction. To obtain features across multiple scales, we designed the model in parallel, with each layer performing feature extraction in a different feature space.

[0064] Step S30: The features output from each feature subspace are fused and classified to obtain the final vector representation of the sentence sequence.

[0065] In a specific implementation, the features output from each subspace and the local features obtained through text convolution are residually concatenated to obtain residually concatenated word vectors; the residually concatenated word vectors are then averaged to obtain the final vector representation of the sentence sequence.

[0066] It should be noted that, as Figure 4 The diagram shown illustrates the input embedding of an embodiment of the text classification method of this invention. In the input embedding layer, the Input Embedding module mainly comprises two parts: Token Embedding and Positional Embedding. We adopt the currently more mainstream learnable embedding method. For a given word sequence, its input representation is the sum of the corresponding word embeddings and positional embeddings. Furthermore, we multiply the initial word vector weights by... Scaling is applied to improve convergence, and Dropout and LayerNorm are applied to the summed embedding vectors to address overfitting and gradient vanishing issues. Thus, the entire original sentence sequence, after word vector embedding and positional encoding, becomes the input to the feature extraction layer.

[0067] Step S40: Perform a linear transformation on the final vector representation of the sentence sequence to obtain the text classification result.

[0068] It should be understood that the CNN module and Attention module in the feature extraction layer learn a large number of local and global features by recognizing the correlations between words in the text. To enhance the model's generalization ability, we perform residual connections between the local features obtained from text convolution and the global features obtained from the Attention layer. The word vectors after residual connections are then averaged to obtain the final vector representation of the entire sentence sequence. Through this process, we obtain a higher-level feature representation of the given text sequence, which is then passed to subsequent fully connected layers for a series of linear transformations before being used for classification and prediction via a Softmax layer.

[0069] In a specific implementation, the final vector representation of the sentence sequence is transformed through a series of linear transformations to obtain a linearly transformed sentence sequence vector; the linearly transformed sentence sequence vector is then used for classification prediction through a Softmax layer to obtain the text classification result.

[0070] In this embodiment, a text sequence to be classified is obtained. Local features are extracted from the text sequence through convolutions of different scales to obtain multiple different feature subspaces. Global attention is calculated for each different feature subspace through an attention mechanism to obtain the features output by each feature subspace. The features output by each feature subspace are fused and classified to obtain the final vector representation of the sentence sequence. A linear transformation is performed on the final vector representation of the sentence sequence to obtain the text classification result. This approach combines CNN and attention mechanisms, fully utilizing the advantages of CNN in local feature extraction. At the same time, the addition of the attention mechanism allows the model to completely avoid the shortcomings of CNN in global dependency modeling, perfectly combining the characteristics of both to achieve accurate and simple text classification.

[0071] Reference Figure 5 , Figure 5 This is a flowchart illustrating the second embodiment of the text classification method of the present invention, based on the above. Figure 2 The first embodiment shown presents a second embodiment of the text classification method of the present invention.

[0072] In the second embodiment, step S10 includes:

[0073] Step S101: Based on the text sequence to be classified, obtain the sentence word vector matrix parallel to the layer.

[0074] It should be noted that, as Figure 6 The diagram shown is a convolutional layer framework diagram of an embodiment of a text classification method. The input of the convolutional layer is a word vector matrix. Here, we use convolutional kernels of different sizes to perform convolution operations on the word vector matrix to extract local features corresponding to different layers.

[0075] Step S102: Perform n-gram convolution on the sentence word vector matrix parallel to the layer using convolution kernels of different scales to obtain different feature subspaces.

[0076] It should be noted that the process by which CNN transforms the original sequence into different local feature spaces is as follows: Figure 6 As shown. The specific computation process of a CNN is as follows: For the sentence word vector matrix in each parallel layer, n-gram convolution is performed on the input matrix using convolution kernels of different scales, transforming the original sequence from the word space into a semantic feature space with more information, and the output matrix of the convolution layer is passed to the attention layer.

[0077] In this embodiment, based on the text sequence to be classified, a sentence word vector matrix parallel to the layer is obtained. This sentence word vector matrix is ​​then subjected to n-gram convolution with kernels of different scales to obtain different feature subspaces. Global attention is calculated for each of these feature subspaces using an attention mechanism to obtain the features output by each subspace. The features output by each subspace are then fused and classified to obtain the final vector representation of the sentence sequence. A linear transformation is performed on the final vector representation of the sentence sequence to obtain the text classification result. This approach combines the advantages of CNNs in local feature extraction and attention mechanisms in global relationship modeling, employing a parallel approach of first focusing on local features and then on global features to extract multi-dimensional features from the text, effectively improving the model's ability to extract rich semantics.

[0078] Reference Figure 7 , Figure 7 This is a flowchart illustrating the third embodiment of the text classification method of the present invention, based on the above. Figure 2 The first embodiment shown is followed by a third embodiment of the text classification method of the present invention.

[0079] In the third embodiment, step S20 includes:

[0080] Step S201: Perform Self-Attention processing multiple times on the different feature subspaces to capture important features in the different feature subspaces.

[0081] In practice, the weight coefficients of keywords in the classification process are adjusted according to the contribution of each word to the text features in different feature subspaces; based on the weight coefficients, secondary feature calculations are performed on the local features extracted by convolution in a global manner to obtain important features in different feature subspaces.

[0082] It's worth noting that the attention mechanism can adjust the weight coefficients of keywords in the classification process based on the different contributions of each word to the text features, thus achieving better results. In this model, the addition of attention allows the model to perform secondary feature calculation on the local features extracted by convolution in a global manner. The attention calculation layer mainly consists of two parts: a multi-head attention mechanism and a two-layer feedforward neural network. The most important part is the multi-head attention calculation.

[0083] Step S202: Concat the important features in the different feature subspaces and multiply them with a mapping matrix to obtain the features output by each subspace.

[0084] It should be noted that, as Figure 8 The diagram shows an example of an Attention layer in a text classification method. As can be seen from the structure, Multi-Head Attention essentially performs multiple (multi-head) Self-Attention operations on the same input word vector from multiple perspectives to capture important features in different word spaces, enhancing the model's ability to capture diverse information. The outputs of each Self-Attention operation are concatenated and multiplied by a mapping matrix (to compress the output matrix) to obtain the output of the entire Multi-Head Attention. Multi-head attention enriches the feature information and reduces the computational cost by transforming the original high-dimensional space into multiple low-dimensional spaces for separate computation.

[0085] In this embodiment, a text sequence to be classified is obtained. Local features are extracted from the text sequence using convolutions of different scales to obtain multiple different feature subspaces. Multiple Self-Attention processes are performed on these different feature subspaces to capture important features. The important features in the different feature subspaces are concatenated and multiplied by a mapping matrix to obtain the output features of each subspace. The features output from each feature subspace are fused and classified to obtain the final vector representation of the sentence sequence. A linear transformation is then performed on the final vector representation of the sentence sequence to obtain the text classification result. This leverages the advantages of convolutional neural networks in local feature extraction. The model uses convolutional kernels of different scales to extract local features from the embedded vectors and transforms them from the word space to different low-dimensional semantic feature spaces. Then, an attention mechanism is used in different feature spaces to extract global relationships. This structure gives the model a significant advantage in feature extraction.

[0086] Furthermore, embodiments of the present invention also propose a storage medium storing a text classification program, which, when executed by a processor, implements the steps of the text classification method described above.

[0087] Since this storage medium can adopt the technical solutions of all the above embodiments, it has at least the beneficial effects brought about by the technical solutions of the above embodiments, which will not be described in detail here.

[0088] Reference Figure 9 , Figure 9 This is a schematic diagram of the functional modules of the first embodiment of the text classification device of the present invention.

[0089] In a first embodiment of the text classification device of the present invention, the text classification device includes:

[0090] The acquisition module 10 is used to acquire the text sequence to be classified, and to extract local features from the text sequence to be classified by convolution at different scales to obtain multiple different feature subspaces.

[0091] Processing module 20 is used to perform global attention calculation on each different feature subspace through the Attention mechanism to obtain the output features of each feature subspace;

[0092] The fusion module 30 is used to fuse and classify the features output from each feature subspace to obtain the final vector representation of the sentence sequence;

[0093] The classification module 40 is used to perform a linear transformation on the final vector representation of the sentence sequence to obtain the text classification result.

[0094] In this embodiment, a text sequence to be classified is obtained. Local features are extracted from the text sequence through convolutions of different scales to obtain multiple different feature subspaces. Global attention is calculated for each different feature subspace through an attention mechanism to obtain the features output by each feature subspace. The features output by each feature subspace are fused and classified to obtain the final vector representation of the sentence sequence. A linear transformation is performed on the final vector representation of the sentence sequence to obtain the text classification result. This approach combines CNN and attention mechanisms, fully utilizing the advantages of CNN in local feature extraction. At the same time, the addition of the attention mechanism allows the model to completely avoid the shortcomings of CNN in global dependency modeling, perfectly combining the characteristics of both to achieve accurate and simple text classification.

[0095] In one embodiment, the processing module 20 is further configured to acquire the text sequence to be classified, and before extracting local features from the text sequence to be classified through convolutions of different scales to obtain multiple different feature subspaces, it further includes:

[0096] The initial word vector weights are scaled according to a preset rule to obtain scaled weights, and the original sentence sequence is summed according to the scaled weights to obtain the summed embedding vector.

[0097] The summed embedding vector is processed using Dropout and LayerNorm to obtain the text sequence to be classified.

[0098] In one embodiment, the processing module 20 is further configured to extract row-local features from the text sequence to be classified through convolutions of different scales, resulting in multiple different feature subspaces, including:

[0099] Based on the text sequence to be classified, a sentence word vector matrix parallel to the layer is obtained;

[0100] The sentence word vector matrix parallel to the layer is subjected to n-gram convolution with convolution kernels of different scales to obtain different feature subspaces.

[0101] In one embodiment, the processing module 20 is further configured to perform global attention calculation on each different feature subspace using an attention mechanism to obtain the features output by each feature subspace, including:

[0102] Multiple Self-Attention processes are performed on the different feature subspaces to capture important features in the different feature subspaces;

[0103] The important features in the different feature subspaces are concatenated using Concat, and the features output by each subspace are obtained by multiplying them with a mapping matrix.

[0104] In one embodiment, the processing module 20 is further configured to perform multiple Self-Attention processes on the different feature subspaces to capture important features in the different feature subspaces, including:

[0105] The weight coefficients of keywords in the classification process are adjusted based on the contribution of each word to the text features in different feature subspaces.

[0106] Based on the weight coefficients, secondary feature calculations are performed on the local features extracted by convolution in a global manner to obtain important features in different feature subspaces.

[0107] In one embodiment, the fusion module 30 is further configured to fuse and classify the features output from each feature subspace to obtain the final vector representation of the sentence sequence, and further includes:

[0108] The features output from each subspace and the local features obtained through text convolution are residually concatenated to obtain the word vectors after residual concatenation.

[0109] The word vectors after residual concatenation are averaged to obtain the final vector representation of the sentence sequence.

[0110] In one embodiment, the classification module 40 is further configured to perform a linear transformation on the final vector representation of the sentence sequence to obtain a text classification result, including:

[0111] The final vector representation of the sentence sequence is transformed into a linearly transformed sentence sequence vector through a series of linear transformations.

[0112] The linearly transformed sentence sequence vector is then used for classification prediction through a Softmax layer to obtain the text classification result.

[0113] Other embodiments or specific implementations of the text classification device of the present invention can refer to the above-described method embodiments, and therefore have at least all the beneficial effects brought about by the technical solutions of the above embodiments, which will not be repeated here.

[0114] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0115] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments. In the unit claims listing several devices, several of these devices may be embodied by the same hardware item. The use of the terms first, second, and third, etc., does not indicate any order and can be interpreted as names.

[0116] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as a read-only memory image (ROM) / random access memory (RAM), magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0117] The above are merely preferred embodiments of the present invention and do not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A text classification method, characterized in that, The method includes the following steps: A text sequence to be classified is obtained, and local features are extracted from the text sequence to be classified through convolution at different scales to obtain multiple different feature subspaces; Global attention is calculated for each different feature subspace through the attention mechanism to obtain the features output by each feature subspace. The features output from each feature subspace are fused and classified to obtain the final vector representation of the sentence sequence; A linear transformation is performed on the final vector representation of the sentence sequence to obtain the text classification result; The step of performing global attention calculation on each different feature subspace through an attention mechanism to obtain the output features of each feature subspace includes: Multiple Self-Attention processes are performed on the different feature subspaces to capture important features in the different feature subspaces; The important features in the different feature subspaces are concatenated using Concat, and the features output by each subspace are obtained by multiplying them with a mapping matrix. The step of performing multiple Self-Attention processes on the different feature subspaces to capture important features in the different feature subspaces includes: The weight coefficients of keywords in the classification process are adjusted based on the contribution of each word to the text features in different feature subspaces. Based on the weight coefficients, secondary feature calculations are performed on the local features extracted by convolution in a global manner to obtain important features in different feature subspaces; The step of fusing and classifying the features output from each feature subspace to obtain the final vector representation of the sentence sequence further includes: The features output from each subspace and the local features obtained through text convolution are residually concatenated to obtain the word vectors after residual concatenation. The word vectors after residual concatenation are averaged to obtain the final vector representation of the sentence sequence; The linear transformation of the final vector representation of the sentence sequence to obtain the text classification result includes: The final vector representation of the sentence sequence is transformed into a linearly transformed sentence sequence vector through a series of linear transformations. The linearly transformed sentence sequence vector is then used for classification prediction through a Softmax layer to obtain the text classification result.

2. The method as described in claim 1, characterized in that, Before obtaining the text sequence to be classified and extracting local features from the text sequence to be classified through convolutions of different scales to obtain multiple different feature subspaces, the method further includes: The initial word vector weights are scaled according to a preset rule to obtain scaled weights, and the original sentence sequence is summed according to the scaled weights to obtain the summed embedding vector. The summed embedding vector is processed using Dropout and LayerNorm to obtain the text sequence to be classified.

3. The method as described in claim 1, characterized in that, The text sequence to be classified is subjected to row-local feature extraction through convolution at different scales to obtain multiple different feature subspaces, including: Based on the text sequence to be classified, a sentence word vector matrix parallel to the layer is obtained; The sentence word vector matrix parallel to the layer is subjected to n-gram convolution with convolution kernels of different scales to obtain different feature subspaces.

4. A text classification device, characterized in that, The text classification device includes: The acquisition module is used to acquire the text sequence to be classified, and to extract local features from the text sequence to be classified by convolution at different scales to obtain multiple different feature subspaces; The processing module is used to perform global attention calculation on each different feature subspace through the Attention mechanism to obtain the output features of each feature subspace. The fusion module is used to fuse and classify the features output from each feature subspace to obtain the final vector representation of the sentence sequence; The classification module is used to perform a linear transformation on the final vector representation of the sentence sequence to obtain the text classification result; The step of performing global attention calculation on each different feature subspace through an attention mechanism to obtain the output features of each feature subspace includes: Multiple Self-Attention processes are performed on the different feature subspaces to capture important features in the different feature subspaces; The important features in the different feature subspaces are concatenated using Concat, and the features output by each subspace are obtained by multiplying them with a mapping matrix. The step of performing multiple Self-Attention processes on the different feature subspaces to capture important features in the different feature subspaces includes: The weight coefficients of keywords in the classification process are adjusted based on the contribution of each word to the text features in different feature subspaces. Based on the weight coefficients, secondary feature calculations are performed on the local features extracted by convolution in a global manner to obtain important features in different feature subspaces; The step of fusing and classifying the features output from each feature subspace to obtain the final vector representation of the sentence sequence further includes: The features output from each subspace and the local features obtained through text convolution are residually concatenated to obtain the word vectors after residual concatenation. The word vectors after residual concatenation are averaged to obtain the final vector representation of the sentence sequence; The linear transformation of the final vector representation of the sentence sequence to obtain the text classification result includes: The final vector representation of the sentence sequence is transformed into a linearly transformed sentence sequence vector through a series of linear transformations. The linearly transformed sentence sequence vector is then used for classification prediction through a Softmax layer to obtain the text classification result.

5. A text classification device, characterized in that, The text classification device includes a memory, a processor, and a text classification program stored in the memory and executable on the processor. When the text classification program is executed by the processor, it implements the text classification method as described in any one of claims 1 to 3.

6. A storage medium, characterized in that, The storage medium stores a text classification program, which, when executed by a processor, implements the text classification method as described in any one of claims 1 to 3.

Images