Text processing method and apparatus, electronic device, and storage medium

Abstract

The application discloses a text processing method and device, an electronic device and a storage medium. The text processing method can include: performing feature extraction on a text sequence obtained based on a text by using M convolution kernels to obtain M first feature maps; wherein the granularity of different convolution kernels is different; wherein M is a positive integer not less than 2; performing pooling processing on the M first feature maps respectively to obtain M second feature maps; encoding the M second feature maps to obtain a feature vector; and classifying the text based on the feature vector. In this way, feature extraction is performed by multiple convolution kernels, so that local key information of different sizes can be obtained, and the text classification based on the features extracted in this way has the effect of more accurate classification results.

Description

Technical Field

[0001] This application relates to the field of information technology, and in particular to a text processing method and apparatus, electronic device and storage medium. Background Technology

[0002] With the development of the internet and the widespread use of digital terminal devices, internet text data has become an important source of information for people. However, the unstructured nature of text data makes it difficult for people to understand and utilize the valuable information within it. Natural Language Processing (NLP) provides an effective solution to this problem. Through tasks such as text classification, useful information can be quickly and accurately extracted from massive amounts of data.

[0003] Related technologies have proposed a variety of text classification methods, such as text classification based on machine learning, text classification based on convolutional neural networks (CNNs), and text classification based on improved bidirectional long short memory networks (BiLSTMs).

[0004] However, machine learning-based classification relies heavily on manual feature engineering, which places very relaxed demands on the classification model. Yet, manual feature engineering is extremely complex, requiring relevant domain experience and exhibiting a high degree of subjectivity. When the quality of feature extraction is poor, the classification performance of this approach is severely impacted.

[0005] CNNs can capture local features in text and perform well in extracting features from short texts. However, when classifying long texts, it's necessary to deepen the CNN layers to learn features, which means increasing the number of model parameters. Increasing the number of model parameters leads to higher training time and can also cause the vanishing gradient phenomenon. Therefore, processing long texts often consumes a lot of time, but the model results are unsatisfactory.

[0006] While improved Long Short-Term Memory (LSTM) networks alleviate the problems of gradient explosion and gradient minimization to some extent, they suffer from high computational overhead. Summary of the Invention

[0007] This application provides text processing methods and apparatus, electronic devices and storage media to solve the problem of inaccurate text classification results.

[0008] A first aspect of this disclosure provides a text processing method, the method comprising:

[0009] Feature extraction is performed on the text sequence obtained from the text using M convolutional kernels to obtain M first feature maps; wherein the granularity of different convolutional kernels is different; wherein M is a positive integer not less than 2;

[0010] Pooling is performed on each of the M first feature maps to obtain M second feature maps;

[0011] Encode the M second feature maps to obtain feature vectors;

[0012] The text is classified based on the feature vector.

[0013] Based on the above scheme, the step of performing pooling processing on M first feature maps to obtain M second feature maps includes:

[0014] K-max pooling is performed on the m-th first feature map to obtain the m-th second feature map; wherein the second feature map is a sparse feature map; m is a positive integer less than or equal to M, and the second feature map has the same size as the first feature map.

[0015] Based on the above scheme, the encoding of the M second feature maps to obtain feature vectors includes:

[0016] By fusing M of the second feature maps, one third feature map is obtained;

[0017] The third feature map is encoded to obtain a feature vector.

[0018] Based on the above scheme, the process of fusing M second feature maps to obtain one third feature map includes:

[0019] The feature values ​​in the i-th row and j-th column of the M second feature maps are concatenated to form the feature vector in the i-th row and j-th column of the third feature map, where i is a positive integer less than or equal to 1; j is a positive integer less than or equal to J, where 1 is the total number of rows in the second feature map; and J is the total number of columns in the second feature map.

[0020] Based on the above scheme, the encoding of the M second feature maps to obtain feature vectors includes:

[0021] The third feature map is encoded using a bidirectional long short-term memory (LSSM) module to obtain the feature vector.

[0022] Based on the above scheme, the bidirectional long short-term memory (LSM) module includes: a forward LSM network and a backward LSM network;

[0023] The process of encoding the M third feature maps using a bidirectional long short memory network module to obtain the feature vector includes:

[0024] The third feature map is encoded from front to back using the forward memory network to obtain the first vector;

[0025] The third feature map is encoded from back to front using the backward memory network to obtain the second vector;

[0026] The first vector and the second vector are fused to obtain the encoded vector.

[0027] Based on the above scheme, classifying the text based on the feature vector includes:

[0028] The softmax module is used to process the concatenated vectors of the encoded vectors at different times to obtain the classification result of the text.

[0029] Based on the above scheme, the method further includes:

[0030] Convert the text into word vectors;

[0031] Generate the original sequence based on the word vectors;

[0032] Add one or more zero elements to both ends of the original sequence to obtain the text sequence.

[0033] A second aspect of this disclosure provides a text processing apparatus, the apparatus comprising:

[0034] The extraction module is used to extract features from the text sequence obtained based on the text using M convolutional kernels to obtain M first feature maps; wherein the granularity of different convolutional kernels is different; wherein M is a positive integer not less than 2;

[0035] The pooling module is used to perform pooling processing on the M first feature maps respectively to obtain M second feature maps;

[0036] The encoding module is used to encode the M second feature maps to obtain feature vectors;

[0037] A classification module is used to classify the text based on the feature vector.

[0038] Based on the above scheme, the pooling module is specifically used to perform K-max pooling on the m-th first feature map to obtain the m-th second feature map; wherein, the second feature map is a sparse feature map; m is a positive integer less than or equal to M, and the second feature map has the same size as the first feature map.

[0039] Based on the above scheme, the encoding module is specifically used to fuse M second feature maps to obtain one third feature map; and to encode the third feature map to obtain a feature vector.

[0040] Based on the above scheme, the encoding module is specifically used to concatenate the feature values ​​of the i-th row and j-th column of the M second feature maps to form the feature vector of the i-th row and j-th column of the third feature map, where i is a positive integer less than or equal to 1; j is a positive integer less than or equal to J, where 1 is the total number of rows of the second feature map; and J is the total number of columns of the second feature map.

[0041] Based on the above scheme, the encoding module is specifically used to encode one of the third feature maps using a bidirectional long short memory network module to obtain the feature vector.

[0042] Based on the above scheme, the bidirectional long short-term memory (LSM) module includes: a forward LSM network and a backward LSM network;

[0043] The encoding module is specifically used to encode the third feature map from front to back using the forward memory network to obtain a first vector; to encode the third feature map from back to front using the backward memory network to obtain a second vector; and to fuse the first vector and the second vector to obtain the encoded vector.

[0044] Based on the above scheme, the classification module is specifically used to process the concatenated vectors of the encoding vectors at different times using the softmax module to obtain the classification result of the text.

[0045] Based on the above scheme, the device further includes:

[0046] The conversion module is used to convert the text into word vectors;

[0047] The generation module is used to generate the original sequence based on the word vectors;

[0048] An add module is used to add one or more zero elements to both ends of the original sequence to obtain the text sequence.

[0049] A third aspect of this disclosure provides an electronic device, comprising:

[0050] Memory;

[0051] A processor, connected to the memory, is configured to implement the text processing method provided by any of the first or second aspects of the foregoing by executing computer-executable instructions stored in the memory.

[0052] A fourth aspect of this disclosure provides a computer storage medium, characterized in that the computer storage medium stores computer-executable instructions; after the computer-executable instructions are executed by a processor, they implement the text processing method provided by any of the technical solutions of the first or second aspect.

[0053] In this embodiment, multiple convolutional kernels of different granularities are used to extract features from the text sequence, thereby extracting M first feature maps. These M first feature maps can achieve feature extraction at different levels. Then, M second feature maps are combined for encoding and text classification, which can achieve more complete extraction of key information in the text features and improve classification accuracy. Attached Figure Description

[0054] Figure 1 A flowchart illustrating a text processing method provided in an embodiment of this application;

[0055] Figure 2 A flowchart illustrating a text processing method provided in an embodiment of this application;

[0056] Figure 3 A flowchart illustrating a text processing method provided in an embodiment of this application;

[0057] Figure 4 This is a schematic diagram of the network model used in a text processing method provided in an embodiment of this application;

[0058] Figure 5 This is a schematic diagram of the structure of a text processing device provided in an embodiment of this application;

[0059] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0060] To gain a more detailed understanding of the features and technical content of this application, the implementation of this application will be described in detail below with reference to the accompanying drawings. The accompanying drawings are for reference and illustration only and are not intended to limit this application.

[0061] like Figure 1 As shown, this disclosure provides an embodiment of...

[0062] A text processing method, characterized in that the method includes:

[0063] S110: Use M convolutional kernels to extract features from the text sequence obtained from the text, and obtain M first feature maps; wherein, the granularity of different convolutional kernels is different; wherein, M is a positive integer not less than 2;

[0064] S120: Perform pooling processing on each of the M first feature maps to obtain M second feature maps;

[0065] S130: Encode the M second feature maps to obtain feature vectors;

[0066] S140: Classify the text based on the feature vector.

[0067] The text processing method provided in this embodiment can be implemented using a deep learning network model. This deep learning network model can be a single neural network or a combination of multiple neural network modules.

[0068] For example, the deep learning model could be a Memorygraphconvolutional neural network (MGCNN)-Bidirectional long short memory network (BiLSTM) model. Of course, this is merely an example, and the specific implementation is not limited to this example.

[0069] This text can be in various natural languages.

[0070] The text includes, but is not limited to, at least one of the following:

[0071] Text converted from speech signals;

[0072] Text extracted from various network data, including but not limited to: social data from social media platforms, forum data from forums, and publishing data from information publishing platforms.

[0073] The above are merely examples of text sources; the specific implementation is not limited to the examples mentioned above.

[0074] The M convolutional kernels can be the convolutional sum of a CNN.

[0075] A CNN module is used to extract features from a text sequence. For example, the CNN module may have one or more convolutional layers, and the convolutional kernels of different convolutional layers may be different.

[0076] Different convolutions have different granularities, meaning the size of the convolution kernel varies. For example, some convolution kernels are 3x3 arrays; some are 4x4 arrays; and some are 2x2 arrays.

[0077] Thus, by performing convolution on the text sequence using a larger convolution kernel, the feature values ​​contained in the resulting first feature map can be used to extract the contextual information in the text.

[0078] Multiple first feature maps are obtained from text sequences based on convolutional kernels of different granularities. Some first feature maps extract local information of the text, while others may extract global information or key information that is biased towards the global context.

[0079] Thus, pooling is performed based on this first feature map to obtain M second features. Then, the M second feature maps are encoded to obtain feature vectors, which can be used for text classification to obtain more accurate classification results.

[0080] The pooling process for the first feature map can be either max pooling or average pooling. There are various specific pooling methods, not limited to the examples here.

[0081] Encoding the M second feature maps yields a feature vector. This feature vector can be a 01 digit string consisting of only 0 and 1 elements.

[0082] When classifying text based on feature vectors, the process may specifically include: outputting a probability value for each text type based on the feature vector; and finally obtaining the classification result of the text based on the highest probability value.

[0083] This text classification can be a binary or multi-class text classification.

[0084] For example, when determining whether a text contains sensitive information that should not be disseminated, the text is classified as a binary category, indicating whether it contains sensitive information or not. This sensitive information may include, but is not limited to, violent, pornographic, or privacy-related content.

[0085] For example, classifying texts by topic could include categories such as those related to daily life, work, education, and friendship.

[0086] In summary, in this embodiment, M convolutional kernels of different granularities are used for feature extraction to obtain key information involving different parts of the text. Based on this richer and more comprehensive key information covering different parts of the text, more accurate classification results can be obtained. This key information can be represented by the feature vector.

[0087] In some embodiments, S120 may include:

[0088] K-max pooling is performed on the m-th first feature map to obtain the m-th second feature map; wherein the second feature map is a sparse feature map; m is a positive integer less than or equal to M, and the second feature map has the same size as the first feature map.

[0089] In this embodiment, the pooling operation is K-max pooling. Therefore, when the pooling window moves, it selects the K maximum values ​​from the first feature map at the current position of the pooling window, instead of simply taking one maximum value or the average. Using K-max pooling, compared to average pooling or basic max pooling, retains more feature values ​​in the first feature map, thereby generating a second feature map that retains more information. It is worth noting that the specific value of K here can be a positive integer equal to or greater than 2.

[0090] In this embodiment of the disclosure, the second feature map obtained after the pooling operation has the same size as the first feature map. Furthermore, the second feature map is a sparse feature map, which can be a feature map where 0 elements are compared to non-zero elements. Here, 0 elements and non-zero elements are both feature values ​​in the second feature map.

[0091] For example, using K-max Pooling, we can extract the K maximum values ​​that each pooling window passes through in the first feature map, maintaining the positions of these K maximum values, and converting all other feature values ​​outside these K values ​​to 0. This transforms the second feature map into a sparse feature map. As a sparse feature map, the second feature map filters out unimportant details from the first feature map, achieving further abstraction of feature values. Furthermore, setting the feature values ​​corresponding to unimportant details to 0 simplifies the calculation of 0 elements in subsequent computations, thus greatly simplifying the calculation and reducing computational overhead.

[0092] In some embodiments, S130 may include:

[0093] S131: Fuse M of the second feature maps to obtain 1 of the third feature maps;

[0094] S132: Encode the third feature map to obtain a feature vector.

[0095] Before encoding the second feature map, the M second feature maps are fused. This fusion method involves concatenating the feature values ​​at corresponding positions of the M second feature maps. The resulting third feature map and the second feature map are still an I*J feature map. However, the element in the i-th row and j-th column of the third feature map is composed of M feature values.

[0096] Although this embodiment involves M second feature maps, since the second feature map processed by Kmax-Pooling is a sparse feature map containing more zero elements, it still has the characteristics of low computational cost and low computational overhead even when the amount of data to be processed increases.

[0097] In some embodiments, S130 may include:

[0098] The third feature map is encoded using a bidirectional long short-term memory (LSSM) module to obtain the feature vector.

[0099] For example, the bidirectional long short-term memory (LSM) module includes a forward LSM network and a backward LSM network.

[0100] The process of encoding the M third feature maps using a bidirectional long short memory network module to obtain the feature vector includes:

[0101] The third feature map is encoded from front to back using the forward memory network to obtain the first vector;

[0102] The third feature map is encoded from back to front using the backward memory network to obtain the second vector;

[0103] The first vector and the second vector are fused to obtain the encoded vector.

[0104] The fusion of the first and second vectors here involves concatenating the encoded values ​​at corresponding positions in the first and second vectors. For example, if both the first and second vectors have a length of S, the s-th encoded value of the first vector is concatenated with the s-th encoded value of the second vector to form the s-th encoded value of the encoded vector. The s-th element of the encoded vector has two elements, one of which is the s-th encoded value of the first vector, and the other is the s-th encoded value of the second vector.

[0105] When a bidirectional long short-term memory (LSSM) module encodes the third feature map, it associates different features from two directions to obtain feature vectors. This allows for more complete contextual information, enabling more accurate text classification.

[0106] In some embodiments, there may be multiple bidirectional long short-short memory (BSSM) modules, which can be stacked, meaning the output of one BSSM module serves as the input to the next. In this case, the output of the last BSSM module is used as the final encoded vector for text classification.

[0107] S140 may include:

[0108] The softmax module is used to process the concatenated vectors of the encoded vectors at different times to obtain the classification result of the text.

[0109] The softmax module can be the activation function corresponding to the network layer.

[0110] The softmax module is used to process the concatenated vector of all encoded vectors at different times to obtain the text classification result.

[0111] Using the softmax module to classify encoded vectors can reduce the gradient explosion and gradient vanishing problems caused by the increase in the number of network layers in multi-layer fully connected networks.

[0112] In some embodiments, such as Figure 3 As shown, the method further includes:

[0113] S101: Convert the text into word vectors;

[0114] S102: Generate the original sequence based on the word vectors;

[0115] S103: Add one or more zero elements to both ends of the original sequence to obtain the text sequence.

[0116] After converting the text into word vectors, for example, the Word2Vec tool is used to distribute the word vectors to obtain the input vectors for the deep learning model.

[0117] For example, the input vector is processed using the embedding layers of the neural network model to obtain the final vector that can be input into the CNN module.

[0118] In this embodiment of the disclosure, in order to reduce the neglect of information at both ends of the original sequence, zero elements are added to both ends of the original sequence. This increases the length of the original sequence without changing the meaning of the content expressed by the original sequence, thereby obtaining a text sequence that can be directly used by the convolutional layer for feature extraction.

[0119] This method can further improve the text classification results.

[0120] This embodiment proposes a method to utilize MGCNN to locate local features of different granularities in text, obtain key information in sentences, and generate feature maps.

[0121] Furthermore, K-max pooling is used during the generation of feature maps to preserve the number and positional relationships of features and filter out redundant elements, resulting in a sparse matrix for the feature maps.

[0122] The feature maps are further encoded using a stacked BiLSTM model to learn long-term dependencies between features, and a stacked LSTM approach is used to learn text vectors with richer features and a higher degree of abstraction, thereby improving the accuracy of classification.

[0123] like Figure 4 As shown, this embodiment consists of multiple modules, which include:

[0124] The module consists of word vector representation (EmbeddingLayers), CNN module (CNNLayers), BiLSTM module (RNNLayers), and softmax module (softmax Layers).

[0125] The following describes the implementation of each module's functions:

[0126] 1. Word vector representation module: text data preprocessing.

[0127] Specifically, the input text sequence is transformed into word vectors and fed into the network model. The Word2Vec tool is used to train a distributed representation of the word vectors. Let X = {x1, x2, ..., x...} n} represents the input text sequence, where x i For the vocabulary w i The word vector representation.

[0128] 2. CNN module: It uses convolutional kernels of different granularities to match language units of different granularities in the input text, and then uses K-max Pooling to sample and generate sparse feature maps.

[0129] Specifically, features of the text sequence X are extracted using M convolutional kernels. The convolutional kernel, also known as the receptive kernel (or convolutional window size), is set to m (a variable, with a value between 2 and 5), and the feature value is k. i It is obtained by convolving a convolution kernel with m consecutive word vectors in the text sequence X:

[0130] k i =tanh(u i *X'+b),

[0131] Among them, u i Represents the convolution kernel, X' = [x i ,...,x i+m-1 ] represents the concatenation of m consecutive word vectors, where b is the bias.

[0132] The convolutional kernel slides across the text sequence X to generate a feature map K, where K = [k1,...,k...]. n ].

[0133] To preserve text edge information, zeros are padded at both ends of the text sequence X before the convolution operation to ensure that the dimension of the feature map K after convolution is the same as the dimension of the text sequence X.

[0134] Next, K-maxPooling is used to sample the feature map. This not only preserves only the number and positional relationship of feature values ​​to prevent overfitting, but also constructs a sparse feature matrix, which greatly reduces the training time cost of the subsequent LSTM model.

[0135] Finally, the feature map is transposed and fed into the subsequent BiLSTM module.

[0136] A Recurrent Neural Network (RNN) module may include one or more BiLSTM modules: BiLSTM modules are used to encode feature maps (or feature matrices), which can learn long-term dependencies between features and extract more abstract feature vectors by using stacked BiLSTM models.

[0137] Specifically, in this step, the BiLSTM sequence length is n, which is equal to the length of the input sequence X and the feature map K. At time t, the input x of the BiLSTM unit... t x is the concatenation of the eigenvalues ​​at time t of M feature maps. t =[K 1t ,K 2t ,...,K lt ].

[0138] K it Let represent the feature value of the i-th feature map in dimension t, where i∈[1,l] and t∈[1,n].

[0139] Therefore, the cell state C at time t t for:

[0140] f t =σ(W f [h t-1 ,x t ]+b f ),

[0141] i t =σ(W i [h t-1 ,x t ]+b i ),

[0142] C t '=tanh(W C [h t-1 ,x t ]+b C ),

[0143] h t-1 For the output of the previous moment, C t-1 f represents the cell state at the previous moment. t W is the forget gate function, which determines what to discard from the previous cell state. f Let b be the weight matrix. f For bias, C t 'This is the current input content, W C As the weight, bC For bias, i t It is the input gate function, W i As the weight, b i For bias, by i t Determine the current input content C t How much content is added to cell state C? t .

[0144] The output of BiLSTM at time t is:

[0145] s t =σ(W s [h l-1 ,x l ]+b s ),

[0146] h t =s t *tanh(C t ),

[0147] h t-1 s represents the output from the previous time step. l W is the output gate control function, which determines the content output from the cell state. s Let b be the weight matrix. s It's a bias, C t Let t represent the cell state at time t. This process is repeated multiple times (2-4), where the output of the previous LSTM layer is used as the input to the next LSTM layer to extract more abstract feature vectors, ultimately yielding the final output vector o of the BiLSTM. f and o b Concatenate them to obtain the feature vector V = [o f ,o b ].

[0148] The softmax module concatenates feature vectors to predict the category of the input text.

[0149] Specifically, this module takes the feature vector V as input and the output is designed as a softmax layer:

[0150] y = softmax(U'*V+c'),

[0151] Where U' and c' are the weights and biases, respectively, directly using the softmax function for classification can avoid the gradient message phenomenon caused by increasing the number of network layers due to using multi-layer fully connected networks, while reducing network parameters and lowering the training time of the model.

[0152] This disclosure uses three metrics to measure the model: precision (P), recall (R), and F1 score. The F1 score is the average of P and R.

[0153] Specifically, accuracy P refers to the proportion of correctly identified entities out of the total number of entities:

[0154]

[0155] Recall rate R refers to the proportion of correctly identified entities out of the total number of entities:

[0156]

[0157] F1 is the harmonic mean of precision and recall:

[0158]

[0159] M represents the number of entities identified, N represents the total number of entities in the test set, and n represents the number of correctly identified entities. This number of entities can be the number of text entries.

[0160]

[0161] Table 1

[0162] Table 1 compares the evaluation metrics of the MGCNN-BiLSTM model with those of other models for text classification.

[0163] As can be seen from the comparison in Table 1, the model proposed in this disclosure has improved in terms of accuracy, recall and F1 score to a certain extent.

[0164] By using convolutional kernels of different granularities, language units of different granularities can be extracted from natural text sequences, ensuring the completeness of feature extraction.

[0165] K-max Pooling is used to sample the convolutional feature map, preserving the positional and quantitative relationships between features, while processing the feature map into a sparse matrix.

[0166] Using stacked BiLSTM layers in the Multilayer Perceptron (MLP) layer can further learn the intrinsic relationships between features and learn higher-level abstract features of the input text.

[0167] The deep learning model provided in this disclosure uses deep learning to automatically learn text features, avoiding reliance on complex manual feature engineering.

[0168] By changing the convolution kernel window of the CNN module in the deep learning model from a fixed value to a variety of different sizes, the feature extraction capability of the model can be improved.

[0169] K-max pooling is used in the pooling layer of the CNN module to preserve the position and quantity relationship of features, generate a sparse matrix to reduce computational burden and avoid overfitting.

[0170] By stacking BiLSTM models, more abstract text features can be extracted, thereby improving the accuracy of text classification.

[0171] Using a softmax layer instead of a traditional MLP classification model reduces the number of training parameters and network layers, thus preventing overfitting.

[0172] Using Dropout can prevent overfitting and train a more robust model.

[0173] like Figure 5 As shown, this disclosure provides a text processing apparatus, the apparatus comprising:

[0174] The extraction module 110 is used to extract features from the text sequence obtained based on the text using M convolutional kernels to obtain M first feature maps; wherein, the granularity of different convolutional kernels is different; wherein, M is a positive integer not less than 2;

[0175] Pooling module 120 is used to perform pooling processing on M first feature maps respectively to obtain M second feature maps;

[0176] Encoding module 130 is used to encode the M second feature maps to obtain feature vectors;

[0177] The classification module 140 is used to classify the text based on the feature vector. In some embodiments, the extraction module 110, the pooling module 120, the encoding module 130, and the classification module 140 can all be program modules, which, after being executed by a processor, can implement the functions of the above-mentioned modules.

[0178] In other embodiments, the extraction module 110, the pooling module 120, the encoding module 130, and the classification module 140 may all be hardware-software hybrid modules; the hardware-software hybrid modules include, but are not limited to, various programmable arrays; the field-programmable arrays include, but are not limited to, field-programmable arrays and / or complex programmable arrays.

[0179] In some embodiments, the extraction module 110, the pooling module 120, the encoding module 130, and the classification module 140 may all be pure hardware modules; the pure hardware modules include, but are not limited to, application-specific integrated circuits.

[0180] In some embodiments, the pooling module 120 is specifically used to perform K-max pooling on the m-th first feature map to obtain the m-th second feature map; wherein the second feature map is a sparse feature map; m is a positive integer less than or equal to M, and the second feature map has the same size as the first feature map.

[0181] In some embodiments, the encoding module 130 is specifically used to fuse M second feature maps to obtain one third feature map; and to encode the third feature map to obtain a feature vector.

[0182] In some embodiments, the encoding module 130 is specifically used to concatenate the feature values ​​of the i-th row and j-th column of the M second feature maps to form the feature vector of the i-th row and j-th column of the third feature map, wherein i is a positive integer less than or equal to 1; j is a positive integer less than or equal to J, wherein 1 is the total number of rows of the second feature map; and J is the total number of columns of the second feature map.

[0183] In some embodiments, the encoding module 130 is specifically used to encode one of the third feature maps using a bidirectional long short memory (LSSM) module to obtain the feature vector.

[0184] In some embodiments, the bidirectional long short-term memory (LSM) module includes: a forward LSM and a backward LSM;

[0185] The encoding module 130 is specifically used to encode the third feature map from front to back using the forward memory network to obtain a first vector; to encode the third feature map from back to front using the backward memory network to obtain a second vector; and to fuse the first vector and the second vector to obtain the encoded vector.

[0186] In some embodiments, the classification module 140 is specifically used to process the concatenated vectors of the encoding vectors at different times using the softmax module to obtain the classification result of the text.

[0187] In some embodiments, the apparatus further includes:

[0188] The conversion module is used to convert the text into word vectors;

[0189] The generation module is used to generate the original sequence based on the word vectors;

[0190] An add module is used to add one or more zero elements to both ends of the original sequence to obtain the text sequence.

[0191] like Figure 6 As shown, this disclosure provides an electronic device, characterized in that the electronic device includes:

[0192] Memory;

[0193] A processor, connected to the memory, is configured to implement the text processing method provided in any of the foregoing embodiments by executing computer-executable instructions stored in the memory, such as performing... Figures 1 to 3 Any text processing method shown.

[0194] The electronic device may be a terminal device and / or a server in a service platform.

[0195] like Figure 6 As shown, the electronic device may also include a network interface, which can be used to interact with peer devices via a network.

[0196] This disclosure provides a computer storage medium storing computer-executable instructions; when executed by a processor, these computer-executable instructions can implement the text processing method provided in any of the foregoing embodiments, for example, performing... Figures 1 to 3 Any text processing method shown.

[0197] The technical solutions described in the embodiments of this application can be combined arbitrarily without conflict.

[0198] In the several embodiments provided in this application, it should be understood that the disclosed methods and smart devices can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0199] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0200] In addition, each functional unit in the various embodiments of this application can be integrated into a second processing unit, or each unit can be a separate unit, or N or more units can be integrated into a unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0201] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A text processing method, characterized in that, The method includes: Feature extraction is performed on the text sequence obtained from the text using M convolutional kernels to obtain M first feature maps; wherein the granularity of different convolutional kernels is different; wherein M is a positive integer not less than 2; Pooling is performed on each of the M first feature maps to obtain M second feature maps; Encode the M second feature maps to obtain feature vectors; Based on the feature vectors, the text is classified; The step of pooling M of the first feature maps to obtain M of the second feature maps includes: K-max pooling is performed on the m-th first feature map to obtain the m-th second feature map; wherein the second feature map is a sparse feature map; m is a positive integer less than or equal to M, and the second feature map has the same size as the first feature map; The process of encoding the M second feature maps to obtain feature vectors includes: By fusing M second feature maps, one third feature map is obtained; The third feature map is encoded to obtain a feature vector; The fusion of M second feature maps to obtain one third feature map includes: The feature values ​​in the i-th row and j-th column of the M second feature maps are concatenated to form the feature vector in the i-th row and j-th column of the third feature map, where i is a positive integer less than or equal to 1; j is a positive integer less than or equal to J, where 1 is the total number of rows in the second feature map; and J is the total number of columns in the second feature map.

2. The method according to claim 1, characterized in that, The process of encoding the M second feature maps to obtain feature vectors includes: The third feature map is encoded using a bidirectional long short-term memory (LSSM) module to obtain the feature vector.

3. The method according to claim 2, characterized in that, The bidirectional long short-term memory (LSSM) module includes: a forward LSSM and a backward LSSM; The process of encoding the M third feature maps using a bidirectional long short-term memory (BSSM) module to obtain the feature vector includes: The third feature map is encoded in both forward and backward directions using a forward memory network to obtain a first vector; The third feature map is encoded from back to front using a backward memory network to obtain the second vector; The first vector and the second vector are fused to obtain the encoded vector.

4. The method according to claim 1, characterized in that, The classification of the text based on the feature vector includes: The softmax module is used to process the concatenated vector of the encoding vectors at different times to obtain the classification result of the text.

5. The method according to claim 1, characterized in that, The method further includes: Convert the text into word vectors; Generate the original sequence based on the word vectors; Add one or more zero elements to both ends of the original sequence to obtain the text sequence.

6. A text processing device, characterized in that, The device includes: The extraction module is used to extract features from the text sequence obtained based on the text using M convolutional kernels to obtain M first feature maps; wherein the granularity of different convolutional kernels is different; wherein M is a positive integer not less than 2; The pooling module is used to perform pooling processing on the M first feature maps respectively to obtain M second feature maps; The encoding module is used to encode the M second feature maps to obtain feature vectors; A classification module is used to classify the text based on the feature vector; The pooling module is specifically used to perform K-max pooling on the m-th first feature map to obtain the m-th second feature map; wherein the second feature map is a sparse feature map; m is a positive integer less than or equal to M, and the second feature map has the same size as the first feature map; The encoding module is specifically used to fuse M second feature maps to obtain one third feature map; and to encode the third feature map to obtain a feature vector. The encoding module is specifically used to concatenate the feature values ​​of the i-th row and j-th column of the M second feature maps to form the feature vector of the i-th row and j-th column of the third feature map, where i is a positive integer less than or equal to 1; j is a positive integer less than or equal to J, where 1 is the total number of rows of the second feature map; and J is the total number of columns of the second feature map.

7. The apparatus according to claim 6, characterized in that, The encoding module is specifically used to encode one of the third feature maps using a bidirectional long short memory network module to obtain the feature vector.

8. The apparatus according to claim 7, characterized in that, The bidirectional long short-term memory (LSSM) module includes: a forward LSSM and a backward LSSM; The encoding module is specifically used to encode the third feature map from front to back using a forward memory network to obtain a first vector; to encode the third feature map from back to front using a backward memory network to obtain a second vector; and to fuse the first vector and the second vector to obtain an encoded vector.

9. The apparatus according to claim 6, characterized in that, The classification module is specifically used to process the concatenated vector of the encoding vectors at different times using the softmax module to obtain the classification result of the text.

10. The apparatus according to claim 6, characterized in that, The device further includes: The conversion module is used to convert the text into word vectors; The generation module is used to generate the original sequence based on the word vectors; An add module is used to add one or more zero elements to both ends of the original sequence to obtain the text sequence.

11. An electronic device, characterized in that, The electronic device includes: Memory; A processor, connected to the memory, is configured to implement the method provided by any one of claims 1 to 5 or 6 to 10 by executing computer-executable instructions stored in the memory.

12. A computer storage medium, characterized in that, The computer storage medium stores computer-executable instructions; when executed by a processor, the computer-executable instructions can implement the method provided by any one of claims 1 to 5 or 6 to 10.

Images