Text classification method based on distributed machine learning

Abstract

The application discloses a text classification method based on distributed machine learning, comprising the following steps: S1, inputting a data set and performing pretreatment; S2, dividing the data set into blocks and proposing a distributed text data stream classification method of Word2Vec; S3, training a Word2Vec model, representing a word as a low-dimensional dense vector form in a continuous space, generating a word vector, and self-defining a weight matrix; S4, a master node broadcasts a classifier weight to a sub node, a data stream distributes data blocks to the sub node through a distributed file management system, the master node polls and waits for a processing result of the data block by the sub node, so as to obtain a sum of weight gradients, and the next data block is updated after processing; S5, initializing a CNN parameter, inputting to a convolutional neural network for parallel training, and constructing a text classification model based on distributed machine learning under a Spark platform. The application explores a convolutional neural network through the adaptability of an Apache Spark framework, fully utilizes the parallel advantage of the Spark framework, and improves the precision of text classification.

Description

Technical Field

[0001] This invention relates to a text classification method based on distributed machine learning, belonging to the field of text classification and processing technology. Background Technology

[0002] Currently, with the development of the information age, text data on the internet is experiencing explosive growth. This massive amount of unclassified data prevents efficient extraction of useful information and wastes network resources. Text classification, as a crucial text processing technique, is widely used in various natural language processing tasks. Text classification involves using specific algorithms to analyze the underlying rules within text data and then matching new text with corresponding categories.

[0003] For a long time, the mainstream method for text classification has been based on statistical machine learning models. These models typically use representation methods such as bag-of-words and n-grams to transform documents into N-dimensional vectors, which are then fed into classifiers such as Support Vector Machines (SVMs) and Naive Bayes to predict document labels. While statistical machine learning models are simple in structure and robust, they also have many problems. For example, the semantic representation of text in statistical methods is relatively superficial, requiring further extraction of deeper features from text vectors. This process necessitates manual design, addition, and combination. Furthermore, researchers, due to differences in experience and understanding of the problem, have varying interpretations of the relationships between words, sentences, and context in different texts during the construction of text feature representations. Consequently, the constructed text feature representations cannot represent the core semantics of the text, and simple machine learning classifiers cannot extract the semantic information from them.

[0004] In 2006, Hinton et al. discovered the powerful expressive capabilities of deep learning, which can extract sentence-level and word-level feature information from documents. They solved the semantic extraction problem, which was difficult to address by traditional machine learning, by constructing deep neural networks. As deep learning has become a popular research area, research and applications in natural language processing using deep learning frameworks have achieved certain results.

[0005] In 2014, Kim conducted research on text classification based on Convolutional Neural Networks (CNNs), including sentiment analysis and semantic analysis. Kim's proposed text classification model, TextCNN, consists of a convolutional layer (using multiple convolutional kernels of different sizes) and a max-pooling layer. Due to its simple structure and good performance, it has been widely used in the field of NLP. This method can effectively classify short texts, but for texts longer than 5000 characters, a single-layer CNN cannot fully extract the features of long texts.

[0006] Robert Keeling et al. applied TextCNN to legal text classification. Experiments showed that when the text length exceeded 2000 characters, TextCNN did not significantly improve classification accuracy compared to traditional neural networks. This is because CNNs can only process information from one convolutional kernel, and information from adjacent convolutional kernels can only be correlated through subsequent convolutional layers. The correlation of information depends on parameters such as the kernel size and stride. Therefore, when using CNNs for long text classification, the model cannot flexibly adjust parameters, leading to prolonged training time.

[0007] Compared to CNNs, Recurrent Neural Networks (RNNs) use a time-series structure to traverse text, considering not only the text itself but also the contextual information, thus achieving better text classification results. Long Short-Term Memory (LSTM) networks within RNNs, with their unique gate structure, allow information to selectively influence the state of the neural network at each time step, solving the dimensionality explosion problem caused by the excessive amount of information to be remembered in traditional sequence models. The TextRNN model proposed by Pengfei Liu et al. and the LSTMs end-to-end text training model proposed by Miwa M et al. can continuously retain information and remember the contextual relationships within the text, making better use of information that traditional neural networks cannot model. However, when the text length exceeds 10,000 characters, the memory requirements of sequence models increase continuously, and the dimensionality of the input data expands, easily leading to resource exhaustion, slow training, and a decrease in model classification accuracy.

[0008] In view of this, it is indeed necessary to propose a text classification method based on distributed machine learning to solve the above problems. Summary of the Invention

[0009] The purpose of this invention is to provide a text classification method based on distributed machine learning to solve the problems of slow training and low accuracy of existing text classification models when the text length is too long.

[0010] To achieve the above objectives, this invention provides a text classification method based on distributed machine learning, comprising the following steps:

[0011] S1. Input the dataset and preprocess it;

[0012] S2. Divide the dataset into blocks and propose a distributed text data stream classification method based on Word2Vec;

[0013] S3. Train the Word2Vec model to represent words as low-dimensional dense vectors in a continuous space, generate word vectors, and define a custom weight matrix.

[0014] S4. In a standard cluster consisting of a master node and three child nodes, the master node broadcasts the classifier weights to the child nodes. The data stream distributes the data blocks to the child nodes through a distributed file management system. The master node polls and waits for the processing results of the child nodes on the data blocks. Then, it obtains the sum of the weight gradients processed by each child node, updates the master node, and then processes the next data block until the data stream ends.

[0015] S5. Initialize the CNN parameters and input them into the convolutional neural network for parallel training. Build a text classification model based on distributed machine learning on the Spark platform to classify text.

[0016] As a further improvement of the present invention, S1 includes the following steps:

[0017] S11. Use a word segmentation system to segment the dataset; the word segmentation system includes Jieba, Han LP, Tsinghua Chinese Lexical Analysis, and THULAC.

[0018] S12. Use a stop word list to remove stop words from the dataset; common stop word lists include the Harbin Institute of Technology stop word list and the Baidu stop word list.

[0019] As a further improvement of the present invention, S2 includes the following steps:

[0020] S21. Divide the given data stream into fixed-size data blocks according to a fixed-length time window mechanism;

[0021] S22. In order to alleviate the sparse information and high-dimensional features in short text data streams, an extended Word2vec model is established using an external corpus, and a fixed-size N-dimensional word vector Vec is used to represent the short text.

[0022] S23. For data block D in the data stream at time t... t Vectorization is performed using a set of word vectors to obtain the vectorized representation Vec(D). t), and then in Vec(D t The model is trained on the first data block, and then trained on the next data block D. t+1 Make predictions;

[0023] S24. Predict the text.

[0024] As a further improvement of the present invention, S24 specifically refers to: using data block D at time t... t+1 Short texts in For example, the vectorization can be predicted using the following formula. text:

[0025]

[0026] in, For predicting labels; For the algorithm ensemble model F t The feature weights of the r-th classifier at time t are represented by σ(·), which is the corresponding activation function. Finally, the extended Word2vec word vector set VecE is updated using text context information.

[0027] As a further improvement of the present invention, S3 includes the following steps:

[0028] S31. Word2vec encodes words into vectors based on word co-occurrence information in the corpus, and calculates the distributed word vector of each word in a given corpus environment.

[0029] S32. Vectorization of short texts by extending the Word2vec model.

[0030] As a further improvement of the present invention, S4 includes the following steps:

[0031] S41. Machine learning algorithms typically iterate over an objective function until convergence, given data and a model. The specific formula is as follows:

[0032] M t =F(M t-1 ,Δ L (M t-1 ,D));

[0033] Where t represents the number of iterations, Δ L For the objective function, F is used to update the model;

[0034] S42. The data stream distributes all data blocks evenly to each child node through the HDFS distributed file system for distributed parameter training.

[0035] As a further improvement of the present invention, S41 specifically involves: when the data volume is large, the data needs to be segmented and run in parallel on multiple nodes. In this case, each working node uses local data for training and updates synchronously through a certain communication algorithm. The iterative formula at this time is:

[0036]

[0037] When the machine learning model is large and needs to be divided into parts for separate training on worker nodes, the iterative function can be expressed as:

[0038]

[0039] in, It is the model scheduling function; the objective function selects parameters on the sub-models trained at each node.

[0040] As a further improvement of the present invention, S42 specifically involves: the master node broadcasting the feature weights of the classifier Ft to all child nodes, and waiting for all child nodes to complete the processing of the short text d (d∈D). t+1 The system predicts the new words in the local short text and sends them back to the master node. The master node obtains the prediction information of all child nodes for the training data block and the set of new words in the data block, and outputs the prediction results.

[0041] As a further improvement of the present invention, S5 includes the following steps:

[0042] S51, convolutional layers, and pooling layers are the core components of CNN. Through convolution and pooling operations, the original features are mapped to a higher-dimensional semantic space.

[0043] S52. The pooling layer is responsible for sampling the information extracted by the convolutional layer and retaining some of the information, while reducing the parameters for the next layer's calculation and speeding up the model's training.

[0044] S53, the fully connected layer maps samples from the feature space to the label space.

[0045] As a further improvement of the present invention, S51 is specifically as follows: assuming that a certain text contains n words and the word vector dimension is k, then the input can be represented as an n×k dimensional text matrix X;

[0046] X = {x1, x2, ..., x} n} T Where, x i Let be the word vector of the i-th word;

[0047] By setting convolutional kernels of different lengths, CNNs can perform rich local feature extraction. Let the size of the convolutional kernel be h, and the weight matrix and bias be W∈R. h×k and b∈R;

[0048] Divide the input X into {x 1:h x 2:h+1 , ..., x i:i+h-1 , ..., x n-h+1:n Each convolution operation affects the information x within the sliding window at step i. i:i+h-1 ∈R h×k Feature extraction is performed, and the attribute value c is obtained. i ∈R, i∈[1, n-h+1], the calculation is as follows:

[0049]

[0050] Where f is a non-linear activation function, For convolution operators;

[0051] Convolutional feature maps can be described as:

[0052] C = {c1, c2, ..., c} n-h+1}

[0053] The beneficial effects of this invention are as follows: This invention proposes a text classification method based on distributed machine learning. It explores convolutional neural networks through the adaptability of the Apache Spark framework, making full use of the parallel advantages of the Spark framework to improve the accuracy and time performance of text classification. Through Word2vec distributed text data stream classification, it enables the construction of Word2vec word vector models from external corpora to complete the vectorization process of short texts. Attached Figure Description

[0054] Figure 1 This is a flowchart of the text classification method based on distributed machine learning in this invention.

[0055] Figure 2 This is a schematic diagram of distributed model prediction and the Word2vec framework. Detailed Implementation

[0056] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0057] Please see Figure 1 and Figure 2 As shown, this invention provides a text classification method based on distributed machine learning for text classification, specifically including the following steps:

[0058] S1. Input the dataset and preprocess it.

[0059] Among them, the preprocessing includes word segmentation and stop word removal for the dataset, specifically including the following steps:

[0060] S11. Use a word segmentation system to segment the dataset.

[0061] Common word segmentation systems include, but are not limited to, the Jieba word segmentation system, the Han LP word segmentation system, the Tsinghua Chinese Lexical Analysis, and the THULAC word segmentation system, etc.

[0062] Since there are many choices for arranging words into phrases in the Chinese language, it is difficult to identify them in the form of a dictionary, making it a major difficulty in Chinese word segmentation to correctly segment Chinese text information. Using the above common word segmentation systems can effectively segment the Chinese language.

[0063] S12. Use a stop word list to remove stop words from the dataset.

[0064] Common stop word lists include, but are not limited to, the Harbin Institute of Technology stop word list and the Baidu stop word list, etc.

[0065] Stop words refer to many meaningless words in Chinese text, such as some conjunctions, prepositions, adverbs, personal pronouns like "and", "of", "he", "le", etc. These words have no meaning in the text and play a negligible role in text classification, and these meaningless words need to be deleted to reduce text noise.

[0066] S2. Divide the dataset into chunks and propose a distributed text data stream classification method based on Word2Vec.

[0067] S21. According to the fixed-length time window mechanism, divide the given data stream into fixed-size data chunks. The data stream D is formulated as D = {D1, D2,..., D t ,..., D T}, where T → ∞, 1 ≤ t ≤ T, |D t | represents the number of short texts in D t , represents the nth text in the tth data chunk in the data stream, is the class label corresponding to the text.

[0068] S22. In order to alleviate the sparse information and high-dimensional features in the short text data stream, use an external corpus to build an extended Word2vec model, and use a fixed-size N-dimensional word vector Vec to represent the short text in vector form.

[0069] Among them, the nth short text in the t-th data block The text vector can be represented as The vectorized set of all short texts in the Dt data block is represented as Vec(Dt). To avoid over-reliance on external corpora leading to model degradation, the model maintains a new word set S to construct the Word2vec model to obtain an extended word vector set VecE.

[0070] S23. For data block D in the data stream at time t... t Vectorization is performed using a set of word vectors to obtain the vectorized representation Vec(D). t ), and then in Vec(D t The model is trained on the first data block, and then trained on the next data block D. t+1 Make predictions.

[0071] Specifically, to adapt the classifier model to the real-time changing data stream, the model constructs a classifier for each class label at time t, denoted as f. r (1≤r≤|R|,|R|≤L), where R is the set of class labels appearing at time t, resulting in an ensemble model of |R| classifiers.

[0072] S24. Predict the text.

[0073] Specifically, take data block D at time t as an example. t+1 Short texts in For example, the vectorization can be predicted using the following formula. text:

[0074]

[0075] in, For predicting labels; For the algorithm ensemble model F t The feature weights of the r-th classifier at time t are denoted by σ(·), which represents the corresponding activation function. Finally, the extended Word2vec word vector set VecE is updated using text context information.

[0076] S3. Train the Word2Vec model to represent words as low-dimensional dense vectors in a continuous space, generate word vectors, and define a custom weight matrix.

[0077] S31. Word2vec encodes words into vectors based on word co-occurrence information in the corpus, and calculates the distributed word vector of each word in a given corpus environment.

[0078] Specifically, Word2vec is a word embedding method that includes two training models: the Skip-gram model and the Continuous Bag of Words (CBOW) model.

[0079] The calculation method of the skip word model reverses the causal relationship of CBOW, using the word w(t) at the current position to predict the probability of words in the context, i.e., P(w(i)|w(t)), where tk≤i≤t+k and i≠k.

[0080] The continuous bag-of-words model uses the context or surrounding words to predict the probability of the word w(t) at the current position, i.e., P(w(t)|w(tk),w(t-(k-1)),…,w(t-1),w(t+1),w(t+2),…,w(t+k)). Specifically, the continuous bag-of-words model is a three-layer neural network.

[0081] S32. Vectorization of short texts by extending the Word2vec model.

[0082] Specifically, define data block D. t Short and medium texts Vectorized representation is denoted as: in, Representing short text Count of words in Vec(wd) i ) indicates short text The i-th word wd i The corresponding word vectors. Iterate through the loop to obtain the vectorized representations of all short texts in the data block Dt, Vec(D). t ).

[0083] S4. In a standard cluster consisting of a master node and three child nodes, the master node broadcasts the classifier weights to the child nodes. The data stream distributes the data blocks to the child nodes through a distributed file management system. The master node polls and waits for the processing results of the child nodes on the data blocks, then obtains the sum of the weight gradients processed by each child node, updates the master node, and then processes the next data block until the data stream ends.

[0084] Spark is a parallel programming model that runs on a cluster. It has a master node, the Driver, which handles Spark drivers and uses a cluster manager to send tasks to three Executors. The Driver is the Spark driver node, primarily responsible for scheduling tasks among the Executors. An Executor is a JVM process within a Spark program that runs the specific tasks in the Spark job and returns the results to the Driver.

[0085] To achieve a standard cluster, the master node (Driver) first broadcasts the classifier weights to all child nodes (Executors). Then, it polls and waits for each child node to finish processing its data block. After that, it obtains the sum of the weight gradients processed by each child node. The master node then updates its execution parameters online and repeats the above steps to process the next data block until the data stream ends.

[0086] In other words, each worker node has its own local training data, and the initial values ​​of each model can be different. The local model parameters are updated using the local training data. After the local model completes one round of parameter updates, all nodes, according to the corresponding topology and synchronization algorithm, begin to communicate on the local model parameters and aggregate to update the global parameters. Finally, each worker node obtains the new global model and prepares to enter the next round of iteration.

[0087] S41. Machine learning algorithms typically iterate over an objective function until convergence, given data and a model. The specific formula is as follows:

[0088] M t =F(M t-1 ,Δ L (M t-1 ,D));

[0089] Where t represents the number of iterations, Δ L Let F be the objective function, and then perform model updates.

[0090] In addition, there are at least two other scenarios that require parallel processing.

[0091] The first scenario is when the data volume is too large and needs to be split and run in parallel on multiple nodes. In this case, each worker node uses local data for training and updates synchronously through a certain communication algorithm. The iterative formula in this case is:

[0092]

[0093] The second scenario is when the machine learning model is so large that it needs to be divided into parts and trained separately on worker nodes. In this case, the iterative function can be expressed as:

[0094]

[0095] in, It is the model scheduling function; the objective function selects parameters on the sub-models trained at each node.

[0096] S42. The data stream distributes all data blocks evenly to each child node through the HDFS distributed file system for distributed parameter training.

[0097] Taking the data block Dt as an example, the master node first broadcasts the classifier weights to all child nodes, then polls and waits for the child nodes to process the data block to obtain the sum of the weight gradients processed by each child node, thus obtaining... Wherein, ΔW(D t ) represents the sum of the weight gradients.

[0098] ΔW'(D t Let be a |R|-dimensional vector, denoted as . in For classifier Following the corresponding weight gradient, the master node updates its parameters online and then processes the next data block in a loop, repeating the steps described above until the data stream ends.

[0099] The HDFS file system will store data block D t+1 Divide into three datasets, namely Based on the word vector set described earlier in Word2vec, the data blocks are then vectorized to obtain a vectorized representation of the short text. and

[0100] Then the master node broadcasts the feature weights of classifier Ft to all child nodes and waits for all child nodes to complete the processing of the short text d (d∈D). t+1 The system predicts the new words in the local short text and sends them back to the master node. The master node obtains the prediction information of all child nodes for the training data block and the set of new words in the data block, and outputs the prediction results.

[0101] S5. Initialize the CNN parameters and input them into the convolutional neural network for parallel training. Build a text classification model based on distributed machine learning on the Spark platform to classify texts. Here, sigmoid is the final activation function of the CNN.

[0102] S51, convolutional layers, and pooling layers are the core components of CNN. Through convolution and pooling operations, the original features are mapped to a higher-level semantic space.

[0103] Suppose a text contains n words and the word vector dimension is k, then the input can be represented as an n×k dimensional text matrix X, X = {x1, x2, ..., xk}. n} T Where, x i Let be the word vector of the i-th word.

[0104] By setting convolutional kernels of different lengths, CNNs can perform rich local feature extraction. Let the size of the convolutional kernel be h, and the weight matrix and bias be W∈R. h×k and b∈R.

[0105] Divide the input X into {x 1:h x 2:h+1 , ..., x i:i+h-1 , ..., x n-h+1:n Each convolution operation affects the information x within the sliding window at step i. i:i+h-1 ∈R h×k Feature extraction is performed, and the attribute value c is obtained. i ∈R, i∈[1, n-h+1], the calculation is as follows:

[0106]

[0107] Where f is a non-linear activation function, This is the convolution operator. A convolutional feature map can be described as:

[0108] C = {c1, c2, ..., c} n-h+1}

[0109] S52. The pooling layer is responsible for sampling the information extracted by the convolutional layer and retaining some of the information. At the same time, it reduces the parameters for the calculation of the next layer and speeds up the training of the model.

[0110] Among them, the pooling layer retains important information from the sampled information. For m convolutional kernels, the generated pooling feature map can be represented as:

[0111]

[0112] S53, the fully connected layer maps samples from the feature space to the label space.

[0113] The main parameter is the weight matrix W. f and bias b f The feature information obtained from the pooling layer is input into the fully connected layer, and then the normalized classification probability y is output through the softmax layer:

[0114]

[0115] Suppose that the d-dimensional vector is V, Vj Let V represent the j-th element. Its softmax value is calculated as follows:

[0116]

[0117] Through parameter sharing mechanisms, CNNs can handle high-dimensional data well, and no manual feature selection is required during the modeling process.

[0118] In addition to the simple structure mentioned above, for datasets with complex features, deep feature extraction networks can be constructed by repeatedly stacking convolutional and pooling layers.

[0119] In summary, this invention provides a text classification method based on distributed machine learning. By using a Spark cluster environment within a distributed framework to optimize text classification, it achieves both high efficiency and accuracy in text classification. Furthermore, by exploring convolutional neural networks through the adaptability of the Apache Spark framework, it fully leverages the parallel advantages of Spark, improving both the accuracy and time performance of text classification. Finally, through distributed text data stream classification using Word2vec, it enables the construction of Word2vec word vector models from external corpora to complete the vectorization process of short texts.

[0120] The above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A text classification method based on distributed machine learning, characterized in that, Includes the following steps: S1. Input the dataset and preprocess it; S2. Divide the dataset into blocks and classify them using the Word2Vec distributed text data stream classification method; including the following steps: S21. Divide the given data stream into fixed-size data blocks according to a fixed-length time window mechanism; S22. In order to alleviate the sparse information and high-dimensional features in short text data streams, an extended Word2vec model is established using an external corpus, and a fixed-size N-dimensional word vector Vec is used to represent the short text. S23. For data block D in the data stream at time t... t Vectorization is performed using a set of word vectors to obtain the vectorized representation Vec(D). t ), and then in Vec(D t The model is trained on the first data block, and then trained on the next data block D. t+1 Make predictions; S24. Predict the text; specifically: data block D at time t. t+1 The short text in the text is represented as The vectorization is predicted using the following formula. text: in, For predicting labels; For the algorithm ensemble model F t The feature weights of the r-th classifier at time t are represented by σ(·), which is the corresponding activation function. Finally, the extended Word2vec word vector set VecE is updated using text context information. S3. Train the Word2Vec model to represent words as low-dimensional dense vectors in a continuous space, generate word vectors, and define a custom weight matrix. S4. In a standard cluster consisting of a master node and three child nodes, the master node broadcasts the classifier weights to the child nodes. The data stream distributes the data blocks to the child nodes through a distributed file management system. The master node polls and waits for the processing results of the child nodes on the data blocks, then obtains the sum of the weight gradients processed by each child node, updates the master node, and then processes the next data block until the data stream ends; including the following steps: S41. Machine learning algorithms typically iterate over an objective function until convergence, given data and a model. The specific formula is as follows: M t =F(M t-1 ,Δ L (M t-1 ,D)); Where t represents the number of iterations, Δ L Let F be the objective function, and let F be the model update function. Specifically, when the data volume is large, the data needs to be split and run in parallel on multiple nodes. In this case, each working node uses local data for training and updates synchronously through a certain communication algorithm. The iterative formula at this time is: When the machine learning model is large and needs to be divided into parts for separate training on worker nodes, the iterative function can be expressed as: in, It is the model scheduling function; the objective function selects parameters on the sub-models trained at each node; S42. The data stream distributes all data blocks evenly to each child node through the HDFS distributed file system for distributed parameter training; specifically, the master node broadcasts the feature weights of classifier Ft to all child nodes and waits for all child nodes to complete the training of the short text d (d∈D). t+1 The system predicts the new words in the local short text and sends them back to the master node. The master node obtains the prediction information of all child nodes for the training data block and the set of new words in the data block, and outputs the prediction results. S5. Initialize the CNN parameters and input them into the convolutional neural network for parallel training. Build a text classification model based on distributed machine learning on the Spark platform to classify text.

2. The text classification method based on distributed machine learning according to claim 1, characterized in that, S1 includes the following steps: S11. Use a word segmentation system to segment the dataset; the word segmentation system includes Jieba, HanLP, Tsinghua Chinese Lexical Analysis, and THULAC. S12. Use a stop word list to remove stop words from the dataset; common stop word lists include the Harbin Institute of Technology stop word list and the Baidu stop word list.

3. The text classification method based on distributed machine learning according to claim 1, characterized in that, S3 includes the following steps: S31. Word2vec encodes words into vectors based on word co-occurrence information in the corpus, and calculates the distributed word vector of each word in a given corpus environment. S32. Vectorization of short texts by extending the Word2vec model.

4. The text classification method based on distributed machine learning according to claim 1, characterized in that, S5 includes the following steps: S51, convolutional layers, and pooling layers are the core components of CNN. Through convolution and pooling operations, the original features are mapped to a higher-dimensional semantic space. S52. The pooling layer is responsible for sampling the information extracted by the convolutional layer and retaining some of the information, while reducing the parameters for the next layer's calculation and speeding up the model's training. S53, the fully connected layer maps samples from the feature space to the label space.

5. The text classification method based on distributed machine learning according to claim 4, characterized in that, S51 specifically means: If a text contains n words and the word vector dimension is k, then the input can be represented as an n×k dimensional text matrix X; X = {x1, x2, ..., x} n } T Where, x i Let be the word vector of the i-th word; By setting convolutional kernels of different lengths, CNNs can perform rich local feature extraction. Let the size of the convolutional kernel be h, and the weight matrix and bias be W∈R. h×k and b∈R; Divide the input X into {x 1:h x 2:h+1 , ..., x i:i+h-1 , ..., x n-h+1:n Each convolution operation affects the information x within the sliding window at step i. i:i+h-1 ∈R h×k Feature extraction is performed, and the attribute value c is obtained. i ∈R, i∈[1, n-h+1], the calculation is as follows: Where f is a non-linear activation function, For convolution operators; Convolutional feature maps can be described as: C={c1,c2,...,c n-h+1 }。

Images