Text classification method and device, electronic equipment and storage medium

Abstract

The application discloses a text classification method and device, electronic equipment and a storage medium. The method comprises the following steps: obtaining a first text; preprocessing the first text to obtain a first feature representing the semantic information of a word vector in the first text; performing offline training on the first feature to generate a center agent text corresponding to different categories of features in the first feature; performing supervised learning on the center agent text corresponding to different categories of features in the first feature to obtain a word shift distance between a first center agent text and a second center agent text; inputting the word shift distance into a preset kernel function classification model to obtain a classification result of the first center agent text and the second center agent text; and determining a category label of the first text according to the classification results of a plurality of first center agent texts and a plurality of second center agent texts.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a text classification method, apparatus, electronic device, and storage medium. Background Technology

[0002] Currently, a vast amount of agricultural information exists online. How to effectively integrate this massive amount of agricultural information for the use of farmers has become a focus of attention. Agricultural text classification can effectively integrate information for application in scenarios such as information retrieval and agricultural recommendations.

[0003] Machine learning (ML) is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence; its applications span all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and instructional learning.

[0004] In Chinese text, characters are the independent basic units that constitute words, and semantic information of words can be inferred from the characters that make up a word. However, current methods for classifying Chinese text typically use word vectors or character vectors as the basic units for constructing text features. Character vectors lose information about Chinese words, while word vectors lack information in the character dimension. Furthermore, existing techniques for calculating inter-text distance often use linear methods, such as Euclidean distance, which are insufficient for capturing textual characteristics. Using distance metrics like Wasserstein distance, on the other hand, suffers from computational inefficiency. Moreover, combining text distance for classification often requires the use of linear algorithms like KNN, which struggles to handle non-linear data classification, resulting in low accuracy. Summary of the Invention

[0005] To address the related technical issues, embodiments of this application provide a text classification method, apparatus, electronic device, and storage medium.

[0006] The technical solution of this application embodiment is implemented as follows:

[0007] This application provides a text classification method, including:

[0008] Get the first text;

[0009] The first text is preprocessed to obtain the first feature representing the semantic information of word vectors in the first text;

[0010] The first feature is trained offline to generate the central proxy text corresponding to the different categories of features in the first feature;

[0011] Supervised learning is performed on the central proxy texts corresponding to different categories of features in the first feature to obtain the word shift distance between the first central proxy text and the second central proxy text; the first central proxy text is the central proxy text corresponding to the first category feature in the different categories of features; the second central proxy text is the central proxy text corresponding to any category feature in the different categories of features other than the first category feature; the first category feature is any category feature in the different categories of features;

[0012] The word shift distance is input into a preset kernel function classification model to obtain the classification results of the first center proxy text and the second center proxy text;

[0013] The category label of the first text is determined based on the classification results of multiple first-center proxy texts and multiple second-center proxy texts.

[0014] In the above scheme, the preprocessing of the first text to obtain the first feature representing the semantic information of word vectors in the first text includes:

[0015] The first text is pre-trained to obtain at least one word vector in the first text and a first weight corresponding to each word vector in the at least one word vector;

[0016] Based on the semantic information of each word vector, determine the set of word vectors that have semantic relationships;

[0017] The first feature is determined based on the word vector set and the first weight corresponding to each word vector in the word vector set.

[0018] In the above scheme, determining the first feature based on the word vector set and the weight corresponding to each word vector in the word vector set includes:

[0019] The second feature of each word vector in the word vector set is obtained by averaging each word vector in the word vector set according to the first weight corresponding to each word vector in the word vector set;

[0020] The third feature corresponding to the largest word vector value is determined based on the word vector set and the weight corresponding to each word vector in the word vector set.

[0021] The first feature is determined based on the second feature and the third feature corresponding to the largest word vector value.

[0022] In the above scheme, the step of performing supervised learning on the central proxy texts corresponding to different categories of features in the first feature to obtain the word shift distance between the first central proxy text and the second central proxy text includes:

[0023] The preset algorithm is initialized based on the first feature to obtain the second weight corresponding to the first feature;

[0024] The central proxy texts corresponding to different categories of features in the first feature are input into the preset algorithm to update the second weights corresponding to the first feature, thereby obtaining the third weights of the central proxy texts corresponding to different categories of features in the first feature.

[0025] The third weights of the central proxy texts corresponding to the different categories of features in the first feature determine the word shift distance between the first central proxy text and the second central proxy text.

[0026] The method in the above scheme further includes:

[0027] The kernel function matrix is ​​obtained based on the word shift distance;

[0028] Based on the improved vector machine (SVM) algorithm using the kernel function matrix, the preset kernel function classification model is obtained.

[0029] In the above scheme, the step of inputting the word shift distance into a preset kernel function classification model to obtain the classification results of the first center proxy text and the second center proxy text includes:

[0030] The word shift distance is input into a preset kernel function classification model to obtain the classification plane function of the first central proxy text and the second central proxy text;

[0031] Determine whether the value of the classification plane function is greater than or equal to a preset threshold;

[0032] If the value of the classification plane function is greater than or equal to the preset threshold, the first category label of the first central agent text is determined;

[0033] If the value of the classification plane function is less than the preset threshold, the second category label of the second central agent text is determined.

[0034] In the above scheme, obtaining the classification result of the first text based on the classification results of multiple first-center proxy texts and multiple second-center proxy texts includes:

[0035] Obtain multiple first category tags corresponding to multiple first center agent texts and multiple second category tags corresponding to multiple second center agent texts;

[0036] The category label of the first text is determined based on the plurality of first category labels and the plurality of second category labels.

[0037] This application also provides a text classification device, including:

[0038] The acquisition unit is used to acquire the first text;

[0039] The preprocessing unit is used to preprocess the first text to obtain a first feature representing the semantic information of word vectors in the first text;

[0040] The training unit is used to perform offline training on the first feature and generate the central proxy text corresponding to the different categories of features in the first feature.

[0041] The learning unit is used to perform supervised learning on the central proxy texts corresponding to different categories of features in the first feature, and to obtain the word shift distance between the first central proxy text and the second central proxy text; the first central proxy text is the central proxy text corresponding to the first category feature in the different categories of features; the second central proxy text is the central proxy text corresponding to any category feature in the different categories of features other than the first category feature; the first category feature is any category feature in the different categories of features;

[0042] The unit is used to input the word shift distance into a preset kernel function classification model to obtain the classification results of the first center proxy text and the second center proxy text;

[0043] The determining unit is used to determine the category label of the first text based on the classification results of multiple first central agent texts and multiple second central agent texts.

[0044] This application also provides an electronic device, including:

[0045] Memory, used to store executable instructions;

[0046] A processor, when executing executable instructions stored in the memory, implements any step of the method described above.

[0047] This application also provides a computer-readable storage medium storing executable instructions for implementing any step of the method described above when executed by a processor.

[0048] The text classification method, apparatus, electronic device, and storage medium provided in this application include: acquiring a first text; preprocessing the first text to obtain a first feature representing semantic information of word vectors in the first text; offline training of the first feature to generate central proxy texts corresponding to different categories of features in the first feature; supervised learning of the central proxy texts corresponding to different categories of features in the first feature to obtain the word shift distance between the first central proxy text and the second central proxy text; wherein the first central proxy text is the central proxy text corresponding to a first category feature among the different categories of features; the second central proxy text is the central proxy text corresponding to any category feature among the different categories of features other than the first category feature; the first category feature is any category feature among the different categories of features; and inputting the word shift distance into a preset kernel function classification model to obtain the first central proxy text and the second central proxy text. The classification results are obtained; the category label of the first text is determined based on the classification results of multiple first central proxy texts and multiple second central proxy texts. The scheme of this application preprocesses the first text to obtain the first feature representing the semantic information of word vectors in the first text, which fully considers the semantic information before and after the word vectors, that is, the similarity between phrases and sentences; the first feature is trained offline to generate central proxy texts corresponding to different categories of features in the first feature; supervised learning is performed on the central proxy texts corresponding to different categories of features in the first feature to obtain the word shift distance between the first central proxy text and the second central proxy text, which fully considers the semantic sparsity transfer of the text and reduces the text training time overhead; by inputting the word shift distance into a preset kernel function classification model, the classification results of the first central proxy text and the second central proxy text are obtained, which effectively solves the nonlinear classification, better fits high-dimensional data, and improves the text classification accuracy. Attached Figure Description

[0049] Figure 1 This is a schematic diagram of the GANs network structure in the embodiments of this application;

[0050] Figure 2 This is a schematic diagram illustrating a text classification method flow provided in an embodiment of this application;

[0051] Figure 3 This is a schematic diagram illustrating the generation of a new node in an embodiment of the present invention;

[0052] Figure 4 This is a schematic diagram of the SWMD algorithm and MP-SWMD in the embodiments of this application;

[0053] Figure 5 This is a flowchart illustrating one of the text classification methods used in this application;

[0054] Figure 6 This is a schematic diagram of the overall model structure of an embodiment of this application;

[0055] Figure 7 This is a schematic diagram of a text classification device according to an embodiment of this application;

[0056] Figure 8 This is a schematic diagram of the hardware entity structure of an electronic device in an embodiment of this application. Detailed Implementation

[0057] Before providing a further detailed description of the embodiments of this application, the nouns and terms involved in the embodiments of this application will be explained, and the nouns and terms involved in the embodiments of this application shall be interpreted as follows.

[0058] 1) Supervised learning

[0059] Supervised learning refers to the machine learning task of inferring functions from labeled training datasets, that is, learning by utilizing label information when inputting data.

[0060] 2) Supervised Wasserstein distance

[0061] Wasserstein distance, also known as earthmoving distance, word movement distance, or Earth Mover's Distance (EMD), is primarily used to measure the distance between two probability distributions within a given region. Supervised Wasserstein distance mainly incorporates text label information as training features, and the weight vector b for each word is redefined as... After explicitly adding the weighted learning vector w and the learning matrix A, the original Wasserstein distance becomes as shown in the following form (1):

[0062]

[0063] In equation (1), Let μ be the supervised Wasserstein distance between text μ and text v;

[0064] μ and v represent two texts: text μ and text v.

[0065] T ij Let be the transition probability from word i to word j;

[0066] A is the learning matrix;

[0067] x i This represents the embedding of the word "i".

[0068] This represents the weight of word i, which is added to the text label;

[0069] n represents the number of unique words in the text μ after deduplication;

[0070] m represents the number of unique words in text v after deduplication;

[0071] To address this issue, by combining the gradient update of the loss function with nearest neighbor component analysis, we can obtain the final gradient update formula and thus obtain the Wasserstein distance.

[0072] 3) Word embedding

[0073] Word vectors are a mathematical representation that maps words or phrases to high-dimensional dense vectors. For similar words, their corresponding word vectors are also close in distance.

[0074] 4) Nearest Neighbor Component Analysis (NCA) and Proxy-NCA

[0075] Nearest Neighbor Component Analysis (NCA) is a supervised learning method that measures sample data according to a given distance metric algorithm and then classifies multivariate data. Assume there are s samples x1, x2, ..., xn. s exist In space, each x i Let y1, y2, ..., y be an N-dimensional eigenvector. s Representing the corresponding category label, defining sample x j For sample x i The probability of its nearest neighbor is p. ij Its formula is p ii =0. Define x i The probability of being correctly classified is p. i Its formula is The loss function can then be obtained as follows: The corresponding matrix L is obtained by minimizing the loss function, as shown in the equation. This is the distance between the two samples.

[0076] The Proxy-NCA algorithm mainly improves upon NCA. It combines the concepts of triples (x, y, Z) and KNN, iteratively clustering samples of the same category during training and separating samples of different categories. Specifically, it learns a small dataset P (|P| << |D|), which serves as an approximation of the original data points, satisfying... st d(x, p) < ε, define p(x) = argmin p∈Pd(x, p) is called a proxy; the proxy data can effectively represent the original data. The loss function is defined as:

[0077]

[0078] Where p(y) = p(x) are the proxies for samples of the same class, and p(Z) is the set of all negative sample proxies.

[0079] 5) Online k-means clustering

[0080] The online k-means clustering algorithm is a gradient-updated algorithm for solving cluster centers. Its main idea is to set the number of clusters k, randomly select a small batch of data from the dataset in each training iteration, update the center points according to the formula c←(1-η)c+ηx, and assign the data to the corresponding clusters. This process is repeated iteratively until the maximum number of iterations is reached, thereby calculating the cluster centers and their corresponding clusters.

[0081] 6) Difficult Sample Mining

[0082] Hard sample mining can be understood as dynamically assigning a weight to each sample pair during the learning process. Different weights are given to different sample pairs when learning from them. If a sample pair contains more information or is more difficult to learn, then it needs a larger weight. Sample pairs with less information are given smaller weights. If some sample pairs are assigned a weight of 0, it means that these sample pairs are not considered when calculating the gradient, which is equivalent to discarding them.

[0083] 7) Support Vector Machine (SVM)

[0084] Support Vector Machine (SVM) is a generalized linear classifier that performs binary classification on data using supervised learning. Its decision boundary is the hyperplane with the maximum margin calculated from the training samples. If samples are linearly inseparable in the feature space, a kernel function is needed to map them to a higher-dimensional space, making them linearly separable in that space. The kernel function is the high-dimensional result obtained by first mapping and then performing an inner product using a low-dimensional function. The introduction of the kernel function effectively avoids dimensionality explosion. Generally, this application defines the kernel function as K(x) i x j If ), then the classification function of SVM can be written as: Where, α j b are the parameters to be solved, and y j For sample x j The tag is . Commonly used kernel functions include polynomial kernel function, Gaussian kernel function, string kernel function, etc.

[0085] 8) GANs network model

[0086] In 2014, Ian J. Goodfellow et al. proposed Generative Adversarial Networks (GANs) to evaluate the performance of generative models. GANs mainly consist of two parts: a generative model and a discriminative model. The generative model simulates the distribution of real data, while the discriminative model determines whether a sample is real or generated. The goal of GANs is to train a generative model that perfectly fits the distribution of real data, making it indistinguishable from the real data by the discriminative model. For better understanding, we can combine... Figure 1 To understand, Figure 1 This is a schematic diagram of the GANs network structure in the embodiments of this application.

[0087] The present application will now be described in further detail with reference to the accompanying drawings and embodiments.

[0088] In related technologies, a vast amount of agricultural information exists online. How to effectively integrate this massive amount of agricultural information for the use of farmers has become a focus of attention. Agricultural text classification can effectively integrate information for application in scenarios such as information retrieval and agricultural recommendations.

[0089] Machine Learning (ML) is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence; its applications span all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and instructional learning.

[0090] In Chinese text, characters are the independent basic units that constitute words, and semantic information of words can be inferred from the characters that make up a word. However, current methods for classifying Chinese text typically use word vectors or character vectors as the basic units for constructing text features. Character vectors lose information about Chinese words, while word vectors lack information in the character dimension. Furthermore, related techniques for calculating inter-text distances often use linear methods, such as Euclidean distance, which are insufficient for capturing textual characteristics. Using Wasserstein distance as a metric, on the other hand, suffers from computational inefficiency. Moreover, combining text distance for classification often requires the use of linear algorithms like KNN, which struggles to handle non-linear data classification, resulting in low accuracy.

[0091] The relevant technologies have the following problems:

[0092] 1) After obtaining the text word vector matrix, the relevant technologies only use the word vector results to calculate the similarity attribute, without fully considering the similarity between phrases and sentences. For example, the phrase "take into consideration" and the phrase "inlight of" do not have a high degree of similarity between words, but they have the same meaning.

[0093] 2) When combining related technologies with distance calculation to update the gradient function, the Wasserstein distance and nearest neighbor component analysis (NCA) are combined to obtain the gradient update function. However, since all data is used for training, there is a problem of low computational efficiency, and the problem of sparse transfer of text semantics is not taken into account.

[0094] 3) When related technologies combine distance for text classification tasks, they often use a threshold method for classification. The fuzzy classification criteria affect the classification effect. Common distance-based classification algorithms often use the K-Nearest Neighbors (KNN) algorithm, which cannot fit high-dimensional data well.

[0095] Based on this, embodiments of this application provide a text classification method applied to an electronic device. The function implemented by this method can be achieved by a processor in the electronic device calling program code. Of course, the program code can be stored in a computer storage medium. Therefore, the electronic device includes at least a processor and a storage medium. As an example, the electronic device can be a mobile phone, computer, terminal, information transceiver, tablet device, personal digital assistant, etc.

[0096] Figure 2 This application provides a schematic diagram of a text classification method flow; as shown in the embodiments. Figure 2 As shown, the method includes:

[0097] Step 201: Obtain the first text;

[0098] Step 202: Preprocess the first text to obtain the first feature representing the semantic information of word vectors in the first text;

[0099] Step 203: Perform offline training on the first feature to generate the central proxy text corresponding to the different categories of features in the first feature;

[0100] Step 204: Perform supervised learning on the central proxy texts corresponding to the different categories of features in the first feature, and obtain the word shift distance between the first central proxy text and the second central proxy text; the first central proxy text is the central proxy text corresponding to the first category feature in the different categories of features; the second central proxy text is the central proxy text corresponding to any category feature in the different categories of features other than the first category feature; the first category feature is any category feature in the different categories of features;

[0101] Step 205: Input the word shift distance into a preset kernel function classification model to obtain the classification results of the first center proxy text and the second center proxy text;

[0102] Step 206: Determine the category label of the first text based on the classification results of multiple first central agent texts and multiple second central agent texts.

[0103] In practical applications, the specific content of the first text in step 201 can be determined according to the actual situation, and is not limited here. The first text can be agricultural text or agricultural input text.

[0104] In step 202, the preprocessing can be determined according to the actual situation and is not limited here. As an example, the preprocessing can be pre-training and / or pre-setting processing. The pre-training processing can specifically be web crawler pre-training processing; the pre-setting processing can specifically be word vector accumulation processing, for example, setting the size of the word vector accumulation window to m and connecting channels p; wherein, the specific values ​​of m and p can be determined according to the actual situation.

[0105] In one embodiment, the preprocessing of the first text to obtain a first feature representing the semantic information of word vectors in the first text includes:

[0106] The first text is pre-trained to obtain at least one word vector in the first text and a first weight corresponding to each word vector in the at least one word vector;

[0107] Based on the semantic information of each word vector, determine the set of word vectors that have semantic relationships;

[0108] The first feature is determined based on the word vector set and the first weight corresponding to each word vector in the word vector set.

[0109] The pre-training can be determined based on the actual situation and is not limited here. As an example, the pre-training can be web crawler pre-training; the first feature can consider not only the most important features but also global features, and can connect contextual semantic information. The first feature can also be called a new node.

[0110] Pre-training the first text to obtain at least one word vector in the first text and a first weight corresponding to each word vector in the at least one word vector can be performed by crawling the first text to obtain at least one word vector in the first text and a first weight corresponding to each word vector in the at least one word vector.

[0111] Determining a set of semantically related word vectors based on the semantic information of each word vector can be achieved by adding a word vector window parameter based on the semantic information of each word vector, and accumulating the word vectors according to the word vector window parameter to obtain the set of semantically related word vectors. For ease of understanding, an example is provided here: the word vector window parameter may include the size m of the accumulated window and the connection channel p. For a word vector within window m, it can be C. i ..., C i+m-1 .

[0112] Determining the first feature based on the word vector set and the first weight corresponding to each word vector in the word vector set can be achieved by performing averaging and maximizing processing on the word vector set and the first weight corresponding to each word vector in the word vector set, respectively. The averaging process can be understood as averaging the word vectors when p = 1, where the importance of each word is consistent, equivalent to consistent word weights, allowing the learning of features for each word, similar to mean-pooling in image convolutional neural networks. This reduces the first type of error and preserves more of the text's context information. The maximizing process can be understood as taking the maximum value in a column of data when p = +∞, according to the definition of Lp distance. This means focusing only on the largest vector value in that column, similar to max-pooling in image operations. This reduces the second type of error and highlights more semantically important features.

[0113] In one embodiment, determining the first feature based on the word vector set and the weight corresponding to each word vector in the word vector set includes:

[0114] The second feature of each word vector in the word vector set is obtained by averaging each word vector in the word vector set according to the first weight corresponding to each word vector in the word vector set;

[0115] The third feature corresponding to the largest word vector value is determined based on the word vector set and the weight corresponding to each word vector in the word vector set.

[0116] The first feature is determined based on the second feature and the third feature corresponding to the largest word vector value.

[0117] The second feature can be understood as a global feature in the word vector set, which represents the average of the word vectors; the third feature can be understood as the most important feature in the word vector set, which represents the semantically important features of the word vectors.

[0118] As an example, the second feature of each word vector in the word vector set is obtained by averaging each word vector in the word vector set according to the first weight corresponding to each word vector in the word vector set. This can be understood as the word vectors being averaged when p=1, where the importance of each word is consistent, which is equivalent to the word weights being consistent. This allows the features of each word to be learned, similar to mean-pooling in image convolutional neural networks. This can reduce the first type of error and retain more of the connection information between the preceding and following parts of the text.

[0119] The third feature corresponding to the largest word vector value, determined by the word vector set and the weights of each word vector in the word vector set, can be understood as follows: when p = +∞, according to the definition of Lp distance, it means taking the maximum value in a column of data, that is, only focusing on the largest vector value in that column, similar to max-pooling in image operations. At this time, the second type of error can be reduced, and more semantically important features can be highlighted.

[0120] Determining the first feature based on the second feature and the third feature corresponding to the largest word vector value can be achieved by averaging the second feature and the third feature corresponding to the largest word vector value.

[0121] In practical applications, while word-to-word similarity is relatively easy to learn in text contexts, it is difficult to learn similarity between similar phrases. For example, two short texts, "Please take care of my mom. Could you?" and "Can you look after my mother?", appear intuitively very similar. However, for computers, word vectors learned through word vector training cannot effectively learn the connection between the phrases "take care of" and "look after." This is because the semantic similarity between the words "take," "care," and "of" and the words "look" and "after" is very sparse.

[0122] To address this issue, this paper proposes a method for obtaining phrase vectors. New node vectors are generated by setting the window size *m* and the value of *p*, fully learning the semantic information of the text for feature extraction. The main approach involves adding a word vector accumulation layer between the matching sentence and the matched sentence. The size of the accumulation window is *m*, which generally does not exceed the window size used during word2vec training. The accumulated result is a new node. Furthermore, considering the different importance of different word dimensions when accumulating nodes, a connection channel *p* is set. For the word vector *C* under window *m*... i ..., C i+m-1 Using formula Obtain the new node vector, where c ij For word vector C i The j-th component, p≥1, and n is the word vector dimension. Two vectors are obtained using p=∞ and p=1 methods, and their average is used as the new node vector expression. To facilitate input into the MP-SWMD algorithm, this application defines the transition probability c between the new and old nodes as infinite, and the transition cost f as 0 for newly generated nodes, as detailed below. Figure 3 As shown, Figure 3 This is a schematic diagram of the generation of a new node in an embodiment of the present invention; the new node can be understood as the first feature.

[0123] The errors in feature extraction mainly come from two aspects: (1) the increased variance of the estimated value caused by the limited size of the neighborhood, i.e., the k value; and (2) the bias of the estimated mean. When p = 1, it means averaging the word vectors. At this time, the importance of each word is consistent, which is equivalent to the word weight being consistent. The features of each word can be learned, which is similar to mean-pooling in image convolutional neural networks. This can reduce the first type of error and retain more of the contextual connection information. When p = +∞, according to the definition of Lp distance, it means taking the maximum value in a column of data. That is, it only focuses on the largest vector value in the column, which is similar to max-pooling in image operations. This can reduce the second type of error and highlight more of the semantically important features. This application effectively considers the effects of the two pooling methods when averaging the two vectors. By setting the p value, the data features can be fully fitted. It not only considers the most important features but also the global features and connects the contextual semantic information.

[0124] The specific operation for generating new nodes is as follows: Assuming the word vectors obtained from training the three words "take", "care", and "of" are (0.001, 0.0023, 0.034, ...), (0.012, 0.0034, 0.012, ...), and (0.00078, 0.000013, 0.0000034, ...), then with p = 1 and m = 3, this application can calculate the new node vector as ((0.001 + 0.012 + 0.00078) / 3, (0.0023 + 0.0034 + 0.000013) / 3 We can obtain the vector expression when p = 1 by calculating (0.034 + 0.012 + 0.0000034) / 3, ..., which is referred to as expression S1 in this application. When p = +∞ and m = 3, this application can calculate (max(0.001, 0.012, 0.00078), max(0.0023, 0.0034, 0.000013), max(0.034, 0.012, 0.0000034), ...), thus obtaining expression S2. The final new node expression can be obtained by adding the vector columns of S1 and S2 and averaging them.

[0125] In this embodiment, after obtaining text word vectors through pre-training with a text crawler, new nodes are formed by adding word vector window settings and defining a new node acquisition method based on word semantic information. By setting the window number parameter to achieve semi-connection and setting the inter-word vector weights to form new node vectors, not only can the inter-word information be effectively utilized, but also the information between phrases and sentences can be fully considered.

[0126] In step S203, offline training is performed on the first feature to generate the central proxy text corresponding to different categories of features in the first feature. This can be understood as using a Multi-center ProxyTxt Generate Net (MCPTG-net) to offline learn multi-center clustering points of multiple text data. This network is mainly divided into five sub-networks: feature learning network, online multi-center K-MEANS clustering, cluster center text generation network, text discrimination network, and classification network. First, multiple cluster center points of the text, also known as proxy center points, are learned through the five sub-networks of the MCPTG network. Through the learning of the MCPTG network, the data domain is converted into multi-center proxy text as a substitute. That is, the multi-center proxy generation network is trained offline to generate multiple categories of multi-center proxy text.

[0127] In step S204, supervised learning is performed on the central proxy texts corresponding to different categories of features in the first feature. The word shift distance between the first central proxy text and the second central proxy text can be understood as multi-center proxy text generated based on the MCPTG network. The improved multi-center proxy NCA loss SMWD algorithm (MP-SWMD) is input to update the gradient parameters, and then the SWMD distance is calculated. Here, the gradient parameters can be understood as weight parameters.

[0128] In one embodiment, the step of performing supervised learning on the central proxy texts corresponding to different categories of features in the first feature to obtain the word shift distance between the first central proxy text and the second central proxy text includes:

[0129] The preset algorithm is initialized based on the first feature to obtain the second weight corresponding to the first feature;

[0130] The central proxy texts corresponding to different categories of features in the first feature are input into the preset algorithm to update the second weights corresponding to the first feature, thereby obtaining the third weights of the central proxy texts corresponding to different categories of features in the first feature.

[0131] The third weights of the central proxy texts corresponding to the different categories of features in the first feature determine the word shift distance between the first central proxy text and the second central proxy text.

[0132] The preset algorithm can be determined according to the actual situation and is not limited here. As an example, the preset algorithm can be the improved SMWD algorithm for multi-center agent NCA loss (MP-SWMD).

[0133] The preset algorithm is initialized based on the first feature, and the second weight corresponding to the first feature can be understood as the original NCA-initialized MP-SWMD algorithm weight vector.

[0134] The central proxy text corresponding to different categories of features in the first feature is input into the preset algorithm to update the second weight corresponding to the first feature. The third weight of the central proxy text corresponding to different categories of features in the first feature can be understood as the center point being input into the MP-SWMD algorithm to update the weight.

[0135] The third weight of the central proxy text corresponding to the different categories of features in the first feature determines the word shift distance between the first central proxy text and the second central proxy text. This can be understood as calculating the WMD distance between the two texts based on the weight.

[0136] In practical applications, the process in this embodiment occurs during the network training phase. When updating parameters based on the original SWMD algorithm, the WMD distance between each pair of texts needs to be calculated in each iteration. However, as a non-linear calculation method, the WMD distance is extremely time-consuming. To address this, this paper proposes an MP-SWMD algorithm, which reduces the computational overhead of calculating the SWMD distance by replacing the parameter update using all samples in each iteration with the use of multiple cluster centers of all samples.

[0137] The algorithm mainly consists of two stages. Stage 1 involves offline training of the MCPTG-net for generating multi-center agent text, which is used to generate multi-class, multi-center agent text. Stage 2 uses the multi-center agent text generated by the MCPTG network as input to the improved MP-SWMD to update the gradient parameters and then calculate the SWMD distance. Specifically... Figure 4 As shown, Figure 4 This is a schematic diagram of the SWMD algorithm and MP-SWMD in the embodiments of this application; Figure 4 The left side of the middle section shows the SWMD algorithm; Figure 4 The rightmost part shows the MP-SWMD algorithm.

[0138] like Figure 4 As shown on the left, the original SWMD algorithm needs to calculate the SWMD distance between the target sample and its N nearest neighbors in each iteration, i.e., it needs to calculate the SWMD distance between data domains B and C and the target sample A, which involves a large amount of computation. The MP-SWMD algorithm proposed in this application, however, is as follows: Figure 4As shown on the right, multiple cluster centers of the text, also known as surrogate centers, are first learned through the five sub-networks of the MCPTG network. Through the learning of the MCPTG network, the data domains B and C are transformed into multi-center surrogate texts D, E, and F as substitutes. The parameters are updated by calculating the SWMD distance between the target sample and the surrogate texts D, E, and F, thereby improving the computational efficiency.

[0139] Considering the change in data domain, and the fact that a category often contains multiple centers in real data, to prevent the network from focusing excessively on local information, this application combines hard sample mining and the idea of ​​triplet loss to define the following formula (2):

[0140]

[0141] In equation (2), p i For sample X i The probability of being correctly classified, where p(X) it ) is text X i Multiple proxies of the same category, where t represents text X i The number of cluster centers for the corresponding category, p(X) J ) is text X i The set of all negative sample proxies. This probability value measures the importance of distance by adding σ as a scaling parameter; and by adding the parameter α as the distance interval between hard samples, it realizes the push and pull between hard samples, thereby separating clusters of the same class from clusters of different classes.

[0142] The objective of this application is to minimize the loss function min-∑ i log(p i The goal is to maximize the probability that a sample is correctly classified. To prevent overfitting and make the resulting model parameter solution more sparse, conforming to the sparsity of text semantic transfer, prior knowledge is introduced, an L1 regularization term is added, and the loss function is defined as shown in the following formula (3):

[0143]

[0144] From a semantic perspective, the regularization part of this loss function considers the following: Since most words are inherently difficult to convert between each other, such as the sentences "I really like playing football" and "He really loves playing guitar," if we calculate their similarity, then according to normal human understanding of sentences, the probability of converting the adverb "very" to "extremely" should actually be much greater than the probability of "very" being converted to "guitar" or other words. Therefore, in actual parameter training, the result will show that the probability of "very" being converted to "extremely" is very high, while the probability of "very" being converted to other words, such as "he," "love," "play," and "guitar," is actually close to 0. This is more in line with people's semantic understanding to a certain extent, and therefore more suitable for measuring text distance.

[0145] Considering that the max function in equation (3) is not differentiable, and combining the LSE approximation idea, we obtain the following formula (4):

[0146]

[0147] Substituting equation (4) into equation (3) to update the gradient, and taking the derivative, we can obtain the following formula (5):

[0148]

[0149]

[0150] Where p(X) K )=p(X I )∪p(X J ), p(X I p(X) represents the set of proxy samples of the same category. J () represents a set of samples from different categories. ρ is a non-negative regularization parameter. From equations (5) and (6), it can be seen that, essentially, different weights are assigned to agents of the same class and agents of different classes during gradient update. Therefore, we can obtain...

[0151]

[0152]

[0153] Where, x i For sample X i In the word vector representation, x k p(X) k The word vector representation in ) , α * and β * To find the dual parameters of the WMD distance, substitute equations (7) and (8) into equations (5) and (6) to update the MP-SWMD algorithm parameters. Finally, update the parameters A, w, and T. ij* Substituting =diag(u)·K·diag(v) into equation (1) yields the corresponding SWMD distance, which is the word shift distance between the first central proxy text and the second central proxy text.

[0154] In step S205, the preset kernel function classification model can be determined according to the actual situation, and is not limited here. As an example, the preset kernel function classification model can be an improved classification model based on the traditional kernel function SVM.

[0155] In one embodiment, the method further includes:

[0156] The kernel function matrix is ​​obtained based on the word shift distance;

[0157] Based on the improved vector machine (SVM) algorithm using the kernel function matrix, the preset kernel function classification model is obtained.

[0158] In this embodiment, the distance classification algorithm in related technologies often uses the KNN algorithm for classification. However, as a linear classification algorithm, the KNN algorithm is difficult to learn non-linear scenarios. Therefore, this application improves upon the traditional kernel function SVM by proposing a new kernel function for classification.

[0159] Specifically, based on Gaussian function The idea proposes a new kernel function, as shown in the following formula (9):

[0160]

[0161] In equation (9), γ is the kernel width parameter. This equation uses WMD(x i x j Explicitly replacing the L2 distance metric of the Gaussian function is more conducive to the implementation of this algorithm.

[0162] This embodiment proposes a novel kernel function. This kernel function effectively addresses the limitation of distance-based classification algorithms, such as KNN and thresholding, and, when combined with the SVM algorithm, effectively solves non-linear classification and better fits high-dimensional data.

[0163] In one embodiment, the step of inputting the word shift distance into a preset kernel function classification model to obtain the classification results of the first center proxy text and the second center proxy text includes:

[0164] The word shift distance is input into a preset kernel function classification model to obtain the classification plane function of the first central proxy text and the second central proxy text;

[0165] Determine whether the value of the classification plane function is greater than or equal to a preset threshold;

[0166] If the value of the classification plane function is greater than or equal to the preset threshold, the first category label of the first central agent text is determined;

[0167] If the value of the classification plane function is less than the preset threshold, the second category label of the second central agent text is determined.

[0168] The classification plane function can be determined according to the actual situation and is not limited here. As an example, the classification plane function can also be called the classification hyperplane function. The preset threshold can be determined according to the actual situation and is not limited here. As an example, the preset threshold can be zero.

[0169] The word-shift distance is input into a preset kernel function classification model to obtain the classification hyperplane function for the first and second center proxy texts. This can be understood as training the preset kernel function classification model with the word-shift distance to obtain datasets of different categories, and then determining the classification hyperplane function based on these datasets. For ease of understanding, determining the classification hyperplane function based on these datasets can be done by using one category as the positive sample dataset and the other categories (excluding the first category) as the negative sample dataset. Training with the positive sample dataset determines a hyperplane function. Substituting the text feature data into this hyperplane function, if the hyperplane function is greater than or equal to zero, the feature data belongs to the same category as the positive sample dataset; otherwise, they belong to different categories. Training with the negative sample dataset also determines a hyperplane function. Substituting the text feature data into this hyperplane function, if the hyperplane function is greater than or equal to zero, the feature data belongs to the same category as the negative sample dataset; otherwise, they belong to different categories. In practical applications, the negative sample dataset can include data from one category or data from multiple categories, which is not limited here.

[0170] In practical applications, this kernel function is placed in the dual problem of SVM. The optimization function of this application is as described in the following formulas (10) and (11):

[0171]

[0172]

[0173] In equations (10) and (11), α i ≥0, i=1,2,…n.,α j b are the parameters to be solved, and y j For sample x jThe label. Based on the above equation, the SMO algorithm is used to solve for the value of α, and from the value of α, we can obtain...

[0174]

[0175] The classification hyperplane can then be obtained.

[0176]

[0177] The text classification result can be obtained by determining whether f(x) is positive or negative. If f(x) is greater than or equal to zero, it is a positive sample, which is text of the same category; if f(x) is less than zero, it is a negative sample, which is text of a different category.

[0178] In one embodiment, obtaining the classification result of the first text based on the classification results of multiple first-center proxy texts and multiple second-center proxy texts includes:

[0179] Obtain multiple first category tags corresponding to multiple first center agent texts and multiple second category tags corresponding to multiple second center agent texts;

[0180] The category label of the first text is determined based on the plurality of first category labels and the plurality of second category labels.

[0181] To better understand, the text classification method described here is a fast text classification method based on supervised Wasserstein distance using multi-center proxy NCA loss, called the MP-SWMD algorithm. This method mainly improves three aspects: the data acquisition stage, the network training stage, and the text classification stage, aiming to enhance both the speed and accuracy of text classification. The core idea of ​​this scheme is as follows:

[0182] 1) Data Acquisition Stage: After the text crawler pre-trains to obtain text word vectors, based on word semantic information, it adds word vector window settings and defines a new node acquisition method to form new nodes. By setting the window number parameter to achieve semi-connection and setting the inter-word vector weights to form new node vectors, it can not only effectively utilize inter-word information, but also fully consider the information between phrases and sentences;

[0183] 2) Network Training Phase: Modifying the network structure, loss function, and gradient update method, the network is divided into two training parts: offline Multi-centers Proxy Txt Generate Net (MCPTG-net) training and Multi-centers Proxy Supervised WordMover's Distance (MP-SWMD) algorithm with modified loss. On one hand, combined with the online K-means clustering algorithm, a supervised deep learning network framework is proposed. This architecture can effectively learn multiple centroids of multiple data categories, so as to replace noisy large datasets with high-quality small datasets, reducing the amount of learning data, while ensuring that the generated data centroids can be used as reasonable text input for the MP-SWMD algorithm to update text parameters. On the other hand, based on the original SWMD algorithm, a multi-centers proxy NCA loss and gradient update method are proposed, and a regularization term is explicitly added. This not only learns multiple proxy centroids in the current text data domain to fully learn the data distribution characteristics, but also satisfies the text semantic sparsity transfer characteristics.

[0184] 3) Text classification stage: Considering that the Wasserstein metric ultimately yields the distance between two texts, based on the idea of ​​the Gaussian kernel function, this distance is improved and combined with the SVM algorithm for classification, which effectively improves the data fitting degree and thus improves the accuracy of text classification.

[0185] The detailed flowchart of this solution is shown below.

[0186] The main technological innovations of this proposal are reflected in three stages: data acquisition, network training, and text classification, specifically as follows: Figure 5 As shown, Figure 5 This is a flowchart illustrating one of the text classification methods used in this application.

[0187] Step 1: Data Acquisition Phase.

[0188] In text-based contexts, word-to-word similarities are relatively easy to learn, but similar phrases are much more difficult to learn. For example, two short texts, "Please take care of my mom. Could you?" and "Can you look after my mother?", appear intuitively very similar. However, for a computer, word vectors learned through word vector training cannot effectively learn the connection between the phrases "take care of" and "look after." This is because the semantic similarity between the words "take," "care," and "of" and the words "look" and "after" is very sparse.

[0189] To address this issue, this paper proposes a method for obtaining phrase vectors. New node vectors are generated by setting the window size *m* and the value of *p*, fully learning the semantic information of the text for feature extraction. The main approach involves adding a word vector accumulation layer between the matching sentence and the matched sentence. The size of the accumulation window is *m*, which generally does not exceed the window size used during word2vec training. The accumulated result is a new node. Furthermore, considering the different importance of different word dimensions when accumulating nodes, a connection channel *p* is set. For the word vector *C* under window *m*... i ..., C i+m-1 Using formula Obtain the new node vector, where c ij For word vector C i The j-th component, p≥1, and n is the word vector dimension. Two vectors are obtained using p=∞ and p=1 methods, and their average is used as the new node vector expression. To facilitate input into the MP-SWMD algorithm, this application defines the transition probability c between the new and old nodes as infinite, and the transition cost f as 0 for newly generated nodes, as detailed below. Figure 3 As shown.

[0190] The errors in feature extraction mainly come from two aspects: (1) the increased variance of the estimated value caused by the limited size of the neighborhood, i.e., the k value; and (2) the bias of the estimated mean. When p = 1, it means averaging the word vectors. At this time, the importance of each word is consistent, which is equivalent to the word weight being consistent. The features of each word can be learned, which is similar to mean-pooling in image convolutional neural networks. This can reduce the first type of error and retain more of the contextual connection information. When p = +∞, according to the definition of Lp distance, it means taking the maximum value in a column of data. That is, it only focuses on the largest vector value in the column, which is similar to max-pooling in image operations. This can reduce the second type of error and highlight more of the semantically important features. This application effectively considers the effects of the two pooling methods when averaging the two vectors. By setting the p value, the data features can be fully fitted. It not only considers the most important features but also the global features and connects the contextual semantic information.

[0191] The specific operation for generating new nodes is as follows: Assuming the word vectors obtained from training the three words "take", "care", and "of" are (0.001, 0.0023, 0.034, ...), (0.012, 0.0034, 0.012, ...), and (0.00078, 0.000013, 0.0000034, ...), then with p = 1 and m = 3, this application can calculate the new node vector as ((0.001 + 0.012 + 0.00078) / 3, (0.0023 + 0.0034 + 0.000013) / 3 We can obtain the vector expression when p = 1 by calculating (0.034 + 0.012 + 0.0000034) / 3, ..., which is referred to as expression S1 in this application. When p = +∞ and m = 3, this application can calculate (max(0.001, 0.012, 0.00078), max(0.0023, 0.0034, 0.000013), max(0.034, 0.012, 0.0000034), ...), thus obtaining expression S2. The final new node expression can be obtained by adding the vector columns of S1 and S2 and averaging them.

[0192] Step 2: Network training phase.

[0193] When updating parameters based on the original SWMD algorithm, the WMD distance between each pair of texts needs to be calculated in each iteration. However, as a non-linear calculation method, the WMD distance is very time-consuming. To address this, this paper proposes a Multi-centers Proxy Supervised Word Mover's Distance (MP-SWMD) algorithm. By changing the parameter update method from using all samples in each iteration to using multiple cluster centers of all samples, the computational cost of calculating the SWMD distance is significantly reduced.

[0194] The algorithm mainly consists of two stages. Stage one involves offline training of a Multi-centersProxy Txt Generate Net (MCPTG-net) to generate multi-center proxy text for multiple categories. Stage two involves inputting the multi-center proxy text generated by the MCPTG network into an improved Multi-centersProxy NCA loss-based SMWD algorithm (MP-SWMD) to update the gradient parameters and then calculate the SWMD distance. Specifically... Figure 4 As shown.

[0195] like Figure 4As shown in the left figure, the original SWMD algorithm needs to calculate the SWMD distance between the target sample and its N nearest neighbors in each iteration, i.e., it needs to calculate the SWMD distance between data domains B and C and the target sample A, which involves a large amount of computation. The MP-SWMD algorithm proposed in this application, however, is as follows: Figure 4 As shown in the right figure, multiple cluster centers of the text, also known as surrogate centers, are first learned through the five sub-networks of the MCPTG network. Through the learning of the MCPTG network, the data domains B and C are transformed into multi-center surrogate texts D, E, and F as substitutes. The parameters are updated by calculating the SWMD distance between the target sample and the surrogate texts D, E, and F, thereby improving the computational efficiency.

[0196] Considering the change in the data domain, and the fact that a category often contains multiple centers in real data, in order to prevent the network from focusing too much on local information, this application combines hard sample mining and the idea of ​​Triplet loss, and defines the loss as shown in the previous formula (2).

[0197] For sample X i The probability of being correctly classified, where p(X) it ) is text X i Multiple proxies of the same category, where t represents text X i The number of cluster centers for the corresponding category, p(X) J ) is text X i The set of all negative sample proxies. This probability value measures the importance of distance by adding σ as a scaling parameter; and by adding the parameter α as the distance interval between hard samples, it realizes the push and pull between hard samples, thereby separating clusters of the same class from clusters of different classes.

[0198] The objective of this application is to minimize the loss function min-∑ i log(p i The goal is to maximize the probability that a sample is correctly classified. To prevent overfitting and make the resulting model parameter solution more sparse, which conforms to the sparsity of text semantic transfer, prior knowledge is introduced and an L1 regularization term is added. The loss function is defined as shown in the previous formula (3).

[0199] From a semantic perspective, the regularization part of this loss function considers the following: Since most words are inherently difficult to convert between each other, such as the sentences "I really like playing football" and "He really loves playing guitar," if we calculate their similarity, then according to normal human understanding of sentences, the probability of converting the adverb "very" to "extremely" should actually be much greater than the probability of "very" being converted to "guitar" or other words. Therefore, in actual parameter training, the result will show that the probability of "very" being converted to "extremely" is very high, while the probability of "very" being converted to other words, such as "he," "love," "play," and "guitar," is actually close to 0. This is more in line with people's semantic understanding to a certain extent, and therefore more suitable for measuring text distance.

[0200] Considering that the max function in equation (3) is not differentiable, and combining the LSE approximation idea, we obtain the above equation (4).

[0201] Substituting equation (4) into equation (3) to update the gradient, and taking the derivative, we can obtain the previous equations (5) and (6).

[0202] Where p(X) K )=p(X I )∪p(X J ), p(X I p(X) represents the set of proxy samples of the same category. J () represents a set of samples from different categories. ρ is a non-negative regularization parameter. From equations (5) and (6), it can be seen that, essentially, different weights are assigned to agents of the same class and agents of different classes during gradient update. This leads to the previous equations (7) and (8).

[0203] Where, x i For sample X i In the word vector representation, x k p(X) k The word vector representation in ) , α * and β * To find the dual parameters of the WMD distance, substitute equations (7) and (8) into equations (5) and (6) to update the MP-SWMD algorithm parameters. Finally, update the parameters A, w, and T. ij * Substituting =diag(u)·K·diag(v) into equation (1) yields the corresponding SWMD distance.

[0204] Multi-centers Proxy Txt Generate Net (MCPTG-net).

[0205] This network is mainly used for offline learning of multi-center clusters of multiple text datasets. It consists of five sub-networks: a feature learning network, an online multi-center K-MEANS clustering network, a cluster center text generation network, a text discrimination network, and a classification network. Its network loss structure function is shown in equation (14-20). Figure 6 This is a schematic diagram of the overall model structure of an embodiment of this application, as shown below. Figure 6 As shown.

[0206] G = argmin G L GAN (G, D) (14)

[0207] D = argmin D L GAN (G, D) (15)

[0208] C = argmin C L C (C, Y) (16)

[0209]

[0210]

[0211]

[0212]

[0213] Where G, C, and D represent the generator network, classification network, and discriminator network, respectively, and L GAN (G, D) represents the network loss of GANs, and L... C (C, Kmeans) represents the loss of the classification network during training. This represents the initial classification network loss. This represents the distribution of data y, where c is the number of categories, and classifier is... out j The classification network outputs a probability value for the current sample to be of class j. This indicates the value of the online k-means prediction for category j, which is either 0 or 1. Let j be the cluster center of the i-th category at time s. This is the learning rate value. The feature vector learned by the classification network at time s-1.

[0214] like Figure 6As shown, during the network training phase, the generation network parameters are updated using Equation (14), the discrimination network parameters are updated using Equation (15), the classification network parameters are updated using Equation (16), and the cluster centers are updated using Equation (20). During training, the parameters are updated cross-referenced to achieve iterative updates of the overall network parameters. During the inference phase, the processed text is input into the network to obtain the multi-center agent text vector matrix.

[0215] The structure of each sub-network is described in detail below.

[0216] 1) Feature learning network + classification network (BERT model classification)

[0217] This network is primarily used for feature extraction and classification. During training, the feature learning network and the classification network are trained together, with the BERT network mainly used as the backbone for text classification. Features learned in the Transformer layers are used as the output of the feature learning network and saved as sample features. The obtained features are then input into the classification network to complete the classification.

[0218] When initializing network parameters, this application adopts... The loss is compared with the ground truth during training; during training, this application uses L... C (C, K-means) are used as the loss update parameters. The reason for this is that when initializing the parameters, this application primarily uses real labels to learn the network parameters, thus the network parameters tend to learn real text labels and can effectively utilize text information. During network training, considering the inclusion of cluster centers, cluster centers and corresponding labels can be obtained through online K-means clustering. This allows the calculation of the corresponding clusters and the acquisition of the corresponding cluster center labels using the original data features, and then using L... C Using (C, Kmeans) as the loss to update the gradient can fully learn the cluster center information, thus making the cluster centers reasonably biased towards the text, and achieving excellent clustering results.

[0219] 2) Online multicenter K-means clustering

[0220] This algorithm is mainly used to learn the cluster center of the sample under the current features, and updates the sample cluster center points through equation (20). In each iteration update, the original online-kmeans uses c←(1-η)c+ηx to update the cluster center. Considering that the feature vector learned by the neural network will be shifted between the two iterations, this application utilizes... Update the cluster centers. Thus, the current cluster centers can be explicitly converted into a weighted average of the previous cluster center and the cluster samples, which can be understood as... For momentum, The change brought about by each data point is equivalent to the gradient provided by each point, γ. ij As the learning rate, equation (20) is equivalent to mini-batch gradient descent with momentum, which is more suitable for training gradient updates embedded in neural networks.

[0221] The specific update steps are as follows:

[0222] A. Initialize centroids. Randomly select t initial centroids for each category from the global dataset, input them into the classification network to obtain the corresponding features as initial cluster centers, and use C... I ={C i , i = 1, ..., k * t} represents, where k represents the number of categories, and t represents the number of centroids selected for each category, which is a constant;

[0223] B. Predicting the Category. The learned feature data is input into the algorithm. The distance between the current feature data and the feature centroid is calculated. The current sample is assigned to the nearest cluster centroid, and the sample is given the cluster category. The result is then used... This represents the value predicted as category j, and is represented by 0 or 1.

[0224] C. Update cluster centroids. Calculate the sample data volume for each cluster. use The update yields the next cluster sample center point, where The current cluster sample features are defined at time s-1.

[0225] 3) Text generation + text discrimination network (Text-GAN network)

[0226] This network is primarily used to generate text proxies. It utilizes learned text cluster center features to generate a reasonable text representation through a generative network, and a discriminative network determines the authenticity of the generated text, making it approximate real text. The generative network mainly uses a traditional decoder model—an LSTM model with attention—to generate the text; the discriminative network, treating it as a binary classification problem, primarily uses a CNN combined with an LSTM for classification, thus obtaining pseudo-realistic clustered text proxies.

[0227] Step 3: Text classification stage.

[0228] This stage primarily relies on the SWMD distance obtained through the MP-SWMD algorithm for text classification. While the MP-SWMD algorithm allows for the calculation of distances between pairs of text, existing distance classification algorithms often employ the KNN algorithm, which, as a linear classification algorithm, struggles to learn non-linear scenarios. Therefore, this application improves upon the traditional kernel function SVM by proposing a novel kernel function for classification.

[0229] Specifically, this application is based on the Gaussian function. The idea is to propose a new kernel function, referring to the previous formula (9).

[0230] Where γ is the kernel width parameter. This formula uses WMD(x) i x j Explicitly replacing the L2 distance metric of the Gaussian function is more conducive to the implementation of this algorithm. At the same time, this application can also prove that the formula satisfies the definition of the kernel function. When applying this kernel function, it is placed in the dual problem of SVM. Then the optimization function of this application is as described in the previous formulas (10) and (11).

[0231] Where, α j b are the parameters to be solved, and y j For sample x j The label. The value of a is obtained by using the SMO algorithm based on the above equation. b can be obtained from the value of a, as described in the previous formula (12).

[0232] The classification hyperplane can then be obtained, as described in the previous formula (13).

[0233] The text classification result can be obtained by determining the sign of f(x).

[0234] This embodiment proposes a fast text classification method based on supervised Wasserstein distance using multi-center surrogate NCA loss—the MP-SWMD algorithm. This method mainly improves three aspects: the data acquisition stage, the network training stage, and the text classification stage, aiming to enhance both the speed and accuracy of text classification.

[0235] Compared with existing technologies, the advantages of this application are mainly reflected in the following aspects:

[0236] First, addressing the issues of high computational cost and overfitting in related technical classification methods, this application improves upon the SWMD algorithm by incorporating a multi-center agent network to learn text agents, thereby reducing the data domain by up to 40 times and significantly lowering time consumption. When updating the SWMD algorithm, the NCA loss is improved to a multi-center loss, and regularization and hard sample mining are added to prevent overfitting and effectively mine hard samples. Furthermore, the text semantic sparsity is combined to increase computational speed and improve accuracy. Finally, by integrating a multi-center agent generation network and the improved multi-center loss SWMD algorithm, distance measurement time is effectively reduced.

[0237] Second: This application addresses the issue that related technical classification methods do not fully consider the similarity between phrases and sentences. In the text word vector stage, this application sets the window number m and the p value to learn semantic similarity. By setting p=1 and p=∞, it learns important semantic features and contextual information, thereby generating new node vectors. Compared with the original technology, this approach can more fully learn text information, that is, it fully considers the similarity between phrases and sentences.

[0238] Third: Addressing the issue that related classification methods have poor fit to high-dimensional data and are unable to learn non-linear classification, this proposal suggests a novel kernel function. This function overcomes the limitation that distance-based classification methods are only applicable to simple classification algorithms such as KNN and thresholding. Combined with the SVM algorithm, it effectively solves the problem of non-linear classification and better fits high-dimensional data.

[0239] To implement the method of this application embodiment, this application embodiment also provides a text classification device 700, which is installed on an electronic device. Figure 7 This is a schematic diagram of a text classification device according to an embodiment of this application; as shown below. Figure 7 As shown, it includes:

[0240] Acquisition unit 701 is used to acquire the first text;

[0241] Preprocessing unit 702 is used to preprocess the first text to obtain a first feature representing the semantic information of word vectors in the first text;

[0242] Training unit 703 is used to perform offline training on the first feature and generate central proxy texts corresponding to different categories of features in the first feature.

[0243] Learning unit 704 is used to perform supervised learning on the central proxy texts corresponding to different categories of features in the first feature, and to obtain the word shift distance between the first central proxy text and the second central proxy text; the first central proxy text is the central proxy text corresponding to the first category feature in the different categories of features; the second central proxy text is the central proxy text corresponding to any category feature in the different categories of features other than the first category feature; the first category feature is any category feature in the different categories of features;

[0244] Unit 705 is used to input the word shift distance into a preset kernel function classification model to obtain the classification results of the first central proxy text and the second central proxy text;

[0245] The determining unit 706 is used to determine the category label of the first text based on the classification results of multiple first central agent texts and multiple second central agent texts.

[0246] Here, in one embodiment, the preprocessing unit 702 is further configured to pre-train the first text to obtain at least one word vector in the first text and a first weight corresponding to each word vector in the at least one word vector; determine a set of word vectors with semantic association based on the semantic information of each word vector; and determine the first feature based on the set of word vectors and the first weight corresponding to each word vector in the set of word vectors.

[0247] Here, in one embodiment, the preprocessing unit 702 is further configured to average each word vector in the word vector set according to the first weight corresponding to each word vector in the word vector set to obtain a second feature of each word vector in the word vector set; determine a third feature corresponding to the largest word vector value according to the word vector set and the weight corresponding to each word vector in the word vector set; and determine the first feature according to the second feature and the third feature corresponding to the largest word vector value.

[0248] Here, in one embodiment, the learning unit 704 is further configured to initialize a preset algorithm based on the first feature to obtain a second weight corresponding to the first feature; input the central proxy texts corresponding to different categories of features in the first feature into the preset algorithm to update the second weights corresponding to the first feature to obtain a third weight of the central proxy texts corresponding to different categories of features in the first feature; and determine the word shift distance between the first central proxy text and the second central proxy text using the third weights of the central proxy texts corresponding to different categories of features in the first feature.

[0249] Here, in one embodiment, the obtaining unit 705 is further configured to obtain a kernel function matrix based on the word shift distance; and improve the vector machine (SVM) algorithm based on the kernel function matrix to obtain the preset kernel function classification model.

[0250] Here, in one embodiment, the obtaining unit 705 is further configured to input the word shift distance into a preset kernel function classification model to obtain a classification plane function of the first central proxy text and the second central proxy text; determine whether the value of the classification plane function is greater than or equal to a preset threshold; if the value of the classification plane function is greater than or equal to the preset threshold, determine the first category label of the first central proxy text; if the value of the classification plane function is less than the preset threshold, determine the second category label of the second central proxy text.

[0251] Here, in one embodiment, the determining unit 706 is further configured to obtain multiple first category tags corresponding to multiple first central agent texts and multiple second category tags corresponding to multiple second central agent texts;

[0252] The category label of the first text is determined based on the plurality of first category labels and the plurality of second category labels.

[0253] It should be noted that the text classification device provided in the above embodiments is only illustrated by the division of the above program modules when performing text classification. In actual applications, the above processing can be assigned to different program modules as needed, that is, the internal structure of the device can be divided into different program modules to complete all or part of the processing described above. In addition, the text classification device and the text classification method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0254] Based on the hardware implementation of the above program modules, this application embodiment also provides an electronic device, including a memory and a processor. The memory stores a computer program that can run on the processor. When the processor executes the program, it implements the steps in the text classification method provided in the above embodiments.

[0255] Correspondingly, embodiments of this application provide a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps in the text classification method provided in the above embodiments.

[0256] It should be noted that the descriptions of the storage medium and device embodiments above are similar to the descriptions of the method embodiments above, and have similar beneficial effects. For technical details not disclosed in the storage medium and device embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0257] It should be noted that, Figure 8 This is a schematic diagram of a hardware entity structure of an electronic device in an embodiment of this application, such as... Figure 8 As shown, the hardware entity of the electronic device 800 includes a processor 801 and a memory 803. Optionally, the electronic device 800 may also include a communication interface 802.

[0258] It is understood that memory 803 can be volatile memory or non-volatile memory, or both. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), ferromagnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disc, or compact disc read-only memory (CD-ROM); magnetic surface memory can be disk storage or magnetic tape storage. Volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Synchronous Static Random Access Memory (SSRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM), Enhanced Synchronous Dynamic Random Access Memory (ESDRAM), SyncLink Dynamic Random Access Memory (SLDRAM), and Direct Rambus Random Access Memory (DRRAM).The memory 803 described in the embodiments of this application is intended to include, but is not limited to, these and any other suitable types of memory.

[0259] The methods disclosed in the embodiments of this application can be applied to or implemented by processor 801. Processor 801 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuit of the hardware in processor 801 or by instructions in software form. The processor 801 may be a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Processor 801 can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. A general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the methods disclosed in the embodiments of this application can be directly manifested as being executed by a hardware decoding processor, or being executed by a combination of hardware and software modules in the decoding processor. The software modules may be located in a storage medium, which is located in memory 803. Processor 801 reads the information in memory 803 and combines it with its hardware to complete the steps of the aforementioned method.

[0260] In an exemplary embodiment, the device may be implemented by one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers (MCUs), microprocessors, or other electronic components to perform the aforementioned method.

[0261] It should be understood that the phrase "one embodiment" or "an embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that in the various embodiments of this application, the sequence numbers of the above-described processes do not imply a sequential order of execution; the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. The sequence numbers of the above-described embodiments are merely descriptive and do not represent the superiority or inferiority of the embodiments.

[0262] It should be noted that, in this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0263] The methods disclosed in the several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0264] The features disclosed in the several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0265] The features disclosed in the several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method or device embodiments.

[0266] The above description is merely an embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A text classification method, characterized in that, include: Get the first text; The first text is preprocessed to obtain the first feature representing the semantic information of word vectors in the first text; The first feature is trained offline to generate the central proxy text corresponding to the different categories of features in the first feature; Supervised learning is performed on the central proxy texts corresponding to different categories of features in the first feature to obtain the word shift distance between the first central proxy text and the second central proxy text; the first central proxy text is the central proxy text corresponding to the first category feature in the different categories of features; the second central proxy text is the central proxy text corresponding to any category feature in the different categories of features other than the first category feature; the first category feature is any category feature in the different categories of features; The word shift distance is input into a preset kernel function classification model to obtain the classification results of the first center proxy text and the second center proxy text; The category label of the first text is determined based on the classification results of multiple first-center proxy texts and multiple second-center proxy texts.

2. The method according to claim 1, characterized in that, The preprocessing of the first text to obtain the first feature representing the semantic information of word vectors in the first text includes: The first text is pre-trained to obtain at least one word vector in the first text and a first weight corresponding to each word vector in the at least one word vector; Based on the semantic information of each word vector, determine the set of word vectors that have semantic relationships; The first feature is determined based on the word vector set and the first weight corresponding to each word vector in the word vector set.

3. The method according to claim 2, characterized in that, Determining the first feature based on the word vector set and the weight corresponding to each word vector in the word vector set includes: The second feature of each word vector in the word vector set is obtained by averaging and weighting each word vector in the word vector set according to the first weight corresponding to each word vector in the word vector set; Max pooling is performed on each word vector in the word vector set based on the word vector set and the weight corresponding to each word vector in the word vector set to obtain the third feature corresponding to the largest word vector value in the word vector set; The first feature is obtained by averaging the second feature and the third feature corresponding to the largest word vector value.

4. The method according to claim 1, characterized in that, The step of performing supervised learning on the central proxy texts corresponding to different categories of features in the first feature, and obtaining the word shift distance between the first central proxy text and the second central proxy text, includes: The preset algorithm is initialized based on the first feature to obtain the second weight corresponding to the first feature; The central proxy texts corresponding to different categories of features in the first feature are input into the preset algorithm to update the second weights corresponding to the first feature, thereby obtaining the third weights of the central proxy texts corresponding to different categories of features in the first feature. The third weights of the central proxy texts corresponding to the different categories of features in the first feature determine the word shift distance between the first central proxy text and the second central proxy text.

5. The method according to claim 1, characterized in that, The method further includes: The kernel function matrix is ​​obtained based on the word shift distance; The vector machine algorithm is improved based on the kernel function matrix to obtain the preset kernel function classification model.

6. The method according to claim 1, characterized in that, The step of inputting the word shift distance into a preset kernel function classification model to obtain the classification results of the first center proxy text and the second center proxy text includes: The word shift distance is input into a preset kernel function classification model to obtain the classification plane function of the first central proxy text and the second central proxy text; Determine whether the value of the classification plane function is greater than or equal to a preset threshold; If the value of the classification plane function is greater than or equal to the preset threshold, the first category label of the first central agent text is determined; If the value of the classification plane function is less than the preset threshold, the second category label of the second central agent text is determined.

7. The method according to claim 6, characterized in that, The step of determining the category label of the first text based on the classification results of multiple first-center proxy texts and multiple second-center proxy texts includes: Obtain multiple first category tags corresponding to multiple first center agent texts and multiple second category tags corresponding to multiple second center agent texts; The category label of the first text is determined based on the plurality of first category labels and the plurality of second category labels.

8. A text classification device, characterized in that, include: The acquisition unit is used to acquire the first text; The preprocessing unit is used to preprocess the first text to obtain a first feature representing the semantic information of word vectors in the first text; The training unit is used to perform offline training on the first feature and generate the central proxy text corresponding to the different categories of features in the first feature. The learning unit is used to perform supervised learning on the central proxy texts corresponding to different categories of features in the first feature, and to obtain the word shift distance between the first central proxy text and the second central proxy text; the first central proxy text is the central proxy text corresponding to the first category feature in the different categories of features; the second central proxy text is the central proxy text corresponding to any category feature in the different categories of features other than the first category feature; the first category feature is any category feature in the different categories of features; The unit is used to input the word shift distance into a preset kernel function classification model to obtain the classification results of the first center proxy text and the second center proxy text; The determining unit is used to determine the category label of the first text based on the classification results of multiple first central agent texts and multiple second central agent texts.

9. An electronic device, characterized in that, include: Memory, used to store executable instructions; A processor, when executing executable instructions stored in the memory, implements the text classification method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, It stores executable instructions for implementing the text classification method according to any one of claims 1 to 7 when executed by a processor.

Images