Text classification method, device, computer equipment and medium based on reinforcement learning

Abstract

The embodiment of the present application belongs to the field of artificial intelligence technology, and relates to a text classification method based on reinforcement learning, including obtaining a training text corpus, extracting semantic features from the training text corpus, and obtaining a semantic feature vector; inputting the semantic feature vector into a trained clustering model, and outputting a semantic cluster; extracting keywords from all semantic clusters, and forming a semantic feature queue corresponding to each semantic cluster according to the extracted keywords; selecting keywords from each semantic feature queue as target keywords, and generating a word semantic vector based on the target keywords; inputting the word semantic vector into a pre-built initial classification model for training, and obtaining a trained target classification model; obtaining a text to be classified, inputting the text to be classified into a target classification model, and outputting a text classification result. The present application also provides a text classification device, a computer device, and a medium based on reinforcement learning. The present application can improve the accuracy of text classification.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a text classification method, device, computer equipment and medium based on reinforcement learning. Background Art

[0002] As a common task in the field of natural language understanding, text classification has developed many methods, which can be generally divided into two major schools: supervised learning and unsupervised learning. In the field of supervised learning, various features that can represent the semantics of the text are extracted and the classification is completed by training the model; in the unsupervised / semi-supervised field, the features of the text are learned autonomously through clustering and other methods, and texts with similar features are clustered to complete the classification.

[0003] However, the current text classification model ignores the imbalance of samples. The model tends to learn more features of the dominant category and ignores the features of the minority category, which makes the learned algorithm prone to overfitting and causes inaccurate text classification. Summary of the invention

[0004] The purpose of the embodiments of the present application is to propose a text classification method, apparatus, computer equipment and medium based on reinforcement learning to solve the technical problem of inaccurate text classification caused by sample imbalance in related technologies.

[0005] In order to solve the above technical problems, the present application embodiment provides a text classification method based on reinforcement learning, which adopts the following technical solution:

[0006] Acquire a training text corpus, perform semantic feature extraction on the training text corpus, and obtain a semantic feature vector;

[0007] Inputting the semantic feature vector into a trained clustering model and outputting a semantic cluster;

[0008] Extracting keywords from all the semantic clusters, and forming a semantic feature queue corresponding to each semantic cluster according to the extracted keywords;

[0009] Selecting a keyword from each of the semantic feature queues as a target keyword, and generating a word semantic vector based on the target keyword;

[0010] Inputting the word semantic vector into a pre-built initial classification model for training to obtain a trained target classification model;

[0011] Obtain the text to be classified, input the text to be classified into the target classification model, and output the text classification result.

[0012] Furthermore, before the step of inputting the semantic feature vector into the trained clustering model and outputting the semantic cluster, the method further includes:

[0013] Inputting the semantic feature vector into a pre-built neural network model and outputting a clustering result;

[0014] Determine a clustering loss function according to the clustering result;

[0015] Adjusting model parameters of the neural network model based on the clustering loss function;

[0016] When the iteration end condition is met, a clustering model is generated according to the model parameters.

[0017] Furthermore, the step of determining a clustering loss function according to the clustering result includes:

[0018] Calculating the silhouette coefficient of each cluster in the clustering result;

[0019] Obtaining a training reward score according to the silhouette coefficient;

[0020] The clustering loss function is obtained based on the clustering result and the training reward score.

[0021] Furthermore, the step of forming a semantic feature queue corresponding to each semantic cluster according to the extracted keywords includes:

[0022] Calculating the similarity between the keywords of each semantic cluster;

[0023] Sorting the keywords according to the similarity to obtain a sorting result;

[0024] A semantic feature queue corresponding to each semantic cluster is generated based on the sorting result.

[0025] Furthermore, the step of generating a word semantic vector based on the target keyword includes:

[0026] Performing feature extraction on the target keyword to obtain a keyword vector;

[0027] The keyword vector and the semantic feature vector are concatenated to obtain a word semantic vector.

[0028] Furthermore, the step of inputting the word semantic vector into a pre-built initial classification model for training to obtain a trained target classification model includes:

[0029] Inputting the word semantic vector into a pre-built initial classification model to obtain a predicted classification result;

[0030] Determine a classification loss function according to the predicted classification result;

[0031] Adjusting the model parameters of the initial classification model according to the classification loss function;

[0032] When the iteration end condition is met, a target classification model is generated based on the model parameters.

[0033] Furthermore, the step of determining the classification loss function according to the predicted classification result includes:

[0034] Calculate a classification reward value according to the predicted classification result;

[0035] A classification loss function is obtained based on the classification reward value and the predicted classification result.

[0036] In order to solve the above technical problems, the embodiment of the present application also provides a text classification device based on reinforcement learning, which adopts the following technical solution:

[0037] A semantic feature extraction module is used to obtain a training text corpus, perform semantic feature extraction on the training text corpus, and obtain a semantic feature vector;

[0038] A clustering module, used for inputting the semantic feature vector into a trained clustering model and outputting a semantic cluster;

[0039] A keyword extraction module, used to extract keywords from all the semantic clusters, and form a semantic feature queue corresponding to each semantic cluster according to the extracted keywords;

[0040] A vector generation module, used to select a keyword from each of the semantic feature queues as a target keyword, and generate a word semantic vector based on the target keyword;

[0041] A training module, used for inputting the word semantic vector into a pre-built initial classification model for training to obtain a trained target classification model;

[0042] The classification module is used to obtain the text to be classified, input the text to be classified into the target classification model, and output the text classification result.

[0043] In order to solve the above technical problems, the embodiment of the present application further provides a computer device, which adopts the following technical solution:

[0044] The computer device includes a memory and a processor, wherein the memory stores computer-readable instructions, and when the processor executes the computer-readable instructions, the steps of the text classification method based on reinforcement learning as described above are implemented.

[0045] In order to solve the above technical problems, the embodiment of the present application further provides a computer-readable storage medium, which adopts the following technical solution:

[0046] The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the steps of the text classification method based on reinforcement learning as described above are implemented.

[0047] Compared with the prior art, the embodiments of the present application have the following beneficial effects:

[0048] The present application obtains training text corpus, extracts semantic features from the training text corpus, and obtains semantic feature vectors; inputs the semantic feature vectors into a trained clustering model and outputs semantic clusters; extracts keywords from all semantic clusters, and forms semantic feature queues corresponding to each semantic cluster based on the extracted keywords; selects keywords from each semantic feature queue as target keywords, and generates word semantic vectors based on the target keywords; inputs the word semantic vectors into a pre-built initial classification model for training, and obtains a trained target classification model; obtains text to be classified, inputs the text to be classified into a target classification model, and outputs text classification results; the present application extracts semantic features from training text corpus, and clusters the extracted semantic feature vectors to obtain semantic clusters of different categories, and then uses semantic clusters of different categories to train the classification model, so that the classification model learns semantic features of different categories in the training text corpus, thereby improving the accuracy of text classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the scheme in the present application, a brief introduction is given below to the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0050] Figure 1 is an exemplary system architecture diagram to which the present application may be applied;

[0051] Figure 2 A flowchart of an embodiment of a text classification method based on reinforcement learning according to the present application;

[0052] Figure 3 is a structural schematic diagram of an embodiment of a text classification device based on reinforcement learning according to the present application;

[0053] Figure 4 It is a structural diagram of an embodiment of a computer device according to the present application. DETAILED DESCRIPTION

[0054] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0055] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0056] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0057] This application provides a text classification method based on reinforcement learning, involving artificial intelligence, which can be applied to Figure 1 In the system architecture 100 shown, the system architecture 100 may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0058] Users can use terminal devices 101, 102, 103 to interact with server 105 through network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, 103, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0059] Terminal devices 101, 102, 103 can be various electronic devices with display screens and supporting web browsing, including but not limited to smart phones, tablet computers, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III), MP4 (Moving Picture Experts Group Audio Layer IV) players, laptop computers, desktop computers, etc.

[0060] The server 105 may be a server that provides various services, such as a background server that provides support for web pages displayed on the terminal devices 101 , 102 , and 103 .

[0061] It should be noted that the reinforcement learning-based text classification method provided in the embodiment of the present application is generally executed by a server / terminal device, and accordingly, the reinforcement learning-based text classification device is generally arranged in the server / terminal device.

[0062] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to the implementation requirements.

[0063] Continue to refer Figure 2 , shows a flowchart of an embodiment of a text classification method based on reinforcement learning according to the present application, comprising the following steps:

[0064] Step S201, obtaining training text corpus, performing semantic feature extraction on the training text corpus, and obtaining a semantic feature vector.

[0065] The training text corpus can be obtained from public data sets, including but not limited to Chinese news data sets, THUCNews data sets, and Online_shoppping_10_cats data sets. The original text corpus is obtained from the public data set, and the original text corpus is preprocessed by word segmentation, stop word removal, etc., and the preprocessed original text corpus is randomly divided into a training set and a test set according to a preset ratio, wherein the training set is the training text corpus, which is a text collection.

[0066] In this embodiment, semantic features are extracted from the training text corpus, and a semantic feature extraction model can be used to extract semantic features. The semantic feature extraction model includes but is not limited to a CNN (Convolutional Neural Networks) model, an RNN (Recurrent Neural Network) model, an LSTM (Long-short term memory) model, and a BERT (Bidirectional Encoder Representations from Transformers) model, etc., which are not limited here.

[0067] As a specific implementation method, the training text corpus can be input into a BERT-based pre-trained language model for semantic feature extraction.

[0068] Step S202: input the semantic feature vector into the trained clustering model and output the semantic cluster.

[0069] In this embodiment, the semantic feature vectors may be clustered using a trained clustering model, wherein the clustering algorithm used by the clustering model includes but is not limited to a K-means algorithm and a Single-Pass algorithm.

[0070] Taking the Single-Pass algorithm as an example, the clustering process is described in detail. The steps include:

[0071] Step A, selecting any semantic feature vector as the cluster center of the first cluster;

[0072] Step B, select any unprocessed semantic feature vector from other unprocessed semantic feature vectors, calculate the similarity value between the semantic feature vector and all existing clusters, select the cluster with the largest similarity value as the closest cluster to the semantic feature vector, and obtain the similarity value of the closest cluster.

[0073] It should be understood that at the beginning of the algorithm, there is only one cluster, that is, the cluster generated in step A. At this time, "all existing clusters" refers to this cluster. As the algorithm runs, the semantic feature vectors are newly created or classified into different clusters, and the number of clusters increases. At this time, "all existing clusters" refers to all the clusters that have been generated so far.

[0074] Step C, judging the similarity value of the closest cluster and a similarity threshold; if the similarity value is greater than the similarity threshold, the semantic feature vector selected in step B is classified into the closest cluster, and the center of the closest cluster is updated; otherwise, the semantic feature vector selected in step B is used as the cluster center of a new cluster.

[0075] In this embodiment, the cluster center is the average value of the semantic feature vectors in the cluster. It is calculated as follows:

[0076]

[0077] Where C represents the centroid vector of the cluster; n represents the number of semantic feature vectors of the cluster; d i Represents the semantic feature vector in this cluster.

[0078] “Update” means to recalculate the average semantic feature vector of the cluster after adding a semantic feature vector.

[0079] Step D, determining whether the semantic feature vectors in the semantic feature vector set to be processed have been processed, if not, returning to step C, otherwise, outputting the clustering result.

[0080] Step S203: extract keywords from all semantic clusters, and form a semantic feature queue corresponding to each semantic cluster according to the extracted keywords.

[0081] In this embodiment, a preset number of keywords are extracted from each semantic cluster, and methods for extracting keywords include but are not limited to TF-IDF (Term Frequency-Inverse Document Frequency) algorithm, LDA algorithm, etc.

[0082] The extracted keywords are sorted by similarity to form a semantic feature sequence. Specifically, the similarity between the keywords of each semantic cluster is calculated, the keywords are sorted according to the similarity to obtain a sorting result, and a semantic feature queue corresponding to each semantic cluster is generated based on the sorting result.

[0083] The similarity between keywords in each semantic cluster can be calculated by using similarity algorithms including but not limited to cosine similarity, Levenshtein distance, etc. After calculating the similarity between keywords, the keywords are arranged in ascending order according to the similarity to form a semantic feature queue of the semantic cluster arranged from high to low similarity.

[0084] Step S204: Select a keyword from each semantic feature queue as a target keyword, and generate a word semantic vector based on the target keyword.

[0085] In this embodiment, the text classification model is trained by reinforcement learning. Before training, the following definitions are made in advance:

[0086] Define each action of reinforcement learning: Before each round of training, a keyword is randomly selected from each semantic feature queue, and its corresponding vector is obtained using the above semantic feature extraction model, which is then multiplied by a preset coefficient.

[0087] Define the classification reward value: The predicted label category value divided by the correct label value of the sample, and then multiplied by the inverse of the proportion of its category in the overall sample, as the classification reward value. For example, suppose there are 5 categories of samples, category 1 accounts for 1 / 10, category 2 accounts for 1 / 5, category 3 accounts for 1 / 4, category 4 accounts for 1 / 3, and category 5 accounts for 7 / 60. Then the reward coefficient for category 1 is 10, the reward coefficient for category 2 is 5, the reward coefficient for category 3 is 4, the reward coefficient for category 4 is 3, and the reward coefficient for category 5 is about 8.57.

[0088] Before each round of training, a preset number of keywords are randomly selected from each semantic feature queue as target keywords, specifically, one keyword is randomly selected.

[0089] To extract features of the target keyword, the above semantic feature extraction model can be used to obtain a vector corresponding to the target keyword, and the vector is multiplied by a preset coefficient to obtain a word semantic vector corresponding to the target keyword.

[0090] It should be noted that the preset coefficient is the reciprocal of the absolute value of the silhouette coefficient of the semantic cluster where the target keyword is located.

[0091] The keyword vector and the semantic feature vector obtained by the semantic feature extraction model are concatenated to obtain the word semantic vector.

[0092] In this embodiment, by concatenating keyword vectors selected from different semantic clusters with semantic feature vectors and using the obtained word semantic vectors to train the classification model, the classification model can learn different semantic features in the training samples and improve the accuracy of model classification.

[0093] Step S205, inputting the word semantic vector into the pre-built initial classification model for training to obtain a trained target classification model.

[0094] In this embodiment, the pre-built initial classification model is a multi-layer neural network model corresponding to N actions. The word semantic vector is input into the multi-layer neural network model, and the multi-layer neural network model is trained and updated according to the classification reward value to maximize the classification reward value, where N is a natural number greater than zero.

[0095] As a specific implementation method, the multi-layer neural network model includes an input layer, a first hidden layer, a second hidden layer and an output layer. The input layer inputs a word semantic vector v; the weight matrix of the first hidden layer is set to w1, the relu activation function is adopted, and the bias is b1, then the output of the first hidden layer is o1=relu(w1*v+b1); the weight matrix of the second hidden layer is set to w2, the relu activation function is adopted, and the bias is b2, then the output of the second hidden layer is o2=relu(w2*o1+b2); the output layer adopts a softmax layer, o2 is input into the softmax layer, and o3 is obtained through the softmax layer, and o3 is the category probability Pa predicted for each training.

[0096] In order to achieve better text classification effect, more hidden layers can be set according to actual conditions.

[0097] Step S206, obtaining the text to be classified, inputting the text to be classified into the target classification model, and outputting the text classification result.

[0098] Once the text to be classified is obtained, it can be input into the target classification model and the text classification result can be output.

[0099] The present application extracts semantic features from a training text corpus and clusters the extracted semantic feature vectors to obtain different semantic clusters. The classification model is then trained based on the different semantic clusters so that the classification model learns the semantic features of different categories in the training text corpus, thereby improving the accuracy of text classification.

[0100] In some optional implementations of this embodiment, before the step of inputting the semantic feature vector into the trained clustering model and outputting the semantic cluster, the following step is further included:

[0101] Input the semantic feature vector into the pre-built neural network model and output the clustering result;

[0102] Determine the clustering loss function according to the clustering results;

[0103] Adjusting model parameters of the neural network model based on the clustering loss function;

[0104] When the iteration end condition is met, a clustering model is generated according to the model parameters.

[0105] In this embodiment, the pre-built neural network model can adopt the same structure as the above classification model, including an input layer, a first hidden layer, a second hidden layer and an output layer, the input layer inputs the semantic feature vector x; the weight matrix of the first hidden layer is set to w1, the relu activation function is used, and the bias is b1, then the output of the first hidden layer is o1=relu(w1*x+b1); the weight matrix of the second hidden layer is set to w2, the relu activation function is used, and the bias is b2, then the output of the second hidden layer is o2=relu(w2*o1+b2); the output layer adopts a softmax layer, o2 is input into the softmax layer, and o3 is obtained through the softmax layer, and o3 is the probability Pc of each action. Different hidden layers can also be set according to actual needs.

[0106] In this embodiment, the model parameters of the neural network model are adjusted based on the loss function, and when the iteration end condition is met, a clustering model is generated according to the model parameters.

[0107] Specifically, the model parameters of the neural network model are adjusted based on the loss function value of the loss function, and iterative training is continued. When the model is trained to a certain extent, the performance of the model reaches the optimal state, and the loss function value hardly changes, that is, convergence. When the iteration end condition is met, the model converges. After the model converges, the final neural network model is output as a clustering model according to the final adjusted model parameters.

[0108] This application uses a pre-built neural network model trained through reinforcement learning as a clustering model, which can improve the accuracy of clustering and further improve the efficiency of text classification.

[0109] In some optional implementations, the step of determining the clustering loss function according to the clustering result includes:

[0110] Calculate the silhouette coefficient of each cluster in the clustering results;

[0111] Get the training bonus score based on the silhouette coefficient;

[0112] The clustering loss function is obtained based on the clustering results and the training reward score.

[0113] The silhouette coefficient can be calculated using the silhouette coefficient, which is an indicator used to measure clustering effect in clustering algorithms.

[0114] For each semantic feature vector o in cluster D, calculate the relationship between o and other objects in the cluster to which o belongs. The average distance between them, a(o), is calculated as follows:

[0115]

[0116] b(o) is the minimum average distance from o to all clusters that do not contain o, and the formula is as follows:

[0117]

[0118] The silhouette coefficient is:

[0119]

[0120] In this embodiment, the clustering model is trained by reinforcement learning. Before training, the following definitions are performed in advance:

[0121] The reward value is defined as: when the silhouette coefficient is less than the preset threshold Tg, a reward value of 1+1 / |s(o)| is given; otherwise, a reward value of -|s(o)| is given.

[0122] At the end of each training cycle, the training reward score S1 up to that moment is calculated using the following formula:

[0123]

[0124] Where γ is the profit attenuation coefficient, n is the number of training cycles, i = 1 to (n-1), S t is the reward score obtained in the tth training cycle, that is, when the silhouette coefficient is less than the preset threshold Tg, S t =1+1 / |s(o)|; otherwise S t =-|s(o)|.

[0125] In an embodiment, a clustering loss function is obtained based on the clustering result and the training reward score, specifically including:

[0126] Calculate the logarithm of the clustering result, multiply the logarithm by the training reward score to get the product value, and take the negative value of the product value as the clustering loss function. The calculation method of the clustering loss function is as follows:

[0127] Loss = -S1×logPc t ;

[0128] Among them, Pc t represents the probability of the action in round t.

[0129] In this embodiment, the model is iteratively updated by using the clustering loss function obtained through the clustering results and the training reward score to maximize the training reward score and ensure the accuracy of the model.

[0130] In some optional implementations of this embodiment, the step of inputting the word semantic vector into the pre-built initial classification model for training to obtain the trained target classification model includes:

[0131] Input the word semantic vector into the pre-built initial classification model to obtain the predicted classification result;

[0132] Determine the classification loss function based on the predicted classification results;

[0133] Adjusting the model parameters of the initial classification model according to the classification loss function;

[0134] When the iteration end condition is met, a target classification model is generated based on the model parameters.

[0135] Specifically, the model parameters of the initial classification model are adjusted based on the loss function value of the classification loss function, and iterative training is continued. When the model is trained to a certain extent, the performance of the model reaches the optimal state, and the loss function value hardly changes, that is, convergence. When the iteration end condition is met, the model converges. After the model converges, the final classification model is output as the target classification model according to the final adjusted model parameters.

[0136] In this embodiment, the classification model is trained based on the word semantic vectors obtained by concatenating the keyword vectors and semantic feature vectors of different semantic clusters, so that the classification model can learn the different categories and implicit semantic features in the training text corpus, and further improve the accuracy of text classification.

[0137] In this embodiment, the step of determining the classification loss function according to the predicted classification result includes:

[0138] The classification reward value is calculated based on the predicted classification result;

[0139] The classification loss function is obtained based on the classification reward value and the predicted classification result.

[0140] Specifically, the logarithm of the predicted classification result Pa is calculated, the logarithm is multiplied by the classification reward value S2 to obtain the product value, and the negative value of the product value is taken as the classification loss function. The calculation formula of the classification loss function is as follows:

[0141] Loss=-S2×logPa t ;

[0142] Among them, Pa t Represents the classification label probability of the t-th round of training output.

[0143] This embodiment performs iterative model updates by predicting the classification result and the classification loss function obtained by the classification reward value, so as to maximize the classification reward value, ensure the accuracy of the classification model, and improve the accuracy of text classification.

[0144] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0145] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through computer-readable instructions, and the computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a non-volatile storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0146] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0147] Further references Figure 3 , as a response to the above Figure 2 The present application provides an embodiment of a text classification device based on reinforcement learning, and the device embodiment is similar to Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0148] like Figure 3As shown, the text classification device 300 based on reinforcement learning described in this embodiment includes: a semantic feature extraction module 301, a clustering module 302, a keyword extraction module 303, a vector generation module 304, a training module 305 and a classification module 306. Among them:

[0149] The semantic feature extraction module 301 is used to obtain a training text corpus, perform semantic feature extraction on the training text corpus, and obtain a semantic feature vector;

[0150] The clustering module 302 is used to input the semantic feature vector into the trained clustering model and output semantic clusters;

[0151] The keyword extraction module 303 is used to extract keywords from all the semantic clusters, and form a semantic feature queue corresponding to each semantic cluster according to the extracted keywords;

[0152] The vector generation module 304 is used to select a keyword from each of the semantic feature queues as a target keyword, and generate a word semantic vector based on the target keyword;

[0153] The training module 305 is used to input the word semantic vector into a pre-built initial classification model for training to obtain a trained target classification model;

[0154] The classification module 306 is used to obtain the text to be classified, input the text to be classified into the target classification model, and output the text classification result.

[0155] The above-mentioned text classification device based on reinforcement learning extracts semantic features from training text corpus and clusters the extracted semantic feature vectors to obtain semantic clusters of different categories, and then uses the semantic clusters of different categories to train the classification model, so that the classification model learns the semantic features of different categories in the training text corpus, which can improve the accuracy of text classification.

[0156] In some optional implementations of this embodiment, the text classification device 300 based on reinforcement learning further includes a clustering training module, including a clustering submodule, a calculation submodule, an adjustment submodule and a generation submodule, wherein:

[0157] The clustering submodule is used to input the semantic feature vector into a pre-built neural network model and output a clustering result;

[0158] The calculation submodule is used to determine the clustering loss function according to the clustering result;

[0159] The adjustment submodule is used to adjust the model parameters of the neural network model based on the clustering loss function;

[0160] The generation submodule is used to generate a clustering model according to the model parameters when the iteration end condition is met.

[0161] This embodiment uses reinforcement learning to train a pre-built neural network model as a clustering model, which can improve the accuracy of clustering and further improve the efficiency of text classification.

[0162] In this embodiment, the calculation submodule is further used for:

[0163] Calculating the silhouette coefficient of each cluster in the clustering result;

[0164] Obtaining a training reward score according to the silhouette coefficient;

[0165] The clustering loss function is obtained based on the clustering result and the training reward score.

[0166] In this embodiment, the model is iteratively updated by using the clustering loss function obtained through the clustering results and the training reward score to maximize the training reward score and ensure the accuracy of the model.

[0167] In some optional implementations of this embodiment, the keyword extraction module 303 includes a similarity calculation submodule, a sorting submodule and a generation submodule, wherein:

[0168] The similarity calculation submodule is used to calculate the similarity between the keywords of each semantic cluster;

[0169] The sorting submodule is used to sort the keywords according to the similarity to obtain a sorting result;

[0170] The generating submodule is used to generate a semantic feature queue corresponding to each semantic cluster based on the sorting result.

[0171] In this embodiment, the vector generation module 304 includes an extraction submodule and a splicing submodule, wherein:

[0172] The extraction submodule is used to extract features of the target keyword to obtain a keyword vector;

[0173] The concatenation submodule is used to concatenate the keyword vector and the semantic feature vector to obtain a word semantic vector.

[0174] In this embodiment, by concatenating keyword vectors selected from different semantic clusters with semantic feature vectors and using the obtained word semantic vectors to train the classification model, the classification model can learn different semantic features in the training samples and improve the accuracy of model classification.

[0175] In some optional implementations of this embodiment, the training module 305 includes a classification submodule, a calculation submodule, an adjustment submodule and an output submodule, wherein:

[0176] The classification submodule is used to input the word semantic vector into the pre-built initial classification model to obtain a predicted classification result;

[0177] The calculation submodule is used to determine the classification loss function according to the predicted classification result;

[0178] The adjustment submodule is used to adjust the model parameters of the initial classification model according to the classification loss function;

[0179] The output submodule is used to generate a target classification model based on the model parameters when the iteration end condition is met.

[0180] In this embodiment, the classification model is trained based on the word semantic vectors obtained by concatenating the keyword vectors and semantic feature vectors of different semantic clusters, so that the classification model can learn the different categories and implicit semantic features in the training text corpus, and further improve the accuracy of text classification.

[0181] In this embodiment, the calculation submodule is further used for:

[0182] Calculate a classification reward value according to the predicted classification result;

[0183] A classification loss function is obtained based on the classification reward value and the predicted classification result.

[0184] This embodiment performs iterative model updates by predicting the classification result and the classification loss function obtained by the classification reward value, so as to maximize the classification reward value, ensure the accuracy of the classification model, and improve the accuracy of text classification.

[0185] To solve the above technical problems, the present application also provides a computer device. Figure 4 , Figure 4 This is a basic structural block diagram of the computer device in this embodiment.

[0186] The computer device 4 includes a memory 41, a processor 42, and a network interface 43 that are interconnected through a system bus. It should be noted that the figure only shows a computer device 4 with components 41-43, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (Application Specific Integrated Circuit, ASIC), programmable gate arrays (Field-Programmable Gate Array, FPGA), digital processors (Digital Signal Processor, DSP), embedded devices, etc.

[0187] The computer device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The computer device may interact with a user through a keyboard, a mouse, a remote controller, a touch pad, or a voice control device.

[0188] The memory 41 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 41 can be an internal storage unit of the computer device 4, such as a hard disk or memory of the computer device 4. In other embodiments, the memory 41 can also be an external storage device of the computer device 4, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (FlashCard), etc. equipped on the computer device 4. Of course, the memory 41 can also include both the internal storage unit of the computer device 4 and its external storage device. In this embodiment, the memory 41 is generally used to store the operating system and various application software installed on the computer device 4, such as computer-readable instructions of a text classification method based on reinforcement learning, etc. In addition, the memory 41 can also be used to temporarily store various types of data that have been output or are to be output.

[0189] The processor 42 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip in some embodiments. The processor 42 is generally used to control the overall operation of the computer device 4. In this embodiment, the processor 42 is used to run computer-readable instructions stored in the memory 41 or process data, such as computer-readable instructions for running the text classification method based on reinforcement learning.

[0190] The network interface 43 may include a wireless network interface or a wired network interface. The network interface 43 is generally used to establish a communication connection between the computer device 4 and other electronic devices.

[0191] This embodiment implements the steps of the text classification method based on reinforcement learning in the above-mentioned embodiment when the processor executes computer-readable instructions stored in the memory. By extracting semantic features from the training text corpus and clustering the extracted semantic feature vectors to obtain different semantic clusters, the classification model is trained according to the different semantic clusters, so that the classification model learns the semantic features of different categories in the training text corpus, which can improve the accuracy of text classification.

[0192] The present application also provides another implementation mode, namely, providing a computer-readable storage medium, wherein the computer-readable storage medium stores computer-readable instructions, and the computer-readable instructions can be executed by at least one processor so that the at least one processor executes the steps of the text classification method based on reinforcement learning as described above, by extracting semantic features from a training text corpus, and clustering the extracted semantic feature vectors to obtain different semantic clusters, and then training a classification model according to the different semantic clusters, so that the classification model learns semantic features of different categories in the training text corpus, thereby improving the accuracy of text classification.

[0193] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0194] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to perform equivalent replacement of some of the technical features therein. Any equivalent structure made using the contents of the specification and drawings of this application, directly or indirectly used in other related technical fields, is similarly within the scope of patent protection of this application.

Claims

1. A text classification method based on reinforcement learning, characterized in that: The steps include: Acquire a training text corpus, perform semantic feature extraction on the training text corpus, and obtain a semantic feature vector; Inputting the semantic feature vector into a trained clustering model and outputting a semantic cluster; Extracting keywords from all the semantic clusters, and forming a semantic feature queue corresponding to each semantic cluster according to the extracted keywords; Selecting a keyword from each of the semantic feature queues as a target keyword, and generating a word semantic vector based on the target keyword; Inputting the word semantic vector into a pre-built initial classification model for training to obtain a trained target classification model; Obtaining a text to be classified, inputting the text to be classified into the target classification model, and outputting a text classification result; Wherein, before the step of inputting the semantic feature vector into the trained clustering model and outputting the semantic cluster, the step further includes: Inputting the semantic feature vector into a pre-built neural network model and outputting a clustering result; Determine a clustering loss function according to the clustering result; The step of determining the clustering loss function according to the clustering result comprises: Calculating the silhouette coefficient of each cluster in the clustering result; Obtaining a training reward score according to the silhouette coefficient; The clustering loss function is obtained based on the clustering result and the training reward score.

2. The text classification method based on reinforcement learning according to claim 1, characterized in that: Before the step of inputting the semantic feature vector into the trained clustering model and outputting the semantic cluster, the step further includes: Adjusting model parameters of the neural network model based on the clustering loss function; When the iteration end condition is met, a clustering model is generated according to the model parameters.

3. The text classification method based on reinforcement learning according to claim 1, characterized in that: The step of forming a semantic feature queue corresponding to each semantic cluster according to the extracted keywords comprises: Calculating the similarity between the keywords of each semantic cluster; Sorting the keywords according to the similarity to obtain a sorting result; A semantic feature queue corresponding to each semantic cluster is generated based on the sorting result.

4. The text classification method based on reinforcement learning according to claim 1, characterized in that: The step of generating a word semantic vector based on the target keyword comprises: Performing feature extraction on the target keyword to obtain a keyword vector; The keyword vector and the semantic feature vector are concatenated to obtain a word semantic vector.

5. The text classification method based on reinforcement learning according to claim 1, characterized in that: The step of inputting the word semantic vector into a pre-built initial classification model for training to obtain a trained target classification model comprises: Inputting the word semantic vector into a pre-built initial classification model to obtain a predicted classification result; Determine a classification loss function according to the predicted classification result; Adjusting the model parameters of the initial classification model according to the classification loss function; When the iteration end condition is met, a target classification model is generated based on the model parameters.

6. The text classification method based on reinforcement learning according to claim 5, characterized in that: The step of determining the classification loss function according to the predicted classification result comprises: Calculate a classification reward value according to the predicted classification result; A classification loss function is obtained based on the classification reward value and the predicted classification result.

7. A text classification device based on reinforcement learning, characterized in that: include: A semantic feature extraction module is used to obtain a training text corpus, perform semantic feature extraction on the training text corpus, and obtain a semantic feature vector; A clustering module, used for inputting the semantic feature vector into a trained clustering model and outputting a semantic cluster; A keyword extraction module, used to extract keywords from all the semantic clusters, and form a semantic feature queue corresponding to each semantic cluster according to the extracted keywords; A vector generation module, used to select a keyword from each of the semantic feature queues as a target keyword, and generate a word semantic vector based on the target keyword; A training module, used for inputting the word semantic vector into a pre-built initial classification model for training to obtain a trained target classification model; A classification module, used to obtain a text to be classified, input the text to be classified into the target classification model, and output a text classification result; The text classification device based on reinforcement learning further includes a clustering training module, which includes a clustering submodule and a calculation submodule, wherein: The clustering submodule is used to input the semantic feature vector into a pre-built neural network model and output a clustering result; The calculation submodule is used to determine the clustering loss function according to the clustering result; The calculation submodule is further used for: Calculating the silhouette coefficient of each cluster in the clustering result; Obtaining a training reward score according to the silhouette coefficient; The clustering loss function is obtained based on the clustering result and the training reward score.

8. A computer device comprising a memory and a processor, wherein the memory stores computer-readable instructions, and when the processor executes the computer-readable instructions, the steps of the text classification method based on reinforcement learning as described in any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by a processor, the steps of the text classification method based on reinforcement learning as described in any one of claims 1 to 6 are implemented.

Images