Text classification model training method and device, computer device and medium

Abstract

This invention relates to the field of artificial intelligence technology, and particularly to a training method, apparatus, computer device, and medium for a text classification model. The invention encodes terms in text samples and concatenates the word encoding vectors to obtain a text encoding vector. This text encoding vector is then input into a term category probability prediction model to obtain a term category probability vector. All term category probability vectors are concatenated to obtain a text smoothing vector. The text encoding vector and the text smoothing vector are fused according to a preset fusion coefficient to obtain a text mixture vector. The text encoding vector and the text mixture vector are then input into a text classification model to obtain the predicted category of the text sample. The text classification model is trained based on the predicted and actual categories to obtain a trained text classification model. When the number of training samples is small, the obtained text encoding vectors are used to augment the text sample data, improving the accuracy of the text classification model.

Description

TECHNICAL FIELD

[0001] The present application is suitable for the field of artificial intelligence, and in particular relates to a text classification model training method and device, computer equipment and a medium. BACKGROUND

[0002] In the text classification scenario, generally, on the basis of keeping the label unchanged, a language model is used to replace the word items in the original text to realize text data enhancement, expand the training samples of the text classification model, and thus improve the classification accuracy of the trained text classification model. However, the word replacement method needs to ensure that the word used for replacement belongs to a natural language. In the case of keeping the label unchanged, the word used for replacement is extremely limited, and in the case of fewer training samples, it is impossible to provide enough enhanced samples to improve the accuracy of the text classification model, resulting in a low accuracy of the text classification model.

[0003] Therefore, in the field of text classification technology, how to improve the accuracy of the text classification model in the case of fewer training samples has become a problem to be solved. SUMMARY

[0004] Therefore, the embodiments of the present application provide a text classification model training method and device, computer equipment and a medium to solve the problem of low accuracy of the text classification model in the case of fewer training samples.

[0005] In a first aspect, the embodiments of the present application provide a text classification model training method, which comprises:

[0006] Obtaining a text sample with a known actual category, encoding at least two word items in the text sample to obtain a word encoding vector corresponding to the word item;

[0007] Splicing the word encoding vectors of all word items to obtain a text encoding vector of the text sample;

[0008] Inputting the text encoding vector into a trained word item category probability prediction model to obtain a word item category probability vector corresponding to each word item in the text encoding vector, and splicing all word item category probability vectors to obtain a text smoothing vector;

[0009] Performing a fusion operation on the text encoding vector and the text smoothing vector according to a preset fusion coefficient to obtain a text hybrid vector;

[0010] Inputting the text encoding vector and the text hybrid vector into a text classification model to obtain a predicted category of the text sample, training the text classification model according to the predicted category and the actual category to obtain a trained text classification model.

[0011] In a second aspect, an embodiment of the present application provides a text classification model training device, the text classification model training device comprising:

[0012] a word item encoding module configured to obtain a text sample with a known actual category, encode at least two word items in the text sample to obtain word encoding vectors corresponding to the word items;

[0013] a vector splicing module configured to splice the word encoding vectors of all the word items to obtain a text encoding vector of the text sample;

[0014] a probability prediction module configured to input the text encoding vector into a trained word item category probability prediction model to obtain a word item category probability vector corresponding to each word item in the text encoding vector, splice all the word item category probability vectors to obtain a text smoothing vector;

[0015] a vector mixing module configured to perform a fusion operation on the text encoding vector and the text smoothing vector according to a preset fusion coefficient to obtain a text mixing vector;

[0016] a model training module configured to input the text encoding vector and the text mixing vector into a text classification model to obtain a predicted category of the text sample, train the text classification model according to the predicted category and the actual category to obtain a trained text classification model.

[0017] In a third aspect, an embodiment of the present application provides a computer device, the computer device comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, and the processor implements the text classification model training method according to the first aspect when executing the computer program.

[0018] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, the computer readable storage medium storing a computer program, and the computer program implements the text classification model training method according to the first aspect when executed by a processor.

[0019] Compared with the prior art, the embodiment of the present application has the beneficial effect that: by encoding at least two word items in the obtained text sample of the known actual category to obtain the word encoding vector corresponding to the word item, then splicing the word encoding vectors of all word items to obtain the text encoding vector of the text sample, and inputting the text encoding vector into the trained word item category probability prediction model to obtain the word item category probability vector corresponding to each word item in the text encoding vector, splicing all word item category probability vectors to obtain the text smoothing vector, further performing fusion operation on the text encoding vector and the text smoothing vector according to the preset fusion coefficient to obtain the text hybrid vector, and finally inputting the text encoding vector and the text hybrid vector into the text classification model to obtain the predicted category of the text sample, training the text classification model according to the predicted category and the actual category to obtain the trained text classification model. In the case of less training samples, the text sample data is enhanced by the obtained text encoding vector, the smoothing text vector is obtained and used as the input of the text classification model, and the accuracy of the text classification model is improved. BRIEF DESCRIPTION OF DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0021] Figure 1 is an application environment schematic diagram of a text classification model training method provided by the first embodiment of the present application;

[0022] Figure 2 is a flow schematic diagram of a text classification model training method provided by the first embodiment of the present application;

[0023] Figure 3 is a structure schematic diagram of a text classification model training device provided by the second embodiment of the present application;

[0024] Figure 4 is a structure schematic diagram of a computer device provided by the third embodiment of the present application. DETAILED DESCRIPTION

[0025] In the following description, specific details are set forth in order to provide a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application can be practiced without these specific details. In other instances, well-known systems, devices, circuits and methods have been described without unnecessary detail in order to avoid obscuring the present application.

[0026] It will be understood that, when used in the specification and the appended claims, the terms "comprise", "comprising", "comprises" and "comprising" indicate the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0027] It will be further understood that the terms "comprise", "comprising", "comprises" and "comprising" when used in this specification and the appended claims specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0028] As used in the specification and the appended claims, the term "if' can be construed to mean "when" or "once" or "in response to determining" or "in response to detecting" depending on the context. Similarly, the phrase "if it is determined" or "if [a described condition or event] is detected" can be construed to mean "once it is determined" or "in response to determining" or "once [the described condition or event] is detected" or "in response to detecting [a described condition or event]," depending on the context.

[0029] In addition, the terms "first", "second", "third", etc. as used in the description of the specification and the appended claims are merely to distinguish descriptions and are not to be construed to indicate or imply relative importance.

[0030] Reference in the specification to "one embodiment" or "some embodiments" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the application. The appearances of the phrase "in one embodiment" or "in some embodiments" in various places in the specification are not necessarily all referring to the same embodiment, although it can. The terms "including," "comprising," "having" and variations thereof are meant to encompass the items listed thereafter and equivalents thereof as well as additional items.

[0031] Embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Wherein, artificial intelligence (AI) is to use digital computers or digital computer controlled machines to simulate, extend and expand human intelligence, to perceive environment, acquire knowledge and use knowledge to obtain the best results.

[0032] The artificial intelligence basic technology generally includes technologies such as sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction system, mechatronics, etc. The artificial intelligence software technology mainly includes computer vision technology, robot technology, biometric identification technology, speech processing technology, natural language processing technology, and machine learning / deep learning, etc.

[0033] It should be understood that the size of the serial number of each step in the following embodiments does not mean the order of execution, and the execution order of each process should be determined according to its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0034] In order to illustrate the technical solutions of the present application, the following will be illustrated by specific embodiments.

[0035] The training method of the text classification model provided by the embodiment one of the present application can be applied in the application environment such as Figure 1 , wherein the client and the server communicate. The client includes but is not limited to palmtop computer, desktop computer, notebook computer, ultra-mobile personal computer (UMPC), netbook, cloud computer device, personal digital assistant (PDA) and other computer devices. The server can be realized by an independent server or a server cluster composed of multiple servers.

[0036] Referring to Figure 2 , it is a flowchart of the training method of the text classification model provided by the embodiment one of the present application, and the training method can be applied to the client in Figure 1 , and the training method can include the following steps:

[0037] In step S201, a text sample with known actual category is obtained, and at least two word items in the text sample are encoded to obtain a word encoding vector corresponding to the word item.

[0038] The text sample can be a sentence, a paragraph or an article, etc. By splitting the text sample, at least two word items constituting the text sample can be obtained.

[0039] The actual categories can be set according to actual classification requirements of the text samples. For example, when the text samples need to be classified according to prosodic emotions in the text samples, the actual categories can be categories such as Ping, Que, Shu, Mi, Ji and Huan representing text prosodic emotions; when the text samples need to be classified according to emotions contained in the text samples, the actual categories can be categories such as positive emotion, negative emotion, affirmative emotion, negative emotion, positive emotion and negative emotion representing text contained emotions.

[0040] Specifically, after obtaining the text samples with known actual categories, in order to facilitate the computer to process the text samples, the text samples need to be digitized. Since the text samples are composed of a plurality of word items, in order to improve the efficiency of coding and reduce the difficulty of coding, in the embodiment, the text samples are first split to obtain at least two word items constituting the text samples, and each word item is coded to digitize each word item and obtain a word coding vector of each word item, so as to facilitate the computer to process the text samples.

[0041] Optionally, coding the at least two word items in the text sample to obtain a word coding vector corresponding to the word item comprises:

[0042] performing word segmentation processing on the text sample to obtain N word items in the text sample, wherein N is an integer greater than 2;

[0043] coding each word item to obtain a word coding vector of each word item.

[0044] Since the text sample is composed of a plurality of word items, in the embodiment, in order to improve the efficiency of coding and reduce the difficulty of coding, the text sample is first processed by a text segmentation method to obtain N word items constituting the text sample, and N is an integer greater than 2. The text segmentation method can be, for example, a shortest path method, a string matching segmentation algorithm or a minimum segmentation method.

[0045] Then, in order to facilitate the computer to process the text sample, in the embodiment, each word item obtained is one-hot coded to obtain a word coding vector corresponding to the word item. Specifically, in the one-hot coding process, the number of preset word items in the preset word library is denoted as M, the preset word item consistent with each word item is determined by a character matching algorithm, and a vector element at a position corresponding to the preset word item consistent with each word item is set to 1, and vector elements at other positions are set to 0, to obtain a M*1-dimensional word coding vector corresponding to the word item.

[0046] For example, the preset word library includes preset word item A', preset word item B' and preset word item C', the number of preset word items is 3, and according to [A', B', C'] TThe text sample includes the word item A and the word item B, and according to the character matching algorithm, the word item A is consistent with the preset word item A', and the word item B is consistent with the preset word item B'. Therefore, the word item coding vector corresponding to the word item A is [1, 0, 0] T The word item coding vector corresponding to the word item B is [0, 1, 0] T That is, the 3*1-dimensional word coding vector is obtained.

[0047] The above obtains the text sample of the known actual category, encodes at least two word items in the text sample to obtain the word coding vector corresponding to the word item, and improves the coding efficiency and reduces the coding difficulty by performing the word segmentation on the text sample of the known actual category. The text content is digitized, which is convenient for the computer to perform subsequent processing on the text sample, and improves the efficiency of text classification.

[0048] In step S202, the word coding vectors of all word items are spliced to obtain the text coding vector of the text sample.

[0049] In the encoding process, in order to improve the coding efficiency and reduce the coding difficulty, the text sample is first subjected to word segmentation and word item encoding to obtain the word coding vector of each word item constituting the text sample.

[0050] Then, when classifying the text sample, the text sample needs to be analyzed as a whole. In this embodiment, in order to ensure the integrity of the coding vector of the text sample, the word coding vectors of all word items constituting the text sample are further spliced to obtain the text coding vector of the text sample, which is used for subsequent classification of the text sample to improve the accuracy of text classification.

[0051] Optionally, the word coding vectors of all word items are spliced to obtain the text coding vector of the text sample, including:

[0052] According to the order of the N word items in the text sample, the N word coding vectors are sorted;

[0053] According to the sorting result, the N word coding vectors are spliced into a text coding vector of a preset dimension.

[0054] In the word segmentation process, the order of the obtained N word items in the text sample is first determined, and then the N word coding vectors constituting the text sample are sorted, and then the N word coding vectors are spliced into a text coding vector of a preset dimension according to the sorting result. When the dimension of each word coding vector is M*1, the dimension of the text coding vector spliced by the N word coding vectors is the preset dimension M*N.

[0055] For example, the text sample includes word item A and word item B, the number of N is 2, the ranking number of word item A is 1, the word item encoding vector of word item A is [1, 0, 0] T , the ranking number of word item B is 2, and the word item encoding vector of the word item B is [0, 1, 0] T , the preset dimension of the text encoding vector spliced according to the word item encoding vector of the word item A and the word item encoding vector of the word item B is 3*2, and the text encoding vector is [[1, 0, 0] T , [0, 1, 0] T ].

[0056] The step of splicing the word encoding vectors of all word items to obtain the text encoding vector of the text sample guarantees the integrity of the encoding vector of the text sample, serves as a basis for subsequent text sample classification, and improves the accuracy of text classification.

[0057] In step S203, the text encoding vector is input into the trained word item category probability prediction model to obtain a word item category probability vector corresponding to each word item in the text encoding vector. All word item category probability vectors are spliced to obtain a text smoothing vector.

[0058] The dimension of the word item category probability vector is consistent with the dimension of the word item encoding vector. Each probability value in the word item category probability vector represents the probability that the word item is a preset word item at the corresponding position. For example, the word item category probability vector of word item A is [0.7, 0.1, 0.2] T , the preset word items at the positions corresponding to the three probability values 0.7, 0.1, and 0.2 are preset word item A', preset word item B', and preset word item C', respectively. The probability value 0.7 indicates that the probability that the word item A is the preset word item A' is 0.7, the probability value 0.1 indicates that the probability that the word item A is the preset word item B' is 0.1, and the probability value 0.2 indicates that the probability that the word item A is the preset word item C' is 0.2.

[0059] In the case of fewer training samples, in order to improve the accuracy of the text classification model, the text encoding vector is input into the trained word item category probability prediction model to obtain a word item category probability vector corresponding to each word item in the text encoding vector. The text encoding vector can still represent the semantic information of the text sample, and the text sample data is enhanced accordingly.

[0060] Then, after obtaining the word item category probability vector corresponding to each word item, all word item category probability vectors are further spliced to obtain a text smoothing vector, which guarantees the integrity of the text smoothing vector of the text sample, so that the capacity of the text classification model can be utilized to the greatest extent, and the accuracy of the text classification model is improved.

[0061] For example, a text sample includes a word item A and a word item B, the word item category probability vector of the word item A is [0.7, 0.1, 0.2] T , and the word item category probability vector of the word item B is [0.25, 0.7, 0.05] T . The word item category probability vector of the word item A and the word item category probability vector of the word item B are spliced in the corresponding order to obtain the text smoothing vector [[0.7, 0.1, 0.2] T , [0.25, 0.7, 0.05] T .

[0062] Optionally, the word item category probability prediction model includes a first encoder and a first decoder, and the text encoding vector is used as a training sample and the text sample is used as a training label.

[0063] The training process of the word item category probability prediction model includes:

[0064] The text encoding vector is input into the first encoder for feature extraction to obtain first text encoding features;

[0065] The first text encoding features are input into the first decoder to obtain the word item category probability vector of each word item in the text encoding vector;

[0066] A vector composed of all the word item category probability vectors is determined as a text smoothing vector, and the normalization exponential function is used to process in the text smoothing vector to obtain a prediction sample;

[0067] A first loss function is calculated according to the prediction sample and the corresponding text sample, and the parameters of the first encoder and the first decoder are corrected in the reverse direction according to the gradient descent method until the first loss function converges, thereby obtaining the trained word item category probability prediction model.

[0068] The training sample of the word item category probability prediction model is a large number of text encoding vectors, and the training label is the text sample corresponding to the text encoding vector, which is used to evaluate the word item category probability vector obtained by the word item category probability prediction model, so that the trained word item category probability prediction model can be obtained according to the evaluation result.

[0069] In the training process of the word item category probability prediction model, the text encoding vector is first input into the first encoder for feature extraction to obtain first text encoding features, and then the first text encoding features are input into the first decoder to obtain the word item category probability vector of each word item in the text encoding vector, and a vector composed of all the word item category probability vectors is determined as a text smoothing vector, and then the normalization exponential function is used to process in the text smoothing vector to obtain a prediction sample.

[0070] Then, a first loss function is calculated based on the predicted sample and the corresponding text sample. When the first loss function is small, it means that the difference between the predicted sample obtained by the word category probability prediction model and the corresponding text sample is small, indicating that the accuracy of the word category probability prediction model is high. When the first loss function is large, it means that the difference between the predicted sample obtained by the word category probability prediction model and the corresponding text sample is large, indicating that the accuracy of the word category probability prediction model is low. It is necessary to correct the parameters of the first encoder and the first decoder in reverse according to the gradient descent method until the first loss function converges, and the trained word category probability prediction model is obtained.

[0071] Optionally, a first loss function is calculated based on the predicted samples and the corresponding text samples, including:

[0072] The predicted samples are converted into predicted sample vectors using word vector technology, and the text samples are converted into text sample vectors using word vector technology.

[0073] Calculate the text similarity between the predicted sample vector and the text sample vector, and then calculate the first loss function based on the text similarity.

[0074] Among them, word vector technology represents text as a series of vectors that can express the semantics of the text. For example, word2vec (a word vector model) has a higher text similarity between the predicted sample vector and the text sample vector. This means that the difference between the predicted sample obtained by the prediction model based on the word category probability is smaller than that between the corresponding text sample and the predicted sample. In this case, the first loss function is smaller.

[0075] Therefore, by calculating the cosine similarity between the predicted sample vector and the text sample vector, and using it as the text similarity between the predicted sample vector and the text sample vector, the value range of this text similarity is [0,1]. The difference between this text similarity and the value 1 is used as the first loss function to represent the accuracy of the word category probability prediction model.

[0076] The above steps involve inputting the text encoding vector into a trained term category probability prediction model to obtain the term category probability vector corresponding to each term in the text encoding vector, and concatenating all term category probability vectors to obtain the text smoothing vector. By calculating the term category probability vector corresponding to each term and concatenating all term category probability vectors to obtain the text smoothing vector, the text sample data is enhanced, making full use of the capabilities of the text classification model and improving the accuracy of the text classification model.

[0077] Step S204: Perform a fusion operation on the text encoding vector and the text smoothing vector according to the preset fusion coefficient to obtain the text hybrid vector.

[0078] In this model, the text encoding vector and the text smoothing vector have the same vector dimension. The text encoding vector contains the category information of the text sample, while the text smoothing vector contains the semantic information of the text sample after text enhancement. Therefore, by inputting either the text encoding vector or the text smoothing vector into the text classification model, text sample classification can be achieved. In this embodiment, to improve the accuracy of text classification, the text encoding vector and the text smoothing vector are fused according to a preset fusion coefficient to obtain a text hybrid vector. This text hybrid vector contains both category and semantic information of the text sample, thus improving the accuracy of the text classification model.

[0079] The fusion coefficient can be set according to the actual situation. In this embodiment, since the text smoothing vector is obtained after enhancing the text sample data, the fusion coefficient of the text encoding vector is set to 0.4, and the fusion coefficient of the text smoothing vector is set to 0.6.

[0080] Optionally, the text encoding vector and the text smoothing vector are fused according to a preset fusion coefficient to obtain a text blending vector, including:

[0081] Based on the vector coordinates of each encoded vector value in the text encoded vector and the vector coordinates of each smoothed vector value in the text smoothed vector, determine the encoded vector value and smoothed vector value with the same vector coordinates.

[0082] The fused vector value is calculated based on each encoded vector value, the first preset fusion coefficient of each encoded vector value, the smoothed vector value with the same vector coordinates as each encoded vector value, and the second preset fusion coefficient of the smoothed vector value.

[0083] The text blending vector is obtained based on each fusion vector value and its vector coordinates; the vector coordinates of the fusion vector values ​​are the same vector coordinates that the encoded vector values ​​and the smoothing vector values ​​have.

[0084] The text encoding vector is obtained by concatenating the word encoding vectors of all terms, and the text smoothing vector is obtained by concatenating the category probability vectors of all terms. The text encoding vector and the text smoothing vector have the same vector dimension, and the encoding vector values ​​in the text encoding vector correspond one-to-one with the smoothing vector values ​​in the text smoothing vector.

[0085] Therefore, in this embodiment, when performing the fusion operation on the text encoding vector and the text smoothing vector, the encoding vector value and the smoothing vector value with the same vector coordinates are first determined according to the vector coordinates of each encoding vector value in the text encoding vector and the vector coordinates of each smoothing vector value in the text smoothing vector. Then, the fused vector value is calculated according to each encoding vector value, the first preset fusion coefficient of each encoding vector value, the smoothing vector value with the same vector coordinates as each encoding vector value, and the second preset fusion coefficient of the smoothing vector value. The first preset fusion coefficient and the second preset fusion coefficient can be set according to the actual situation. In this embodiment, the first preset fusion coefficient is set to 0.4 and the second preset fusion coefficient is set to 0.6.

[0086] Then, based on each fusion vector value and its vector coordinates, a text blending vector is obtained, where the vector coordinates of the fusion vector values ​​are the same vector coordinates shared by the encoded vector values ​​and the smoothing vector values.

[0087] For example, if the number of terms is N, the preset number of terms is M, the first preset fusion coefficient is α1 = 0.4, the second preset fusion coefficient is α2 = 0.6, and the encoded vector value of the i-th row (i = 1, ..., M) and j-th column (j = 1, ..., N) in the text encoding vector is x. ij The smoothed vector value in the i-th row and j-th column of the text smoothing vector is y. ij Then the encoded vector value x ij and smooth vector value y ij Correspondingly.

[0088] Then calculate the encoded vector value x ij The product of the first preset fusion coefficient α1 and the smoothing vector value y ij The product of the first product and the second preset fusion coefficient α2 is used as the sum of the two products as the fusion vector value r corresponding to the i-th row and j-th column. ij That is, the fusion vector value r in the i-th row and j-th column. ij for:

[0089] r ij =α1x ij +α2y ij

[0090] In the formula, α1 is the first preset fusion coefficient, x ij Let y be the encoded vector value in the i-th row and j-th column, α2 be the second preset fusion coefficient, and y be the encoded vector value in the j-th row and α2 be the encoded vector value in the j-th column. ij Let be the smoothed vector value in the i-th row and j-th column.

[0091] By determining each fusion vector value and its coordinates, the text blending vector can be obtained.

[0092] The above steps, which fuse text encoding vectors and text smoothing vectors according to preset fusion coefficients to obtain a text hybrid vector, improve the accuracy of the text classification model by fusing text encoding vectors containing category information of text samples and text smoothing vectors containing semantic information of enhanced text samples.

[0093] Step S205: Input the text encoding vector and the text mixing vector into the text classification model to obtain the predicted category of the text sample. Train the text classification model based on the predicted category and the actual category to obtain the trained text classification model.

[0094] The text classification model is used to classify text samples corresponding to the input text encoding vector and text mixture vector. Specifically, the text encoding vector and text mixture vector are input into the text classification model. After feature extraction and feature mapping of the text encoding vector and text mixture vector, the predicted category of the text sample is obtained. At the same time, in order to improve the accuracy of the text classification model, the text classification model is trained based on the predicted category and the actual category to obtain a trained text classification model.

[0095] Optionally, the text classification model includes a second encoder, a third encoder, and a fully connected layer. The text encoding vector and the text mixture vector are used as training samples, and the actual category of the training samples is used as the training label to train the text classification model, thus obtaining a trained text classification model.

[0096] The text classification model includes a second encoder for extracting features from the text encoding vector, a third encoder for extracting features from the text mixture vector, and a fully connected layer for mapping the extracted features to obtain the predicted category of the text sample.

[0097] The training samples consist of a large number of text encoding vectors and text mixture vectors. The training labels are the actual categories of the text samples corresponding to the text encoding vectors and text mixture vectors. These labels are used to evaluate the predicted categories obtained from text classification, so that the text classification model can be trained based on the evaluation results to obtain a well-trained text classification model.

[0098] Optionally, the training process for the text classification model includes:

[0099] The text encoding vector is input into the second encoder for feature extraction to obtain the second text encoding features;

[0100] The text blending vector is input into a third encoder for feature extraction to obtain the text blending features;

[0101] The second text encoding feature and the text fusion feature are fused to obtain the text fusion feature. The text fusion feature is then input into the fully connected layer to obtain the predicted category.

[0102] The second loss function is calculated based on the predicted category and the corresponding actual category. The parameters of the second encoder, the third encoder, and the fully connected layer are then corrected in reverse using the gradient descent method until the second loss function converges, resulting in a trained text classification model.

[0103] In the training process of the text classification model, text encoding vectors and text mixture vectors are input into different encoders to extract different features, so as to comprehensively extract the content features of the corresponding text samples. Specifically, the text encoding vectors are input into the second encoder for feature extraction to obtain the second text encoding features, and the text mixture vectors are input into the third encoder for feature extraction to obtain the text mixture features. The extracted second text encoding features and text mixture features are then fused to obtain the text fusion features, which are then input into the fully connected layer to obtain the predicted category.

[0104] Then, a second loss function is calculated based on the predicted category and the corresponding actual category. When the second loss function is small, it means that the difference between the predicted category and the actual category obtained by the text classification model is small, indicating that the accuracy of the text classification model is high. Conversely, when the second loss function is large, it means that the difference between the predicted category and the actual category obtained by the text classification model is large, indicating that the accuracy of the text classification model is low. It is necessary to back-correct the parameters of the second encoder, the third encoder, and the fully connected layer using the gradient descent method until the second loss function converges, thus obtaining the trained text classification model.

[0105] The above steps involve inputting the text encoding vector and the text mixture vector into the text classification model to obtain the predicted category of the text sample, and training the text classification model based on the predicted category and the actual category to obtain a trained text classification model. By extracting different text features from the text encoding vector and the text mixture vector, the content features of the corresponding text sample are comprehensively extracted. The text classification model is then trained based on the predicted category and the actual category to obtain a trained text classification model, thereby improving the accuracy of the text classification model.

[0106] This invention encodes at least two terms in a text sample with a known actual category to obtain a word encoding vector for each term. Then, the word encoding vectors of all terms are concatenated to obtain a text encoding vector for the text sample. This text encoding vector is input into a trained term category probability prediction model to obtain term category probability vectors for each term in the text encoding vector. All term category probability vectors are then concatenated to obtain a smoothed text vector. Furthermore, the text encoding vector and the smoothed text vector are fused according to a preset fusion coefficient to obtain a mixed text vector. Finally, the text encoding vector and the mixed text vector are input into a text classification model to obtain the predicted category of the text sample. The text classification model is then trained based on the predicted category and the actual category to obtain a trained text classification model. When the training samples are limited, the obtained text encoding vectors are used to enhance the text sample data, resulting in a smoothed text vector that serves as input to the text classification model, thus improving the accuracy of the text classification model.

[0107] Corresponding to the training method in the above embodiments, Figure 3 A structural block diagram of the training device for the text classification model provided in Embodiment 2 of the present invention is given. For ease of explanation, only the parts related to the embodiments of the present invention are shown.

[0108] See Figure 3 The training apparatus for this text classification model includes:

[0109] The term encoding module 31 is used to acquire text samples with known actual categories, encode at least two terms in the text samples, and obtain the term encoding vector of the corresponding terms.

[0110] Vector concatenation module 32 is used to concatenate the word encoding vectors of all terms to obtain the text encoding vector of the text sample;

[0111] The probability prediction module 33 is used to input the text encoding vector into the trained word category probability prediction model, obtain the word category probability vector corresponding to each word in the text encoding vector, and concatenate all the word category probability vectors to obtain the text smooth vector.

[0112] The vector blending module 34 is used to perform a blending operation on the text encoding vector and the text smoothing vector according to the preset blending coefficient to obtain the text blending vector;

[0113] The model training module 35 is used to input the text encoding vector and the text mixing vector into the text classification model to obtain the predicted category of the text sample. Based on the predicted category and the actual category, the text classification model is trained to obtain the trained text classification model.

[0114] Optionally, the term encoding module 31 mentioned above includes:

[0115] The text segmentation submodule is used to segment text samples to obtain N word terms from the text samples;

[0116] The term encoding submodule is used to encode each term and obtain the term encoding vector for each term.

[0117] Optionally, the vector concatenation module 32 mentioned above includes:

[0118] The vector sorting submodule is used to sort the corresponding N word encoding vectors according to the order of N word terms in the text sample;

[0119] The vector concatenation submodule is used to concatenate N word encoding vectors into a text encoding vector of a preset dimension according to the sorting result.

[0120] Optionally, the probability prediction module 33 mentioned above includes:

[0121] The prediction model determination submodule is used to determine the probability prediction model of word category. The prediction model includes a first encoder and a first decoder, using text encoded vectors as training samples and text samples as training labels.

[0122] The first feature extraction submodule is used to input the text encoding vector into the first encoder for feature extraction to obtain the first text encoding feature;

[0123] The probability vector prediction submodule is used to input the first text encoding features into the first decoder to obtain the word category probability vector of each word in the text encoding vector;

[0124] The prediction sample determination submodule is used to determine that the vector composed of all word category probability vectors is a text smoothing vector. The normalized exponential function is used to process the text smoothing vector to obtain the prediction sample.

[0125] The first parameter correction submodule is used to calculate the first loss function based on the predicted sample and the corresponding text sample, and to correct the parameters of the first encoder and the first decoder in reverse according to the gradient descent method until the first loss function converges, thus obtaining the trained term category probability prediction model.

[0126] Optionally, the vector blending module 34 mentioned above includes:

[0127] The vector value matching submodule is used to determine the encoded vector value and the smoothed vector value with the same vector coordinates based on the vector coordinates of each encoded vector value in the text encoded vector and the vector coordinates of each smoothed vector value in the text smoothed vector.

[0128] The vector value calculation submodule is used to calculate the fused vector value based on each encoded vector value, the first preset fusion coefficient of each encoded vector value, the smoothed vector value with the same vector coordinates as each encoded vector value, and the second preset fusion coefficient of the smoothed vector value.

[0129] The vector blending submodule is used to obtain the text blending vector based on each blended vector value and its vector coordinates; the vector coordinates of the blended vector values ​​are the same vector coordinates that the encoded vector values ​​and the smoothed vector values ​​have.

[0130] Optionally, the above model training module 35 includes:

[0131] The model training submodule is used to determine that the text classification model includes a second encoder, a third encoder, and a fully connected layer. The text encoding vector and the text mixture vector are used as training samples, and the actual category of the training samples is used as the training label to train the text classification model and obtain the trained text classification model.

[0132] Optionally, the above model training submodule includes:

[0133] The second feature extraction unit is used to input the text encoding vector into the second encoder for feature extraction to obtain the second text encoding features;

[0134] The third feature extraction unit is used to input the text mixing vector into the third encoder for feature extraction to obtain the text mixing features;

[0135] The category prediction unit is used to fuse the second text encoding features and the text fusion features to obtain the text fusion features. The text fusion features are then input into the fully connected layer to obtain the predicted category.

[0136] The second parameter correction unit is used to calculate the second loss function based on the predicted category and the corresponding actual category, and to correct the parameters of the second encoder, the third encoder and the fully connected layer in reverse according to the gradient descent method until the second loss function converges, thus obtaining the trained text classification model.

[0137] It should be noted that the information interaction and execution process between the above modules are based on the same concept as the method embodiments of the present invention. For details on their specific functions and technical effects, please refer to the method embodiments section, which will not be repeated here.

[0138] Figure 4 This is a schematic diagram of the structure of a computer device provided in Embodiment 3 of the present invention. Figure 4 As shown, the computer device of this embodiment includes: at least one processor ( Figure 4Only one is shown in the diagram), a memory, and a computer program stored in the memory and executable on at least one processor, which, when executed by the processor, implements the steps in any of the above-described training method embodiments.

[0139] This computer device may include, but is not limited to, a processor and memory. Those skilled in the art will understand that... Figure 4 The examples of computer devices are merely examples and do not constitute a limitation on computer devices. Computer devices may include more or fewer components than shown in the illustration, or combinations of certain components, or different components, such as network interfaces, displays, and input devices.

[0140] The processor referred to can be a CPU, but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0141] Memory includes readable storage media, internal memory, etc., wherein internal memory can be the RAM of a computer device, providing an environment for the operation of the operating system and computer-readable instructions stored in the readable storage media. The readable storage media can be the hard drive of a computer device, or in other embodiments, it can be an external storage device of the computer device, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card. Furthermore, memory can include both internal storage units and external storage devices of a computer device. Memory is used to store the operating system, applications, bootloader, data, and other programs, such as program code for computer programs. Memory can also be used to temporarily store data that has been output or will be output.

[0142] Those skilled in the art will understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the functions described above can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this invention. The specific working process of the units and modules in the above device can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here. If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention can implement all or part of the processes in the methods of the above embodiments by instructing related hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the above method embodiments. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. A computer-readable medium can include at least: any entity or device capable of carrying computer program code, a recording medium, a computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0143] The present invention can implement all or part of the processes in the methods of the above embodiments, or it can be accomplished by a computer program product. When the computer program product is run on a computer device, the computer device executes the steps in the above method embodiments.

[0144] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0145] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0146] In the embodiments provided by this invention, it should be understood that the disclosed apparatus / computer devices and methods can be implemented in other ways. For example, the apparatus / computer device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0147] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0148] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A method for training a text classification model, characterized in that, The training method of the text classification model comprises: obtaining a text sample with a known actual category, encoding at least two word items in the text sample to obtain word encoding vectors corresponding to the word items; concatenating the word encoding vectors of all the word items to obtain a text encoding vector of the text sample; inputting the text encoding vector into a trained word item category probability prediction model to obtain a word item category probability vector corresponding to each word item in the text encoding vector, and concatenating all the word item category probability vectors to obtain a text smoothing vector; performing a fusion operation on the text encoding vector and the text smoothing vector according to a preset fusion coefficient to obtain a text hybrid vector; inputting the text encoding vector and the text hybrid vector into a text classification model to obtain a predicted category of the text sample, training the text classification model according to the predicted category and the actual category to obtain a trained text classification model; the fusion operation on the text encoding vector and the text smoothing vector according to the preset fusion coefficient to obtain a text hybrid vector comprises: determining the encoding vector values and the smoothing vector values with the same vector coordinates according to the vector coordinates of each encoding vector value in the text encoding vector and the vector coordinates of each smoothing vector value in the text smoothing vector; calculating a fusion vector value according to each encoding vector value, a first preset fusion coefficient of each encoding vector value, and the smoothing vector value with the same vector coordinates of each encoding vector value and a second preset fusion coefficient of the smoothing vector value; obtaining a text hybrid vector according to each fusion vector value and the vector coordinates of each fusion vector value; the vector coordinates of the fusion vector value are the same vector coordinates between the encoding vector value and the smoothing vector value. 2.The method of claim 1, wherein, the encoding of at least two word items in the text sample to obtain word encoding vectors corresponding to the word items comprises: performing word segmentation processing on the text sample to obtain N word items in the text sample, wherein N is an integer greater than 2; encoding each word item to obtain a word encoding vector of each word item. 3.The method of claim 1, wherein, the concatenation of the word encoding vectors of all the word items to obtain a text encoding vector of the text sample comprises: sorting the corresponding N word encoding vectors according to the order of the N word items in the text sample; concatenating the N word encoding vectors into a text encoding vector of a preset dimension according to the sorting result. 4.The method of claim 1, wherein, The word item category probability prediction model comprises a first encoder and a first decoder, and the text encoding vector is taken as a training sample and the text sample is taken as a training label. The training process of the word item category probability prediction model comprises: inputting the text encoding vector into the first encoder for feature extraction to obtain a first text encoding feature; inputting the first text encoding feature into the first decoder to obtain a word item category probability vector of each word item in the text encoding vector. determining a vector composed of all word category probability vectors as the text smoothing vector, processing in the text smoothing vector by using a normalized exponential function, and obtaining a prediction sample; calculating a first loss function according to the prediction sample and the corresponding text sample, and reversely correcting parameters of the first encoder and the first decoder by using a gradient descent method until the first loss function converges, thereby obtaining a trained word category probability prediction model. 5.The method of Claim 1, wherein, The text classification model comprises a second encoder, a third encoder and a full connection layer, the text encoding vector and the text mixing vector are used as training samples, and an actual category of the training sample is used as a training label, the text classification model is trained, and a trained text classification model is obtained. 6.The method of Claim 5, wherein, The training process of the text classification model comprises: inputting the text encoding vector into the second encoder to extract features, thereby obtaining a second text encoding feature; inputting the text mixing vector into the third encoder to extract features, thereby obtaining a text mixing feature; fusing the second text encoding feature and the text mixing feature to obtain a text fusion feature, and inputting the text fusion feature into the full connection layer to obtain a predicted category; calculating a second loss function according to the predicted category and the corresponding actual category, and reversely correcting parameters of the second encoder, the third encoder and the full connection layer by using a gradient descent method until the second loss function converges, thereby obtaining a trained text classification model. 7.A device for training a text classification model, characterized in that, The training device of the text classification model comprises: a word encoding module configured to obtain a text sample with a known actual category, encode at least two words in the text sample to obtain a word encoding vector corresponding to the words, and splice the word encoding vectors of all words to obtain a text encoding vector of the text sample; a vector splicing module configured to splice the word encoding vectors of all words to obtain a text encoding vector of the text sample; a probability prediction module configured to input the text encoding vector into a trained word category probability prediction model to obtain a word category probability vector corresponding to each word in the text encoding vector, splice all word category probability vectors to obtain a text smoothing vector, and perform a fusion operation on the text encoding vector and the text smoothing vector according to a preset fusion coefficient to obtain a text mixing vector; a model training module configured to input the text encoding vector and the text mixing vector into a text classification model to obtain a predicted category of the text sample, train the text classification model according to the predicted category and the actual category, and obtain a trained text classification model; the vector mixing module comprises: a vector value matching sub-module configured to determine the encoding vector value and the smoothing vector value with the same vector coordinates according to vector coordinates of each encoding vector value in the text encoding vector and vector coordinates of each smoothing vector value in the text smoothing vector; ​ a vector value calculation submodule, configured to calculate a fusion vector value according to each of the encoding vector values, a first preset fusion coefficient of each of the encoding vector values, and the smoothing vector value having a same vector coordinate as each of the encoding vector values, and a second preset fusion coefficient of the smoothing vector value; a vector mixing submodule, configured to obtain a text mixing vector according to each of the fusion vector values and a vector coordinate of each of the fusion vector values, the vector coordinate of the fusion vector value being the same vector coordinate between the encoding vector value and the smoothing vector value.

8. A computer device, comprising: The computer device includes a processor, a memory, and a computer program stored in the memory and capable of running on the processor, and the processor implements the training method of the text classification model according to any one of claims 1 to 6 when executing the computer program.

9. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 8. The computer program is executed by the processor to implement the training method of the text classification model according to any one of claims 1 to 6.

Images