Text classification model training method and device, equipment and storage medium

Abstract

The embodiment of the invention discloses a text classification model training method and device, equipment and a storage medium, and belongs to the technical field of artificial intelligence. Comprising the steps of obtaining training texts, and obtaining target text coding features corresponding to each text in the training texts; inputting each target text coding feature into a trained feature extraction sub-model, and outputting an enhanced text feature corresponding to each target text coding feature; inputting the enhanced text features into the clustering sub-model, and outputting a predicted text category corresponding to the enhanced text features; performing category center iteration on the clustering sub-model according to the predicted text category and the tag category to obtain an updated clustering sub-model; and when the number of iterations does not meet the preset number of iterations, returning to execute the acquisition of the training text until the number of iterations meets the preset number of iterations to obtain a trained clustering sub-model, the trained text classification model comprising the trained feature extraction sub-model and the trained clustering sub-model. The text classification efficiency can be improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a text classification model training method, apparatus, device and storage medium. Background Art

[0002] With the development of computer technology, many texts are currently stored in databases in digitized form. For example, some contracts, purchase orders, bills, etc. are stored in databases in digitized form. When there is a large amount of text data in the database, if the text data is not classified and sorted, it will be difficult to find the corresponding text data.

[0003] In the related art, one way is to annotate the text data in advance and then save the annotated text data in a database, but this will consume a lot of manpower and time to annotate the text data, and there will be technical problems such as inaccurate annotation. Another way is to classify different text data by searching for keywords, but this will cause problems such as inaccurate classification of text data.

[0004] Therefore, there is a technical problem in the related art that the efficiency of classifying and organizing text data is low. Summary of the invention

[0005] The embodiments of the present application provide a text classification model training method, apparatus, device and storage medium. The trained text classification model can improve the efficiency of text classification.

[0006] To achieve the above-mentioned purpose, a first aspect of an embodiment of the present application proposes a text classification model training method, wherein the text classification model includes a feature extraction sub-model and a clustering sub-model, and the method includes:

[0007] Obtain training texts, and obtain target text encoding features corresponding to each text in the training texts;

[0008] Input each of the target text encoding features into the trained feature extraction sub-model, and output the enhanced text features corresponding to each of the target text encoding features;

[0009] Inputting the enhanced text features into a clustering sub-model, and outputting a predicted text category corresponding to the enhanced text features;

[0010] Performing category center iteration on the clustering sub-model according to the predicted text category and the label category to obtain an updated clustering sub-model;

[0011] When the number of iterations does not meet the preset number of iterations, return to execute to obtain training text until the number of iterations meets the preset number of iterations, and obtain the trained clustering sub-model. The trained text classification model includes the trained feature extraction sub-model and the trained clustering sub-model.

[0012] To achieve the above-mentioned purpose, a second aspect of an embodiment of the present application proposes a text classification model training device, wherein the text classification model includes a feature extraction sub-model and a clustering sub-model, and the device includes:

[0013] An acquisition module is used to acquire training texts and acquire target text encoding features corresponding to each text in the training texts;

[0014] An extraction module, used for inputting each of the target text encoding features into the trained feature extraction sub-model, and outputting the enhanced text features corresponding to each of the target text encoding features;

[0015] A clustering module, used for inputting the enhanced text features into a clustering sub-model and outputting a predicted text category corresponding to the enhanced text features;

[0016] An updating module, used for performing category center iteration on the clustering sub-model according to the predicted text category and the label category to obtain an updated clustering sub-model;

[0017] The iteration module is used to return to execute the acquisition of training text when the number of iterations does not meet the preset number of iterations, until the number of iterations meets the preset number of iterations, and obtain the trained clustering sub-model. The trained text classification model includes the trained feature extraction sub-model and the trained clustering sub-model.

[0018] In some embodiments, the text classification model training device further includes a pre-training module, which is used to obtain sample texts and obtain text encoding features corresponding to each text in the sample texts before obtaining the training texts;

[0019] Input each of the text encoding features into the feature extraction sub-model, and output the predicted text features corresponding to each of the text encoding features;

[0020] Determine the loss value between each of the predicted text features and the corresponding label text feature. When the loss value is greater than a preset loss value, adjust the model parameters of the feature extraction sub-model to implement iteration of the feature extraction sub-model, and return to execute to obtain sample text until the loss value is no greater than the preset loss value, thereby obtaining a trained feature extraction sub-model.

[0021] In some implementations, the pre-training module is used to perform text segmentation processing on the sample text to obtain a plurality of segmented texts;

[0022] Encode each split text to obtain text features corresponding to each split text;

[0023] Determine a text feature sequence composed of a plurality of text features, and perform position coding on each text feature according to the text feature sequence to obtain a position coding feature;

[0024] A text encoding feature corresponding to each text in the sample text is generated according to each text feature and the position encoding feature corresponding to each text feature.

[0025] In some embodiments, the feature extraction sub-model includes a plurality of attention modules, and the pre-training module is used to input each of the text encoding features into a first attention module to obtain an attention feature output by the first attention module;

[0026] Input the attention feature output by the first attention module into the next attention module to obtain the attention feature output by the next attention module;

[0027] Repeatedly input the attention features output by the previous attention module into the next attention module until the last attention module outputs the predicted text features corresponding to each of the text encoding features.

[0028] In some embodiments, the attention module includes an attention submodule, a feedforward network submodule, a first residual and normalization submodule, and a second residual and normalization submodule, and the pre-training module is used to input each of the text encoding features into the attention submodule of the first attention module, and output the sub-attention feature corresponding to each of the text encoding features;

[0029] Input the text encoding feature and the sub-attention feature into the first residual and normalization sub-module of the first attention module, and output the first extracted feature corresponding to each text encoding feature;

[0030] Inputting the first extracted features into the feedforward network submodule of the first attention module, and outputting the second extracted features corresponding to each text encoding feature;

[0031] The second extracted feature and the sub-attention feature are input into the second residual and normalization sub-module of the first attention module, and the attention feature corresponding to each text encoding feature is output.

[0032] In some embodiments, the pre-training module is used to repeatedly input the attention feature output by the previous attention module into the attention submodule of the next attention module, and output the sub-attention feature corresponding to each text encoding feature;

[0033] Input the sub-attention features into the feedforward network sub-module of the latter attention module, and output the extracted features corresponding to each sub-attention feature;

[0034] The extracted features and the sub-attention features are input into the residual and normalization sub-module of the subsequent attention module to obtain the attention features output by the first attention module, until the last attention module outputs the predicted text features corresponding to each of the text encoding features.

[0035] In some implementations, an updating module is used to obtain a category center corresponding to each tag category;

[0036] determining a distance between the predicted text category and each category center;

[0037] Determine the nearest target category center and the target label category of the target category center;

[0038] The predicted text category is divided into the target label category, and the target category center is updated to obtain an updated category center corresponding to the target label category, thereby obtaining an updated clustering sub-model.

[0039] To achieve the above-mentioned purpose, the third aspect of an embodiment of the present application proposes a computer-readable storage medium, which stores multiple instructions, and the instructions are suitable for a processor to load to execute the text classification model training method provided in the embodiment of the present application.

[0040] To achieve the above-mentioned objectives, the fourth aspect of an embodiment of the present application proposes a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the text classification model training method provided in the embodiment of the present application when executing the computer program.

[0041] In an embodiment of the present application, a text classification model includes a feature extraction submodel and a clustering submodel, which obtains training text and obtains target text encoding features corresponding to each text in the training text; inputs each target text encoding feature into the trained feature extraction submodel, and outputs enhanced text features corresponding to each target text encoding feature; inputs the enhanced text features into the clustering submodel, and outputs predicted text categories corresponding to the enhanced text features; performs category center iteration on the clustering submodel according to the predicted text category and the label category to obtain an updated clustering submodel; when the number of iterations does not meet the preset number of iterations, returns to execute the acquisition of training text until the number of iterations meets the preset number of iterations, and obtains the trained clustering submodel, and the trained text classification model includes the trained feature extraction submodel and the trained clustering submodel. In this way, after obtaining the text features of the text, the feature extraction sub-model can further enhance the representation of the text features to obtain enhanced text features, and the clustering sub-model can accurately classify different texts through the enhanced text features. Therefore, the trained feature extraction sub-model and the trained clustering sub-model cooperate with each other to achieve the classification of different types of texts. Compared with the method of labeling or keyword classification in the related art, the trained text classification model in the present application is more efficient in text classification.

[0042] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or understood by practicing the present application. The purpose and other advantages of the present application can be realized and obtained by the structures specifically pointed out in the description, claims and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0044] Figure 1 is a schematic diagram of a text classification scenario provided by an embodiment of the present application;

[0045] Figure 2 It is a flowchart of a text classification model training method provided in an embodiment of the present application;

[0046] Figure 3 It is a schematic diagram of the process of training the feature extraction sub-model provided in the embodiment of the present application;

[0047] Figure 4 It is a schematic diagram of the structure of the feature extraction sub-model provided in the embodiment of the present application;

[0048] Figure 5 is another structural schematic diagram of the feature extraction sub-model provided in an embodiment of the present application;

[0049] Figure 6 is another flowchart of the text classification model training method provided in an embodiment of the present application;

[0050] Figure 7 It is a structural schematic diagram of a text classification model training device provided in an embodiment of the present application;

[0051] Figure 8 It is a schematic diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0052] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0053] It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0054] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0055] First, some nouns involved in this application are analyzed:

[0056] Artificial intelligence (AI) is a new technical science that studies and develops theories, methods, technologies and application systems for simulating, extending and expanding human intelligence. AI is a branch of computer science that attempts to understand the essence of intelligence and produce a new type of intelligent machine that can respond in a similar way to human intelligence. Research in this field includes robotics, speech recognition, image recognition, natural language processing and expert systems. AI can simulate the information process of human consciousness and thinking. AI is also a theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0057] Self-Attention Mechanism: The self-attention mechanism mainly allows the model to learn to pay attention to the information at different locations within itself, thereby automatically assigning different importance weights to different parts. Unlike the traditional attention mechanism, the self-attention mechanism does not rely on external information sources, but calculates the attention weights entirely based on the input data itself. In tasks such as image classification, object detection, and image segmentation, the self-attention mechanism allows the model to focus on different areas in the image, capture global and local features, and improve the accuracy of the model.

[0058] Secondly, describe the technical problems existing in related technologies:

[0059] With the development of computer technology, many texts are currently stored in databases in digitized form. For example, some contracts, purchase orders, bills, etc. are stored in databases in digitized form. When there is a large amount of text data in the database, if the text data is not classified and sorted, it will be difficult to find the corresponding text data.

[0060] In the related art, one way is to annotate the text data in advance and then save the annotated text data in a database, but this will consume a lot of manpower and time to annotate the text data, and there will be technical problems such as inaccurate annotation. Another way is to classify different text data by searching for keywords, but this will cause problems such as inaccurate classification of text data.

[0061] Therefore, there is a technical problem of low efficiency in classifying and organizing text data in the related technology.

[0062] In order to solve the above technical problems, in an embodiment of the present application, after obtaining the text features of the text, the text features can be further enhanced and represented through the feature extraction sub-model, thereby obtaining enhanced text features, and the clustering sub-model can accurately classify different texts through the enhanced text features. Therefore, the trained feature extraction sub-model and the trained clustering sub-model cooperate with each other to realize the classification of different types of texts. Compared with the method of labeling or keyword classification in the related art, the trained text classification model in the present application is more efficient in text classification.

[0063] See also Figure 1 , Figure 1 It is a schematic diagram of a scenario of text classification provided in an embodiment of the present application.

[0064] The trained text classification model includes a trained feature extraction sub-model and a trained clustering sub-model. Compared with directly encoding each text in the form to obtain text features, the trained feature extraction sub-model can perform attention calculation on the input text features, thereby obtaining the dependency relationship between different texts, thereby obtaining target enhanced text features related to the entire form.

[0065] Among them, the trained feature extraction sub-model includes a self-attention mechanism. The trained feature extraction sub-model can capture the long-distance dependency of the text in the form through the self-attention mechanism. Specifically, the information in the text often does not exist in isolation, and the relationship between words may span a long distance. The self-attention mechanism can effectively capture this long-distance dependency.

[0066] For example, in a long sentence, a keyword is crucial to understanding the meaning of the entire sentence, but its position may be far away from other related words. The self-attention mechanism can highlight the relationship between important words by calculating the degree of association between each word and other words, no matter how far apart they are in the text. This enables the clustering sub-model after subsequent training to better understand the overall semantics of the text, thereby improving the accuracy of text classification.

[0067] The trained feature extraction sub-model can highlight the different importance of different words in text classification through the self-attention mechanism. The self-attention mechanism can automatically learn and highlight key information, and provide more valuable features for the classification task of the subsequent trained clustering sub-model. For example, for text classification tasks, certain specific words or phrases may play a decisive role in determining the category of the text. The self-attention mechanism can assign different weights to each word according to the context and global information of the word, thereby emphasizing those parts that have a greater impact on the classification results. This allows the model to pay more attention to important features, reduce the interference of noise and irrelevant information, and improve the efficiency and accuracy of classification.

[0068] In addition, the trained feature extraction sub-model can obtain different structural and semantic characteristics of different texts through the self-attention mechanism, and the self-attention mechanism can dynamically adapt to various text situations. When processing different types of text, the self-attention mechanism can automatically adjust the way attention is allocated according to the specific content of the text. For example, for scientific and technological texts, more attention may be paid to professional terms and technical concepts; while for literary texts, more attention may be paid to emotional expression and rhetoric. This dynamic adaptability makes the self-attention mechanism more versatile and flexible when dealing with diverse text classification tasks.

[0069] From the above, we can see that the trained feature extraction sub-model can enhance the various text features in the form, and provide more clear target enhanced text features for the classification task of the subsequent trained clustering sub-model, so that the trained clustering sub-model can achieve efficient and accurate classification of the form through the target enhanced text features. For example, for different forms such as financial forms, purchase orders, equipment lists, contract forms, etc., the trained clustering sub-model can achieve efficient and accurate classification of the form through the target enhanced text features corresponding to each form.

[0070] The above is the text classification scenario corresponding to the trained text classification model described in this application.

[0071] It can be understood that the text classification model includes a feature extraction sub-model and a clustering sub-model. The training process of the feature extraction sub-model and the clustering sub-model will be described in detail below.

[0072] The embodiments of the present application provide a text classification model training method, apparatus, device and storage medium. The trained text classification model can improve the efficiency of text classification.

[0073] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0074] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0075] The text classification model training method provided in the embodiment of the present application relates to the field of artificial intelligence technology. The text classification model training method provided in the embodiment of the present application can be applied to a terminal, can also be applied to a server side, and can also be software running in a terminal or a server side. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc.; the server side can be configured as an independent physical server, or can be configured as a server cluster or a distributed system composed of multiple physical servers, and can also be configured as a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements the text classification model training method, etc., but is not limited to the above forms.

[0076] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer computer devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0077] It should be noted that in each specific implementation of the present application, when it comes to relevant processing of text data, the user's permission or consent will be obtained first, and the collection, use and processing of these data will comply with relevant laws, regulations and standards. In addition, when the embodiment of the present application needs to obtain the user's sensitive personal information, the user's separate permission or consent will be obtained through a pop-up window or jump to a confirmation page. After the user's separate permission or consent is clearly obtained, the necessary user-related data for the normal operation of the embodiment of the present application is obtained.

[0078] See also Figure 2 , Figure 2 : is a flowchart of a text classification model training method provided in an embodiment of the present application. The text classification model training method provided in an embodiment of the present application may include the following steps:

[0079] Step 110: obtaining training texts, and obtaining target text encoding features corresponding to each text in the training texts;

[0080] Step 120: input each target text encoding feature into the trained feature extraction sub-model, and output the enhanced text feature corresponding to each target text encoding feature;

[0081] Step 130: input the enhanced text features into the clustering sub-model, and output the predicted text category corresponding to the enhanced text features;

[0082] Step 140: performing category center iteration on the clustering sub-model according to the predicted text category and the label category to obtain an updated clustering sub-model;

[0083] Step 150, when the number of iterations does not meet the preset number of iterations, return to execute to obtain the training text until the number of iterations meets the preset number of iterations, and obtain the trained clustering sub-model, the trained text classification model includes the trained feature extraction sub-model and the trained clustering sub-model.

[0084] Before describing step 110 to step 150 in detail, the overall training idea of ​​the text classification model is first explained, wherein the text classification model includes a feature extraction sub-model and a clustering sub-model. In the present application, the feature extraction sub-model is first trained to obtain a trained feature extraction sub-model, and then the clustering sub-model is trained by the trained feature extraction sub-model to obtain a trained clustering sub-model. Finally, the trained text classification model includes a trained feature extraction sub-model and a trained clustering sub-model.

[0085] See also Figure 3 , Figure 3 It is a flow chart of training the feature extraction sub-model provided in the embodiment of the present application. Before obtaining the training text, it also includes:

[0086] Step 201: Obtain sample text, and obtain text encoding features corresponding to each text in the sample text;

[0087] Step 202: input each text encoding feature into the feature extraction sub-model, and output the predicted text feature corresponding to each text encoding feature;

[0088] Step 203, determine the loss value between each predicted text feature and the corresponding label text feature. When the loss value is greater than the preset loss value, adjust the model parameters of the feature extraction sub-model to implement iteration of the feature extraction sub-model, and return to execute to obtain sample text until the loss value is no greater than the preset loss value, thereby obtaining the trained feature extraction sub-model.

[0089] The following will describe steps 201 to 203 in detail.

[0090] In step 201, a sample text is obtained, and a text encoding feature corresponding to each text in the sample text is obtained.

[0091] Among them, the sample text is the text for training the feature extraction sub-model, and the sample text contains multiple texts. The text can include words, sentences, etc.

[0092] The sample text can be obtained based on an existing form image. For example, the form image can be first input into an optical character recognition model, and the optical character recognition model can recognize the text therein to obtain the extracted text, which contains words, sentences, characters, etc.

[0093] Then, punctuation marks are removed from the extracted text and converted to lowercase letters to unify the text format. Optionally, operations such as stemming or lemmatization can be performed to restore the words to their basic forms, and finally the sample text is obtained, which contains multiple texts. After processing the form image and processing the extracted text therein and filtering out useless text, such as removing punctuation marks, special characters, and stop words (such as common words with little semantic contribution like "of", "is", "in", etc.). The obtained sample text is more suitable for model training, which can improve the model training efficiency of the feature extraction sub-model.

[0094] In some embodiments, obtaining the text encoding feature corresponding to each text in the sample text includes:

[0095] (1.1) Perform text splitting processing on the sample text to obtain multiple split texts;

[0096] (1.2) Encode each split text to obtain the text feature corresponding to each split text;

[0097] (1.3) Determine the text feature sequence composed of multiple text features, and perform positional encoding on each text feature according to the text feature sequence to obtain the positional encoding feature;

[0098] (1.4) Generate the text encoding feature corresponding to each text in the sample text according to each text feature and the positional encoding feature corresponding to each text feature.

[0099] It can be understood that the sample text is all the text in the entire form image, and there will be some long texts among them, which are not conducive to subsequent data processing. Therefore, text splitting processing is required to obtain multiple split texts.

[0100] Each split text can be encoded to obtain text features. For example, different words in all texts are collected to build a vocabulary, and each word in the vocabulary has a unique index. For each split text, the number of times each word in the vocabulary appears in the text is counted to form a vector. The length of the vector is equal to the size of the vocabulary, and each element in the vector corresponds to a word in the vocabulary, indicating the number of times the word appears in the split text.

[0101] For another example, we build a vocabulary and index all the words or characters that appear in the training data. Then, we initialize an embedding matrix whose dimension is the vocabulary size multiplied by the embedding dimension. The embedding dimension refers to the dimension of the continuous vector space into which the words or characters are embedded; it should be noted that for each word or character in the input sequence, the corresponding embedding vector is found by querying the index in the vocabulary. The row vector of the corresponding index is obtained from the embedding matrix as the embedding representation, so as to obtain the text features corresponding to each split text, which can be understood as a vector.

[0102] Then, a text feature sequence composed of multiple text features is determined, for example, multiple text features are sorted to obtain a text feature sequence, and each text feature is position-encoded according to the text feature sequence to obtain a position-encoded feature.

[0103] It is understandable that after obtaining each text feature, these text features do not have the position information required by the feature extraction sub-model for data processing, and the feature extraction sub-model is also different from the traditional recurrent neural network model and convolutional neural network model, and does not have its own position information. Therefore, it is necessary to set positional encoding (Positional Encoding) for each text feature, so that the feature extraction sub-model can determine the positional relationship between different text features, thereby distinguishing different texts.

[0104] Finally, the text encoding feature corresponding to each text in the sample text can be generated according to each text feature and the position encoding feature corresponding to each text feature. For example, each text feature and the position encoding feature corresponding to each text feature are concatenated to obtain the text encoding feature.

[0105] In step 202, each text encoding feature is input into the feature extraction sub-model, and the predicted text feature corresponding to each text encoding feature is output.

[0106] Among them, the feature extraction sub-model is a model with a self-attention mechanism, which can capture the long-distance dependencies of the text in the form. Specifically, the information in the text is often not isolated, and the relationship between words may span a long distance. The self-attention mechanism can effectively capture this long-distance dependency. Therefore, each text encoding feature is input into the feature extraction sub-model, and the predicted text feature corresponding to each text encoding feature is output. The predicted text feature can better represent the corresponding text.

[0107] Please also read Figure 4 , Figure 4 It is a structural diagram of the feature extraction sub-model provided in an embodiment of the present application.

[0108] Among them, the feature extraction sub-model includes multiple attention modules, and the multiple attention modules are connected in sequence, and the output of the previous attention module is connected to the input of the next attention module.

[0109] In some implementations, each text encoding feature is input into a feature extraction sub-model, and a predicted text feature corresponding to each text encoding feature is output, including:

[0110] (1.1) Input each text encoding feature into the first attention module to obtain the attention feature output by the first attention module;

[0111] (1.2) Input the attention feature output by the first attention module into the next attention module to obtain the attention feature output by the next attention module;

[0112] (1.3) Repeatedly input the attention features output by the previous attention module into the next attention module until the last attention module outputs the predicted text features corresponding to each text encoding feature.

[0113] Among them, firstly, each text encoding feature is input into the first attention module to obtain the attention feature output by the first attention module, and then the attention feature output by the first attention module is input into the next attention module, and the next attention module continues to process the attention feature output by the previous attention module, and the next attention module outputs the corresponding attention feature. By repeatedly inputting the attention feature output by the previous attention module into the next attention module, until the last attention module outputs the predicted text feature corresponding to each text encoding feature. Among them, the number of attention modules corresponding to the feature extraction sub-model can be 6, then the 6th attention module outputs the predicted text feature corresponding to each text encoding feature.

[0114] In some embodiments, the attention module includes an attention submodule, a feedforward network submodule, a first residual and normalization submodule, and a second residual and normalization submodule, and each text encoding feature is input into the first attention module to obtain an attention feature output by the first attention module, including:

[0115] (1.1.1) Input each text encoding feature into the attention submodule of the first attention module, and output the sub-attention feature corresponding to each text encoding feature;

[0116] (1.1.2) Input the text encoding features and sub-attention features into the first residual and normalization sub-module of the first attention module, and output the first extracted features corresponding to each text encoding feature;

[0117] (1.1.3) Inputting the first extracted features into the feedforward network submodule of the first attention module, and outputting the second extracted features corresponding to each text encoding feature;

[0118] (1.1.4) The second extracted features and sub-attention features are input into the second residual and normalization sub-module of the first attention module, and the attention features corresponding to each text encoding feature are output.

[0119] Please also read Figure 5 , Figure 5 It is another structural schematic diagram of the feature extraction sub-model provided in an embodiment of the present application.

[0120] The structure of each attention module in the feature extraction sub-model is the same. The number of attention modules can be N. Taking one of the attention modules as an example, it includes an attention sub-module, a first residual and normalization sub-module, a feedforward network sub-module, and a second residual and normalization sub-module. The input of the attention sub-module of the first attention module is the text encoding feature, and the input of the attention sub-module of the attention module after the first attention module is the output of the previous attention module.

[0121] Taking the first attention module as an example, each text encoding feature is input into the attention submodule of the first attention module, and the sub-attention feature corresponding to each text encoding feature is output. Among them, the attention submodule contains three channels, the Q channel outputs the query vector (Query), the K channel outputs the key vector (Key), and the V channel outputs the value vector (Value).

[0122] The query vector and key vector are used to calculate the similarity score between a certain text encoding feature and each text encoding feature among all text encoding features, and the value vector is the feature representation corresponding to each text encoding feature. The value vector and the above similarity score are used to calculate the weighted sum corresponding to a certain text encoding feature. The weighted sum can be understood as the sub-attention feature corresponding to the text encoding feature. The text encoding feature and its related query vector, key vector and value vector are obtained by multiplying the text encoding feature by different weights.

[0123] For example, based on the query vector of a text encoding feature, the similarity between the query vector and the key vector of each encoding feature is calculated to obtain the similarity score between the text encoding feature and each text encoding feature, and then the similarity score is multiplied by the value vector corresponding to each text encoding feature to obtain the multiplication result, and all the multiplication results are added together to obtain the sub-attention feature corresponding to the text encoding feature.

[0124] The self-attention submodule can be a multi-head attention module, which applies the self-attention mechanism multiple times and generates multiple attention heads. In the multi-head attention mechanism, multiple parallel attention heads (usually 8 or more) are used, and each head performs independent attention calculations. Each head generates an attention weight matrix, which is used to weight the value of the corresponding head. The value matrices of all attention heads are concatenated together and projected through a linear transformation. This linear transformation maps the results of multiple heads back to the dimensions of the original input. Finally, the result after the linear transformation is used as the output of the multi-head attention for use by the next layer or the next module.

[0125] Then the text encoding features and sub-attention features are input into the first residual and normalization submodule of the first attention module, and the first extracted features corresponding to each text encoding feature are output. The first residual and normalization submodule has two functions: one is to play the role of residual network, and the other is to normalize.

[0126] The purpose of residual connection is to solve the problem of gradient disappearance and gradient explosion, so that gradient can flow more easily in the network. Residual connection also helps to retain the details of low-level features and avoid information loss, thereby improving the representation and learning ability of the model. Layer normalization is used to normalize the representation of each position. Layer normalization calculates the mean and variance of the representation of each position and normalizes the representation so that the representation of each position has a similar distribution and scale.

[0127] The first extracted feature is then input into the feedforward network submodule of the first attention module, and the second extracted feature corresponding to each text encoding feature is output. Among them, the feedforward network submodule can pass the input through a linear transformation layer to map it to a higher dimensional feature space. After the linear transformation, a nonlinear activation function, such as ReLU (Rectified Linear Unit), is applied to introduce nonlinear properties. The output of the activation function is passed through another linear transformation layer again to map it back to the dimension of the original input. The output of the feedforward network submodule is the representation after linear transformation and activation function processing. Through multiple layers of nonlinear transformation, the feedforward network submodule can perform a more complex mapping of the representation of each first extracted feature, thereby enhancing the model's ability to model local features. It can learn the complex nonlinear relationship between different text encoding features, extract higher-level feature representations, and enhance the expression ability of the attention submodule.

[0128] Finally, the second extracted features and sub-attention features are input into the second residual and normalization submodule of the first attention module, and the attention features corresponding to each text encoding feature are output. The functions of the second residual and normalization submodule and the first residual and normalization submodule are basically the same, and will not be repeated here. The second residual and normalization submodule outputs the attention features corresponding to each text encoding feature. That is, the attention features corresponding to each text encoding feature output by the first attention module.

[0129] In some implementations, the attention feature output by the previous attention module is repeatedly input into the next attention module until the last attention module outputs the predicted text feature corresponding to each text encoding feature, including:

[0130] (1.3.1) Repeatedly input the attention features output by the previous attention module into the attention submodule of the next attention module, and output the sub-attention features corresponding to each text encoding feature;

[0131] (1.3.2) Input the sub-attention features into the feedforward network sub-module of the next attention module, and output the extracted features corresponding to each sub-attention feature;

[0132] (1.3.3) The extracted features and sub-attention features are input into the residual and normalization sub-module of the next attention module to obtain the attention features output by the first attention module, until the last attention module outputs the predicted text features corresponding to each text encoding feature.

[0133] Among them, for the attention module after the first attention module, the attention features output by the previous attention module can be repeatedly input into the attention sub-module of the next attention module to output the sub-attention features corresponding to each text encoding feature, and then the sub-attention features are input into the feedforward network sub-module of the next attention module to output the extracted features corresponding to each sub-attention feature; finally, the extracted features and the sub-attention features are input into the residual and normalization sub-module of the next attention module to obtain the attention features output by the first attention module, until the last attention module outputs the predicted text features corresponding to each text encoding feature.

[0134] Its specific processing process is similar to the data processing process of the first attention module and will not be repeated here.

[0135] In step 203, the loss value between each predicted text feature and the corresponding label text feature is determined. When the loss value is greater than the preset loss value, the model parameters of the feature extraction sub-model are adjusted to implement iteration of the feature extraction sub-model, and the execution is returned to obtain the sample text until the loss value is no greater than the preset loss value, thereby obtaining the trained feature extraction sub-model.

[0136] It can be understood that the text encoding features correspond to the label text features. Each predicted text feature and the corresponding label text feature can be input into the loss function, and the loss value between each predicted text feature and the corresponding label text feature can be calculated to determine the difference between the two.

[0137] If the loss value is greater than the preset loss value, it means that the prediction result of the extraction sub-model deviates too far from the true result. At this time, the model parameters of the feature extraction sub-model need to be adjusted to change the prediction performance of the feature extraction sub-model to enhance the accuracy of its prediction results, implement iteration of the feature extraction sub-model, and then return to execute to obtain the sample text, that is, return to step 201, and continue to iterate the feature extraction sub-model until the loss value is no greater than the preset loss value, and obtain the trained feature extraction sub-model.

[0138] From step 201 to step 203, it can be seen that in the present application, the trained feature extraction sub-model has a self-attention mechanism, which can capture the attention relationship between different texts, and can accurately extract features from the text in the form, thereby highlighting the features related to the text in the form. In other words, the trained feature extraction sub-model can accurately obtain the text features in the form, and the text features can be used for subsequent classification of the form text, which can improve the accuracy of text classification.

[0139] In step 110, training text is obtained, and target text encoding features corresponding to each text in the training text are obtained.

[0140] Among them, the training text can be split into multiple sub-texts; each sub-text is encoded to obtain the text features corresponding to each sub-text; a text feature sequence composed of multiple text features is determined, and each text feature is position-encoded according to the text feature sequence to obtain a position coding feature; based on each text feature and the position coding feature corresponding to each text feature, a target text coding feature corresponding to each sub-text is generated.

[0141] In step 120, each target text encoding feature is input into the trained feature extraction sub-model, and the enhanced text feature corresponding to each target text encoding feature is output.

[0142] The trained feature extraction sub-model can highlight the different importance of different words in text classification through the self-attention mechanism. The self-attention mechanism can automatically learn and highlight key information, and provide more valuable features for the classification task of the subsequent trained clustering sub-model. For example, for text classification tasks, certain specific words or phrases may play a decisive role in determining the category of the text. The self-attention mechanism can assign different weights to each word according to the context and global information of the word, thereby emphasizing those parts that have a greater impact on the classification results. This allows the model to pay more attention to important features, reduce the interference of noise and irrelevant information, and improve the efficiency and accuracy of classification.

[0143] In this way, outputting enhanced text features can achieve enhanced representation of the corresponding text in the training text, which is conducive to the subsequent accurate classification of the entire training text.

[0144] In step 130, the enhanced text features are input into the clustering sub-model, and the predicted text category corresponding to the enhanced text features is output.

[0145] The enhanced text features of each text in the training text can be input into the clustering sub-model, and the predicted text category corresponding to the enhanced text features can be output.

[0146] The specific algorithm used by the clustering sub-model may be the K-mean clustering algorithm. In the clustering sub-model, multiple label categories may be set first. For example, different texts may be divided into accounting, reimbursement, contract, etc. Among the multiple label categories, each label category corresponds to a category center. These category centers can be understood as specific points in a category space of the same dimension.

[0147] The enhanced text features corresponding to each target text encoding feature can be used to generate a target feature sequence. The target feature sequence can be understood as a vector composed of the enhanced text features of each text in the training text and can represent the training text. The enhanced text features are then input into the clustering sub-model to output the predicted text category corresponding to the enhanced text features.

[0148] In step 140, the clustering sub-model is iterated based on the predicted text category and the label category to obtain an updated clustering sub-model.

[0149] After obtaining the predicted text category, it is equivalent to adding a new feature in the category space. Therefore, it is necessary to perform category center iteration on the clustering sub-model according to the predicted text category and the label category to obtain an updated clustering sub-model.

[0150] In some implementations, performing category center iteration on the clustering sub-model according to the predicted text category and the label category to obtain an updated clustering sub-model includes:

[0151] (1.1) Obtain the category center corresponding to each label category;

[0152] (1.2) Determine the distance between the predicted text category and the center of each category;

[0153] (1.3) Determine the target category center that is closest to the target category center and the target label category of the target category center;

[0154] (1.4) The predicted text category is divided into the target label category, and the target category center is updated to obtain the updated category center corresponding to the target label category, and the updated clustering sub-model is obtained.

[0155] For example, the category center corresponding to each label category and the specific position information corresponding to each category center can be determined in the category space, and then the position information of the predicted text category in the category space can be determined. According to the specific position information corresponding to each category center and the position information of the predicted text category, the distance between the predicted text category and each category center can be determined.

[0156] In the category center, determine the target category center closest to the target category center and the target label category of the target category center, divide the predicted text category into the target label category, and update the target category center. For example, the category center of the target label category can be recalculated by combining the corresponding position information of the training text in the category space and the corresponding position information of other texts in the target label category in the category space, and the updated category center corresponding to the target label category can be obtained to obtain the updated clustering sub-model. The updated clustering sub-model contains the data corresponding to the above category space.

[0157] The advantage of doing this is that by continuously updating the category center of each label category, the category center will become more and more accurate, so that the updated clustering sub-model can more accurately predict the category of the text.

[0158] In step 150, when the number of iterations does not meet the preset number of iterations, return to execute to obtain training text until the number of iterations meets the preset number of iterations, and obtain the trained clustering sub-model. The trained text classification model includes the trained feature extraction sub-model and the trained clustering sub-model.

[0159] It is understandable that when the number of iterations does not meet the preset number of iterations, it means that the category center in the clustering sub-model is not very accurate and needs to be continued. At this time, the execution returns to obtain the training text until the number of iterations meets the preset number of iterations, and the trained clustering sub-model is obtained. The trained text classification model includes the trained feature extraction sub-model and the trained clustering sub-model. The trained clustering sub-model determines an accurate category center for each category for all known categories. Therefore, the trained clustering sub-model can accurately classify the features output by the trained feature extraction sub-model, thereby achieving accurate classification of the text.

[0160] As can be seen from the above, in an embodiment of the present application, the text classification model includes a feature extraction submodel and a clustering submodel, which obtains training text and obtains the target text encoding features corresponding to each text in the training text; inputs each target text encoding feature into the trained feature extraction submodel, and outputs the enhanced text features corresponding to each target text encoding feature; inputs the enhanced text features into the clustering submodel, and outputs the predicted text category corresponding to the enhanced text features; performs category center iteration on the clustering submodel according to the predicted text category and the label category to obtain an updated clustering submodel; when the number of iterations does not meet the preset number of iterations, returns to execute the acquisition of training text until the number of iterations meets the preset number of iterations, and obtains the trained clustering submodel, and the trained text classification model includes the trained feature extraction submodel and the trained clustering submodel. In this way, after obtaining the text features of the text, the feature extraction sub-model can further enhance the representation of the text features to obtain enhanced text features, and the clustering sub-model can accurately classify different texts through the enhanced text features. Therefore, the trained feature extraction sub-model and the trained clustering sub-model cooperate with each other to achieve the classification of different types of texts. Compared with the method of labeling or keyword classification in the related art, the trained text classification model in the present application is more efficient in text classification.

[0161] See also Figure 6 , Figure 61 is another flow chart of the text classification model training method provided in the embodiment of the present application. The text classification model training method may include the following steps:

[0162] Step 301: Obtain sample text, and obtain text encoding features corresponding to each text in the sample text;

[0163] Step 302: input each text encoding feature into the feature extraction sub-model, and output the predicted text feature corresponding to each text encoding feature;

[0164] Step 303: determine the loss value between each predicted text feature and the corresponding label text feature. When the loss value is greater than a preset loss value, adjust the model parameters of the feature extraction sub-model to implement iteration of the feature extraction sub-model, and return to execute to obtain sample text until the loss value is no greater than the preset loss value, thereby obtaining a trained feature extraction sub-model.

[0165] Step 304: obtaining training texts, and obtaining target text encoding features corresponding to each text in the training texts;

[0166] Step 305: input each target text encoding feature into the trained feature extraction sub-model, and output the enhanced text feature corresponding to each target text encoding feature;

[0167] Step 306: input the enhanced text features into the clustering sub-model, and output the predicted text category corresponding to the enhanced text features;

[0168] Step 307: Obtain the category center corresponding to each tag category;

[0169] Step 308, determining the distance between the predicted text category and each category center;

[0170] Step 309: determine the nearest target category center and the target label category of the target category center;

[0171] Step 310: classify the predicted text category into the target label category, and update the target category center to obtain the updated category center corresponding to the target label category, and obtain the updated clustering sub-model;

[0172] Step 311, when the number of iterations does not meet the preset number of iterations, return to execute to obtain the training text until the number of iterations meets the preset number of iterations, and obtain the trained clustering sub-model. The trained text classification model includes the trained feature extraction sub-model and the trained clustering sub-model.

[0173] In the above embodiments, the description of each embodiment has its own focus. For the part that is not described in detail in a certain embodiment, please refer to the detailed description of the above text classification model training method, which will not be repeated here.

[0174] See also Figure 7 , Figure 7 1 is a structural diagram of a text classification model training device provided in an embodiment of the present application. The text classification model training device can execute the above-mentioned text classification model training method, and the text classification model training device is arranged in a computer device.

[0175] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program with a predetermined function, and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories) or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0176] like Figure 7 As shown, the text classification model includes a feature extraction sub-model and a clustering sub-model, and the text classification model training device 400 includes:

[0177] The acquisition module 410 is used to acquire training texts and acquire target text encoding features corresponding to each text in the training texts;

[0178] An extraction module 420, used for inputting each target text encoding feature into the trained feature extraction sub-model, and outputting the enhanced text feature corresponding to each target text encoding feature;

[0179] A clustering module 430, used to input the enhanced text features into the clustering sub-model and output the predicted text category corresponding to the enhanced text features;

[0180] An updating module 440 is used to perform category center iteration on the clustering sub-model according to the predicted text category and the label category to obtain an updated clustering sub-model;

[0181] Iteration module 450 is used to return to execute and obtain training text when the number of iterations does not meet the preset number of iterations, until the number of iterations meets the preset number of iterations, and obtain the trained clustering sub-model. The trained text classification model includes the trained feature extraction sub-model and the trained clustering sub-model.

[0182] In some embodiments, the text classification model training device 400 further includes a pre-training module 460, which is used to obtain sample texts and obtain text encoding features corresponding to each text in the sample texts before obtaining the training texts;

[0183] Input each text encoding feature into the feature extraction sub-model, and output the predicted text feature corresponding to each text encoding feature;

[0184] Determine the loss value between each predicted text feature and the corresponding label text feature. When the loss value is greater than the preset loss value, adjust the model parameters of the feature extraction sub-model to implement the iteration of the feature extraction sub-model, and return to execute to obtain the sample text until the loss value is no greater than the preset loss value, thereby obtaining the trained feature extraction sub-model.

[0185] In some implementations, the pre-training module 460 is used to perform text segmentation processing on the sample text to obtain a plurality of segmented texts;

[0186] Encode each split text to obtain the text features corresponding to each split text;

[0187] Determine a text feature sequence composed of multiple text features, and perform position encoding on each text feature according to the text feature sequence to obtain a position encoding feature;

[0188] According to each text feature and the position encoding feature corresponding to each text feature, a text encoding feature corresponding to each text in the sample text is generated.

[0189] In some embodiments, the feature extraction sub-model includes a plurality of attention modules, and the pre-training module 460 is used to input each text encoding feature into a first attention module to obtain an attention feature output by the first attention module;

[0190] Input the attention feature output by the first attention module into the next attention module to obtain the attention feature output by the next attention module;

[0191] Repeatedly input the attention features output by the previous attention module into the next attention module until the last attention module outputs the predicted text features corresponding to each text encoding feature.

[0192] In some embodiments, the attention module includes an attention submodule, a feedforward network submodule, a first residual and normalization submodule, and a second residual and normalization submodule, and a pre-training module 460 is used to input each text encoding feature into the attention submodule of the first attention module, and output a sub-attention feature corresponding to each text encoding feature;

[0193] Input the text encoding features and the sub-attention features into the first residual and normalization sub-module of the first attention module, and output the first extracted features corresponding to each text encoding feature;

[0194] Input the first extracted features into the feedforward network submodule of the first attention module, and output the second extracted features corresponding to each text encoding feature;

[0195] The second extracted features and sub-attention features are input into the second residual and normalization sub-module of the first attention module, and the attention features corresponding to each text encoding feature are output.

[0196] In some embodiments, the pre-training module 460 is used to repeatedly input the attention feature output by the previous attention module into the attention sub-module of the next attention module, and output the sub-attention feature corresponding to each text encoding feature;

[0197] Input the sub-attention features into the feedforward network sub-module of the next attention module, and output the extracted features corresponding to each sub-attention feature;

[0198] The extracted features and sub-attention features are input into the residual and normalization sub-module of the next attention module to obtain the attention features output by the first attention module, until the last attention module outputs the predicted text features corresponding to each text encoding feature.

[0199] In some implementations, the updating module 440 is used to obtain the category center corresponding to each tag category;

[0200] Determine the distance between the predicted text category and the center of each category;

[0201] Determine the target category center that is closest to the target category center and the target label category of the target category center;

[0202] The predicted text category is divided into the target label category, and the target category center is updated to obtain the updated category center corresponding to the target label category, and the updated clustering sub-model is obtained.

[0203] In the above embodiments, the description of each embodiment has its own focus. For the part that is not described in detail in a certain embodiment, please refer to the detailed description of the above text classification model training method, which will not be repeated here.

[0204] As can be seen from the above, in the embodiment of the present application, the acquisition module 410 is used to obtain the training text and obtain the target text encoding features corresponding to each text in the training text; the extraction module 420 is used to input each target text encoding feature into the trained feature extraction sub-model, and output the enhanced text features corresponding to each target text encoding feature; the clustering module 430 is used to input the enhanced text features into the clustering sub-model, and output the predicted text category corresponding to the enhanced text features; the update module 440 is used to perform category center iteration on the clustering sub-model according to the predicted text category and the label category to obtain the updated clustering sub-model; the iteration module 450 is used to return to execute the acquisition of the training text when the number of iterations does not meet the preset number of iterations, until the number of iterations meets the preset number of iterations, and obtain the trained clustering sub-model, and the trained text classification model includes the trained feature extraction sub-model and the trained clustering sub-model. In this way, after obtaining the text features of the text, the feature extraction sub-model can further enhance the representation of the text features to obtain enhanced text features, and the clustering sub-model can accurately classify different texts through the enhanced text features. Therefore, the trained feature extraction sub-model and the trained clustering sub-model cooperate with each other to achieve the classification of different types of texts. Compared with the method of labeling or keyword classification in the related art, the trained text classification model in the present application is more efficient in text classification.

[0205] See also Figure 8 , Figure 8 It is a schematic diagram of the structure of a computer device provided in an embodiment of the present application.

[0206] The embodiment of the present application also provides a computer device, the computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the above-mentioned text classification model training method when executing the computer program. The computer device can be any intelligent terminal including a tablet computer, a car computer, etc.

[0207] See also Figure 8 , Figure 8 The hardware structure of a computer device according to another embodiment is shown, and the computer device includes:

[0208] The processor 501 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0209] The memory 502 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 502 can store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 502, and the processor 501 calls and executes the text classification model training method of the embodiment of this application;

[0210] Input / output interface 503, used to implement information input and output;

[0211] Communication interface 504, used to realize communication interaction between the device and other devices, which can be realized by wired mode (such as USB, network cable, etc.) or wireless mode (such as mobile network, WIFI, Bluetooth, etc.);

[0212] A bus 505 that transmits information between the various components of the device (e.g., the processor 501, the memory 502, the input / output interface 503, and the communication interface 504);

[0213] The processor 501 , the memory 502 , the input / output interface 503 and the communication interface 504 are connected to each other in communication within the device via the bus 505 .

[0214] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, the above-mentioned text classification model training method is implemented.

[0215] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0216] The text classification model training method, text classification model training device, computer equipment and storage medium provided in the embodiments of the present application obtain the target text encoding features corresponding to each text in the training text; input each target text encoding feature into the trained feature extraction sub-model, and output the enhanced text features corresponding to each target text encoding feature; input the enhanced text features into the clustering sub-model, and output the predicted text category corresponding to the enhanced text features; perform category center iteration on the clustering sub-model according to the predicted text category and the label category to obtain an updated clustering sub-model; when the number of iterations does not meet the preset number of iterations, return to execute the acquisition of training text until the number of iterations meets the preset number of iterations, and obtain the trained clustering sub-model, and the trained text classification model includes the trained feature extraction sub-model and the trained clustering sub-model. In this way, after obtaining the text features of the text, the feature extraction sub-model can further enhance the representation of the text features to obtain enhanced text features, and the clustering sub-model can accurately classify different texts through the enhanced text features. Therefore, the trained feature extraction sub-model and the trained clustering sub-model cooperate with each other to achieve the classification of different types of texts. Compared with the method of labeling or keyword classification in the related art, the trained text classification model in the present application is more efficient in text classification.

[0217] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0218] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0219] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0220] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0221] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0222] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0223] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0224] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0225] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0226] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0227] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.

Claims

1. A text classification model training method, characterized in that: The text classification model includes a feature extraction sub-model and a clustering sub-model, and the method includes: Obtain training texts, and obtain target text encoding features corresponding to each text in the training texts; Input each of the target text encoding features into the trained feature extraction sub-model, and output the enhanced text features corresponding to each of the target text encoding features; Inputting the enhanced text features into a clustering sub-model, and outputting a predicted text category corresponding to the enhanced text features; Performing category center iteration on the clustering sub-model according to the predicted text category and the label category to obtain an updated clustering sub-model; When the number of iterations does not meet the preset number of iterations, return to execute to obtain training text until the number of iterations meets the preset number of iterations, and obtain the trained clustering sub-model. The trained text classification model includes the trained feature extraction sub-model and the trained clustering sub-model.

2. The text classification model training method according to claim 1, characterized in that: Before obtaining the training text, the method further includes: Obtaining sample text, and obtaining text encoding features corresponding to each text in the sample text; Input each of the text encoding features into the feature extraction sub-model, and output the predicted text features corresponding to each of the text encoding features; Determine the loss value between each of the predicted text features and the corresponding label text feature. When the loss value is greater than a preset loss value, adjust the model parameters of the feature extraction sub-model to implement iteration of the feature extraction sub-model, and return to execute to obtain sample text until the loss value is no greater than the preset loss value, thereby obtaining a trained feature extraction sub-model.

3. The text classification model training method according to claim 2, characterized in that: The obtaining of the text encoding feature corresponding to each text in the sample text includes: Performing text splitting processing on the sample text to obtain multiple split texts; Encode each split text to obtain text features corresponding to each split text; Determine a text feature sequence composed of a plurality of text features, and perform position coding on each text feature according to the text feature sequence to obtain a position coding feature; A text encoding feature corresponding to each text in the sample text is generated according to each text feature and the position encoding feature corresponding to each text feature.

4. The text classification model training method according to claim 2, characterized in that: The feature extraction sub-model includes a plurality of attention modules, and each of the text encoding features is input into the feature extraction sub-model, and a predicted text feature corresponding to each of the text encoding features is output, including: Input each of the text encoding features into a first attention module to obtain an attention feature output by the first attention module; Input the attention feature output by the first attention module into the next attention module to obtain the attention feature output by the next attention module; Repeatedly input the attention features output by the previous attention module into the next attention module until the last attention module outputs the predicted text features corresponding to each of the text encoding features.

5. The text classification model training method according to claim 4, characterized in that: The attention module includes an attention submodule, a feedforward network submodule, a first residual and normalization submodule, and a second residual and normalization submodule, and each of the text encoding features is input into the first attention module to obtain the attention feature output by the first attention module, including: Input each of the text encoding features into the attention submodule of the first attention module, and output the sub-attention feature corresponding to each of the text encoding features; Input the text encoding feature and the sub-attention feature into the first residual and normalization sub-module of the first attention module, and output the first extracted feature corresponding to each text encoding feature; Inputting the first extracted features into the feedforward network submodule of the first attention module, and outputting the second extracted features corresponding to each text encoding feature; The second extracted feature and the sub-attention feature are input into the second residual and normalization sub-module of the first attention module, and the attention feature corresponding to each text encoding feature is output.

6. The text classification model training method according to claim 5, characterized in that: The step of repeatedly inputting the attention feature output by the previous attention module into the next attention module until the last attention module outputs the predicted text feature corresponding to each of the text encoding features comprises: Repeatedly input the attention feature output by the previous attention module into the attention submodule of the next attention module, and output the sub-attention feature corresponding to each text encoding feature; Input the sub-attention features into the feedforward network sub-module of the latter attention module, and output the extracted features corresponding to each sub-attention feature; The extracted features and the sub-attention features are input into the residual and normalization sub-module of the subsequent attention module to obtain the attention features output by the first attention module, until the last attention module outputs the predicted text features corresponding to each of the text encoding features.

7. The text classification model training method according to claim 1, characterized in that: The performing category center iteration on the clustering sub-model according to the predicted text category and the label category to obtain an updated clustering sub-model includes: Get the category center corresponding to each label category; determining a distance between the predicted text category and each category center; Determine the nearest target category center and the target label category of the target category center; The predicted text category is divided into the target label category, and the target category center is updated to obtain an updated category center corresponding to the target label category, thereby obtaining an updated clustering sub-model.

8. A text classification model training device, characterized in that: The text classification model includes a feature extraction sub-model and a clustering sub-model, and the device includes: An acquisition module is used to acquire training texts and acquire target text encoding features corresponding to each text in the training texts; An extraction module, used for inputting each of the target text encoding features into the trained feature extraction sub-model, and outputting the enhanced text features corresponding to each of the target text encoding features; A clustering module, used for inputting the enhanced text features into a clustering sub-model and outputting a predicted text category corresponding to the enhanced text features; An updating module, used for performing category center iteration on the clustering sub-model according to the predicted text category and the label category to obtain an updated clustering sub-model; The iteration module is used to return to execute the acquisition of training text when the number of iterations does not meet the preset number of iterations, until the number of iterations meets the preset number of iterations, and obtain the trained clustering sub-model. The trained text classification model includes the trained feature extraction sub-model and the trained clustering sub-model.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor to execute the text classification model training method according to any one of claims 1 to 7.

10. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the text classification model training method described in any one of claims 1 to 7 is implemented.

Images