A text classification method, a terminal device and a storage medium

Abstract

The application relates to a text classification method, a terminal device and a storage medium. In the method, a feature acquisition process of a first domain text is taken as a first task in continuous learning, a feature acquisition process of a next domain text is taken as a next task in continuous learning, a linear transformation result of the feature of the next domain text is obtained based on continuous learning, multi-classification calculation is performed through two full connection layers, a judgment result of whether the next domain text is a start position of an entity and whether the next domain text is an end position of the entity is obtained, and a category to which the next domain text belongs is judged through a classification network according to a result pair composed of the start position and the end position in the next domain text. The application greatly improves the recognition efficiency of the entity word cross-domain problem faced by the named entity recognition task of different domains.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of text classification, and in particular to a text classification method, a terminal device and a storage medium. BACKGROUND

[0002] Named entity recognition is a very important information extraction task in natural language processing, which obtains key information in text by identifying several kinds of word categories defined by human beings, that is, information extraction. Therefore, complex structured or unstructured text can be understood, and important key information can be extracted. The general way is to classify each character in the text to determine whether it belongs to the defined entity category, which is sequence labeling. In addition, the start position and end position of a certain entity category in the text are determined to determine the entity word, which is a pointer network.

[0003] Entity categories are defined by human beings, and there are a large number of words belonging to such entity categories, and even the expressions in different fields are different. Therefore, for the named entity recognition task in different fields, it is difficult to use a model to cross-domain text named entity recognition information extraction.

[0004] Although there are some cross-domain methods to transfer the information learned by other models to new models (such as transfer learning and knowledge distillation), they cannot be well adapted to the task of named entity recognition, because named entity recognition even across domains needs a single model to have its own decoding method, and if different decoders are used, the existing model is not convenient to train. SUMMARY

[0005] In order to solve the above problems, the present application provides a text classification method, a terminal device and a storage medium.

[0006] The specific scheme is as follows:

[0007] A text classification method comprises the following steps:

[0008] S1: extracting the features of a domain text by a pre-trained text feature extraction model corresponding to the first domain for the text of the first domain;

[0009] S2: obtaining the features of the next domain text by a pre-trained text feature extraction model corresponding to the next domain when receiving the text of the next domain;

[0010] S3: The feature extraction process of the first domain text is taken as the first task in continuous learning. Based on the index i of the next domain, the feature extraction process of the next domain text is taken as the i-th task in continuous learning. Based on continuous learning, the linear transformation result of the features of the next domain text is obtained; i = 2, 3, 4, ...

[0011] S4: The linear transformation result of the next domain text is processed through two fully connected layers for multi-classification calculation to obtain the determination results of whether the next domain text is the start position and whether it is the end position of an entity.

[0012] S5: Based on the result pair consisting of the start and end positions in the next domain text, the classification network determines the category to which the next domain text belongs;

[0013] S6: When receiving text from other domains, repeat steps S2-S5 to determine its category.

[0014] Furthermore, the text feature extraction model includes a named entity extraction network and a key information extraction network. After the input text is processed by the named entity extraction network to obtain the text vector, the key information features of the text are obtained based on the text vector, and the key information features are used as the features of the text.

[0015] Furthermore, the method for obtaining key information features is as follows: input the text vector into a bidirectional long short-term memory network model to obtain the sequence information of the text; based on the sequence information of the text, extract the relatively important key information in the text through an attention mechanism.

[0016] Furthermore, the batch_size parameter in the named entity recognition model is set to 8.

[0017] A text classification terminal device includes a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the method described above in the embodiments of the present invention.

[0018] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the method described above in the embodiments of the present invention.

[0019] This invention employs the above-described technical solution, transferring information learned from the previous task to the next task within the same text feature extraction layer. This significantly improves the recognition efficiency for named entity recognition tasks across different domains, addressing the cross-domain issue. In practical named entity recognition systems, it solves the inefficiency of requiring separate training of multiple named entity recognition models and the corresponding annotation of large amounts of data from different domains. Furthermore, by using a pointer network approach, it effectively adapts to the consistency of continuous learning while improving the accuracy of named entity recognition within a single domain, achieving a significant performance improvement in practical production. Attached Figure Description

[0020] Figure 1 The diagram shown is a flowchart of Embodiment 1 of the present invention.

[0021] Figure 2 The diagram shown is a schematic of the network structure for continuous learning in this embodiment. Detailed Implementation

[0022] To further illustrate the various embodiments, the present invention provides accompanying drawings. These drawings are part of the disclosure of the present invention, primarily used to illustrate the embodiments, and can be used in conjunction with the relevant descriptions in the specification to explain the operating principles of the embodiments. With reference to these drawings, those skilled in the art should be able to understand other possible implementations and the advantages of the present invention.

[0023] The present invention will now be further described in conjunction with the accompanying drawings and specific embodiments.

[0024] Example 1:

[0025] This invention provides a text classification method, such as... Figure 1 As shown, the method includes the following steps:

[0026] S1: Extract the features of the text in the first domain using a pre-trained text feature extraction model corresponding to the first domain.

[0027] In this embodiment, the text feature extraction model includes a named entity extraction network and a key information extraction network. After the input text is processed by the named entity extraction network to obtain the text vector (embedding), the key information features of the text are obtained based on the text vector, and the key information features are used as the features of the text.

[0028] Natural Language Processing (NLP) requires representing text as seen by humans by learning the co-occurrence probabilities of words. This text representation is a major research area. Through learning text representations, we can obtain vectors containing text features (i.e., text vectors). For example, in the sentences "This apple is delicious" and "This watermelon is very sweet," learning that the meanings of the words "apple" and "watermelon" are similar demonstrates that these two words have similar meanings. Thus, the computer can learn to interpret these two texts, recognizing their semantic similarity. Since model training is done in batches using a matrix approach, the parameter `batch_size` is set (manually set, typically 64). However, due to the characteristics of continuous learning algorithms, multiple model parameters are loaded into memory simultaneously; therefore, in this embodiment, `batch_size` is set to 8.

[0029] In addition, each token (character) of the learned text representation is represented as a vector, and the length of the vector, embed_dim, is 256 or 512 (set manually). Therefore, the text vector representation obtained by the model is a high-dimensional tensor of size [batch_size, max_length, embed_dim].

[0030] In this embodiment, the key information features are obtained by inputting the text vector into a bidirectional long short-term memory network (Bi-LSTM) model to obtain the sequence information of each text. Then, based on the sequence information of the text, the relatively important key information in the text is extracted through the attention mechanism to obtain the key information features of the text.

[0031] The reason for extracting key information features instead of directly using text vectors as text features is that text vectors are learned from each text token by a named entity recognition model, and they do not possess semantic information from the current text context. To learn the semantic information of the current text context, it is necessary to first learn the sequence information of the text using a Bi-LSTM model, and then use an attention mechanism to learn the key information in the text, reinforcing important information and weakening unimportant information. The resulting output is denoted as h1, i.e. Figure 2 As shown

[0032] S2: After receiving the text of the next domain, the features of the text of the next domain are obtained through the pre-trained text feature extraction model corresponding to the next domain.

[0033] S3: The feature extraction process of the first domain text is taken as the first task in continuous learning. Based on the index i of the next domain, the feature extraction process of the next domain text is taken as the i-th task in continuous learning. Based on continuous learning, the linear transformation result of the features of the next domain text is obtained; i = 2, 3, 4, ...

[0034] If the next domain is the second domain, then the feature acquisition process of the text in the second domain is taken as the second task in the continuous learning process, and so on.

[0035] like Figure 2 As shown, in continuous learning, when a new task is added, a new training process is added. The text representation logits (i.e., features) obtained from the previous task are combined with the text representation logits of the current task and weighted summed. The result is then input into the subsequent linear transformation to obtain the linear transformation result.

[0036] S4: The linear transformation result of the next domain text is processed through two fully connected (Dense) layers for multi-classification calculation to obtain the determination results of whether the next domain text is the start position (start_index) and whether it is the end position (end_index) of an entity.

[0037] Step S4 belongs to the pointer network. Since the pointer network uses fully connected layers, it maintains that the features learned in successive learning are combined in a linear transformation manner. Therefore, due to the "convexity" of the transformation function, subsequent tasks in successive learning can learn the knowledge and features of previous tasks.

[0038] S5: Based on the result pair consisting of the start and end positions in the next domain text, the classification network determines the category to which the next domain text belongs.

[0039] In this embodiment, a softmax network is used for classification.

[0040] S6: When receiving text from other domains, repeat steps S2-S5 to determine its category.

[0041] This invention significantly improves the efficiency of named entity recognition (NENT) tasks across different domains by passing information learned from the previous task to the next task within the same text feature extraction layer. In practical NENT systems, it solves the inefficiency of requiring multiple NENT models to be trained separately and large amounts of labeled data from different domains. Furthermore, by using a pointer network approach, it effectively adapts to the consistency of continuous learning while improving the accuracy of NENT recognition within a single domain, achieving a significant increase in efficiency in practical production.

[0042] Example 2:

[0043] The present invention also provides a text classification terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps in the method embodiment described above in Embodiment 1 of the present invention.

[0044] Furthermore, as an executable solution, the text classification terminal device can be a computing device such as a desktop computer, laptop, handheld computer, or cloud server. The text classification terminal device may include, but is not limited to, a processor and memory. Those skilled in the art will understand that the above-described structure of the text classification terminal device is merely an example and does not constitute a limitation on the text classification terminal device. It may include more or fewer components than described above, or combine certain components, or different components. For example, the text classification terminal device may also include input / output devices, network access devices, buses, etc., and this embodiment of the invention does not limit this.

[0045] Furthermore, as an executable solution, the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc. The processor is the control center of the text classification terminal device, connecting all parts of the text classification terminal device via various interfaces and lines.

[0046] The memory can be used to store the computer programs and / or modules. The processor implements various functions of the text classification terminal device by running or executing the computer programs and / or modules stored in the memory and by calling the data stored in the memory. The memory may mainly include a program storage area and a data storage area. The program storage area may store the operating system and at least one application program required for a function; the data storage area may store data created based on the use of the mobile phone, etc. In addition, the memory may include high-speed random access memory and may also include non-volatile memory, such as hard disk, RAM, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0047] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the method described in the embodiments of the present invention.

[0048] If the modules / units integrated in the text classification terminal device are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), and a software distribution medium, etc.

[0049] Although the invention has been specifically shown and described in conjunction with preferred embodiments, those skilled in the art should understand that various changes in form and detail may be made to the invention without departing from the spirit and scope of the invention as defined in the appended claims, all of which shall be within the scope of protection of the invention.

Claims

1. A text classification method, characterized in that, Includes the following steps: S1: Extract the features of the text in the first domain using a pre-trained text feature extraction model corresponding to the first domain; S2: After receiving the text of the next domain, the features of the text of the next domain are obtained through the pre-trained text feature extraction model corresponding to the next domain. S3: Take the feature extraction process of the first domain text as the first task in continuous learning, and take the feature extraction process of the next domain text as the i-th task in continuous learning according to the index i of the next domain. Based on continuous learning, obtain the linear transformation result of the features of the next domain text; i=2,3,4,…; In continuous learning, when a new task is added, the features of the domain text obtained from the previous task are combined with the features of the domain text of the current task, weighted and summed, and then input into the subsequent linear transformation to obtain the linear transformation result. S4: The linear transformation result of the next domain text is processed through two fully connected layers for multi-classification calculation to obtain the determination results of whether the next domain text is the start position and whether it is the end position of an entity. S5: Based on the result pair consisting of the start and end positions in the next domain text, the classification network determines the category to which the next domain text belongs; S6: When receiving text from other domains, repeat steps S2-S5 to determine its category.

2. The text classification method according to claim 1, characterized in that: The text feature extraction model includes a named entity extraction network and a key information extraction network. After the input text is processed by the named entity extraction network to obtain the text vector, the key information features of the text are obtained based on the text vector and used as the text features.

3. The text classification method according to claim 1, characterized in that: The method for obtaining key information features is as follows: input the text vector into a bidirectional long short-term memory network model to obtain the sequence information of the text; based on the sequence information of the text, extract the relatively important key information in the text through an attention mechanism.

4. The text classification method according to claim 1, characterized in that: In the named entity recognition model, the value of the parameter batch_size is set to 8.

5. A text classification terminal device, characterized in that: It includes a processor, a memory, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program to implement the steps of the method as described in any one of claims 1 to 4.

6. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 4.

Images