Text classification method, device, terminal and medium based on classification network model
Through multi-source information fusion and feature extraction technology based on classification network model, the classification inaccurate classification problems caused by differences in text diversity and writing habits within the enterprise/company are solved, and the accurate classification of text is achieved.
Patent Information
- Application Number
- CN202110877266.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-07-31
- Publication Date
- 2024-11-01
- Estimated Expiration
- 2041-07-31
AI Technical Summary
The prior art is difficult to accurately handle the diversity of texts within the enterprise/company and different writing habits in text classification, resulting in inaccurate classification.
Using a classification network model-based method, by obtaining multi-source information text (including text, suffix, title and other information text), using multi-source information fusion and feature extraction classification network to build an initial model, preprocess and train, and generate a classification network Model for text classification.
It improves the accuracy of text classification, can adapt to different types of writing styles, fully utilizes information other than the main text, avoids the problem of difficult to extract global features of the text, and realizes accurate text classification.
Smart Images

Figure CN113672728B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of natural language processing, and in particular to a text classification method, device, terminal and medium based on a classification network model. Background Art
[0002] Natural language processing is an important direction in the fields of computer science and artificial intelligence. It studies various theories and methods that can achieve effective communication between humans and computers using natural language. Natural language processing is a science that integrates linguistics, computer science, and mathematics. Text classification in the field of natural language processing is an important and meaningful task in practice. Many companies and enterprises use it to determine the categories of internal texts and achieve text classification.
[0003] Currently, various neural network algorithms for text classification tasks emerge in an endless stream. However, these algorithms lose the information about the order in which words appear in the text, only symbolize the words, and do not consider the semantic connection between words, making it difficult to extract the global features of the text, resulting in inaccurate text classification. In addition, due to the particularity of internal text data, writing habits vary from enterprise to enterprise, and even different people in the same enterprise / company can write texts of the same category in very different styles. Traditional neural networks are weak in processing such non-standard internal texts, which further affects the accuracy of text classification.
[0004] Therefore, how to accurately classify various texts of enterprises / companies is a technical problem that needs to be solved urgently. Summary of the invention
[0005] The main purpose of the present invention is to provide a text classification method, device, terminal and medium based on a classification network model, aiming to solve the technical problem of how to accurately classify various texts of an enterprise / company in the prior art.
[0006] To achieve the above object, the present invention provides a text classification method based on a classification network model, and the text classification method based on the classification network model includes:
[0007] Acquire multiple types of sample files, and convert each of the sample files into multi-source information texts, wherein the multi-source information texts at least include body texts, suffix texts, title texts, and other information texts;
[0008] Calling the classification initial model constructed based on the multi-source information fusion and feature extraction classification network, preprocessing the multi-source information text, generating labeled training data and transmitting it to the classification initial model, training the classification initial model, and obtaining a classification network model;
[0009] The text to be classified is classified and identified based on the classification network model to determine the text category to which the text to be classified belongs.
[0010] Optionally, the preprocessing includes word segmentation and cleaning, and the step of preprocessing the multi-source information text to generate labeled training data and transmit it to the classification initial model includes:
[0011] Performing word segmentation processing on each of the multi-source information texts according to a preset word segmentation rule to obtain a word segmentation group corresponding to each of the multi-source information texts, and filtering and cleaning the words in each of the word segmentation groups to obtain a selected word segmentation group;
[0012] Encoding each of the selected word groups based on a preset length to generate a word embedding vector corresponding to each of the multi-source information texts;
[0013] A category embedding vector corresponding to a preset number of categories is constructed, and after the dimension of the category embedding vector is set to the dimension of the word embedding vector, the word embedding vector and the category embedding vector are generated as labeled training data and transmitted to the classification initial model.
[0014] Optionally, the step of generating labeled training data and transmitting it to the initial classification model, training the initial classification model, and obtaining the classification network model includes:
[0015] After generating labeled training data, transmitting the training data to the initial classification model, training the initial classification model, and calculating the cross entropy classification loss function value of the initial classification model;
[0016] According to the cross entropy classification loss function value, it is judged whether the initial classification model reaches the convergence condition. If the convergence condition is reached, the training of the initial classification model is completed, and based on the target model parameters obtained by the training, the initial classification model is generated as a classification network model.
[0017] Optionally, after the step of judging whether the initial classification model reaches a convergence condition according to the cross entropy classification loss function value, the following steps are included:
[0018] If the initial classification model does not meet the convergence condition, optimizing the model parameters of the initial classification model based on a preset optimization mechanism;
[0019] Based on the optimized model parameters, the step of training the initial classification model is performed until the initial classification model reaches a convergence condition.
[0020] Optionally, the step of training the initial classification model and calculating the cross entropy classification loss function value of the initial classification model includes:
[0021] Processing the training data based on the classification initial model, generating a type information vector at the feature extraction classification layer of the classification initial model;
[0022] The type information vector is transmitted to the fully connected layer of the classification initial model, and the fully connected layer processes the type information vector to obtain the training result of the classification initial model;
[0023] The cross entropy classification loss function of the initial classification model is calculated based on the training result to obtain the cross entropy classification loss function value.
[0024] Optionally, the step of preprocessing the multi-source information text includes:
[0025] A public data set is obtained, and public data in the public data set is transmitted to the initial classification model, and the initial classification model is pre-trained to update the initial classification model.
[0026] Optionally, the step of generating labeled training data and transmitting it to the initial classification model, training the initial classification model, and obtaining the classification network model includes:
[0027] After generating labeled training data, the training data is transmitted to the classification initial model, the classification initial model is trained, and the number of training times of the classification initial model is counted;
[0028] Determine whether the number of training times matches a preset number of iterations. If so, complete the training of the initial classification model, and generate the initial classification model into a classification network model based on the target model parameters obtained during the training.
[0029] If the number of training times does not match the preset number of iterations, the model parameters of the initial classification model are optimized based on a preset optimization mechanism, and based on the optimized model parameters, the step of training the initial classification model is performed.
[0030] Furthermore, to achieve the above-mentioned purpose, the present invention also provides a text classification device based on a classification network model, and the text classification device based on the classification network model comprises:
[0031] A conversion module, used for acquiring multiple types of sample files, and converting each of the sample files into multi-source information texts, wherein the multi-source information texts at least include body texts, suffix texts, title texts and other information texts;
[0032] A calling module is used to call the classification initial model constructed based on the multi-source information fusion and feature extraction classification network, and pre-process the multi-source information text to generate labeled training data and transmit it to the classification initial model, train the classification initial model, and obtain a classification network model;
[0033] The determination module is used to classify and identify the text to be classified based on the classification network model, and determine the text category to which the text to be classified belongs.
[0034] Furthermore, to achieve the above-mentioned purpose, the present invention also provides a text classification terminal based on a classification network model, wherein the text classification terminal based on a classification network model comprises: a memory, a processor, and a control program stored in the memory and executable on the processor, wherein when the control program is executed by the processor, the steps of the text classification method based on the classification network model as described above are implemented.
[0035] Furthermore, to achieve the above-mentioned purpose, the present invention also provides a readable storage medium, on which a control program is stored, and when the control program is executed by a processor, the steps of the text classification method based on the classification network model as described above are implemented.
[0036] The text classification method, device, terminal and medium based on the classification network model of the present invention first obtain various types of sample files of multiple enterprises / companies, and convert each sample file into a multi-source information text containing a main text, a suffix text, a title text and other information texts; then obtain the classification initial model constructed based on the multi-source information fusion and feature extraction classification network, pre-process the converted multi-information text, generate labeled training data and transmit it to the classification initial model, train the classification initial model, and obtain the classification network model; thereafter, when there is a text classification demand, the trained classification network model can be used to classify and identify the text to be classified, so as to determine the text category to which the text to be classified belongs. Among them, the multi-source information fusion makes great use of the suffix, title and other information other than the main text, and can be fully applicable to different types of writing styles, which is conducive to improving the accuracy of classification; at the same time, the feature extraction classification network can completely rely on the attention mechanism to model the global dependency relationship between input and output, avoiding the problem that the global features of the text are difficult to extract, and is also conducive to improving the accuracy of classification. Therefore, the present invention trains the initial classification model constructed by the multi-source information fusion and feature extraction classification network through the multi-source information text, generates a classification network model to classify the text, and realizes the accurate classification of the text. BRIEF DESCRIPTION OF THE DRAWINGS
[0037] Figure 1It is a structural schematic diagram of the hardware operating environment involved in the text classification terminal embodiment scheme based on the classification network model of the present invention;
[0038] Figure 2 It is a flow chart of the first embodiment of the text classification method based on the classification network model of the present invention;
[0039] Figure 3 It is a flow chart of a second embodiment of a text classification method based on a classification network model of the present invention;
[0040] Figure 4 It is a flow chart of a third embodiment of a text classification method based on a classification network model of the present invention;
[0041] Figure 5 It is a flowchart of a fourth embodiment of a text classification method based on a classification network model of the present invention;
[0042] Figure 6 A schematic diagram of a process of classifying text by a classification network model in a text classification method based on a classification network model of the present invention;
[0043] Figure 7 This is a schematic diagram of data processing by the fully connected layer of the classification network model in the text classification method based on the classification network model of the present invention.
[0044] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION
[0045] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.
[0046] The invention provides a text classification terminal based on a classification network model.
[0047] Please refer to Figure 1 , Figure 1 It is a structural schematic diagram of the server hardware operating environment involved in the text classification terminal embodiment scheme based on the classification network model of the present invention.
[0048] like Figure 1As shown, the text classification terminal based on the classification network model may include a processor 1001, such as a CPU, a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory, or a stable memory (non-volatile memory), such as a disk memory. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.
[0049] Those skilled in the art will understand that Figure 1 The hardware structure of the text classification terminal based on the classification network model shown in the figure does not constitute a limitation of the text classification terminal based on the classification network model, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.
[0050] like Figure 1 As shown, the memory 1005 as a readable storage medium may include an operating system, a network communication module, a user interface module and a control program. The operating system is a program for managing and controlling the text classification terminal and software resources based on the classification network model, and supports the operation of the network communication module, the user interface module, the control program and other programs or software; the network communication module is used to manage and control the network interface 1004; the user interface module is used to manage and control the user interface 1003.
[0051] exist Figure 1 In the hardware structure of the text classification terminal based on the classification network model shown, the network interface 1004 is mainly used to connect to the background server and communicate data with the background server; the user interface 1003 is mainly used to connect to the client (user end) and communicate data with the client; the processor 1001 can call the control program stored in the memory 1005 and perform the following operations:
[0052] Acquire multiple types of sample files, and convert each of the sample files into multi-source information texts, wherein the multi-source information texts at least include body texts, suffix texts, title texts, and other information texts;
[0053] Calling the classification initial model constructed based on the multi-source information fusion and feature extraction classification network, preprocessing the multi-source information text, generating labeled training data and transmitting it to the classification initial model, training the classification initial model, and obtaining a classification network model;
[0054] The text to be classified is classified and identified based on the classification network model to determine the text category to which the text to be classified belongs.
[0055] Furthermore, the preprocessing includes word segmentation and cleaning, and the step of preprocessing the multi-source information text to generate labeled training data and transmit it to the classification initial model includes:
[0056] Performing word segmentation processing on each of the multi-source information texts according to a preset word segmentation rule to obtain a word segmentation group corresponding to each of the multi-source information texts, and filtering and cleaning the words in each of the word segmentation groups to obtain a selected word segmentation group;
[0057] Encoding each of the selected word groups based on a preset length to generate a word embedding vector corresponding to each of the multi-source information texts;
[0058] A category embedding vector corresponding to a preset number of categories is constructed, and after the dimension of the category embedding vector is set to the dimension of the word embedding vector, the word embedding vector and the category embedding vector are generated as labeled training data and transmitted to the classification initial model.
[0059] Furthermore, the steps of generating labeled training data and transmitting it to the classification initial model, training the classification initial model, and obtaining the classification network model include:
[0060] After generating labeled training data, transmitting the training data to the initial classification model, training the initial classification model, and calculating the cross entropy classification loss function value of the initial classification model;
[0061] According to the cross entropy classification loss function value, it is judged whether the initial classification model reaches the convergence condition. If the convergence condition is reached, the training of the initial classification model is completed, and based on the target model parameters obtained by the training, the initial classification model is generated as a classification network model.
[0062] Further, after the step of determining whether the initial classification model reaches the convergence condition according to the cross entropy classification loss function value; the processor 1001 may call the control program stored in the memory 1005 and perform the following operations:
[0063] If the initial classification model does not meet the convergence condition, optimizing the model parameters of the initial classification model based on a preset optimization mechanism;
[0064] Based on the optimized model parameters, the step of training the initial classification model is performed until the initial classification model reaches a convergence condition.
[0065] Furthermore, the step of training the initial classification model and calculating the cross entropy classification loss function value of the initial classification model includes:
[0066] Processing the training data based on the classification initial model, generating a type information vector at the feature extraction classification layer of the classification initial model;
[0067] The type information vector is transmitted to the fully connected layer of the classification initial model, and the fully connected layer processes the type information vector to obtain the training result of the classification initial model;
[0068] The cross entropy classification loss function of the initial classification model is calculated based on the training result to obtain the cross entropy classification loss function value.
[0069] Furthermore, before the step of preprocessing the multi-source information text, the processor 1001 may call the control program stored in the memory 1005 and perform the following operations:
[0070] A public data set is obtained, and public data in the public data set is transmitted to the initial classification model, and the initial classification model is pre-trained to update the initial classification model.
[0071] Furthermore, the steps of generating labeled training data and transmitting it to the classification initial model, training the classification initial model, and obtaining the classification network model include:
[0072] After generating labeled training data, the training data is transmitted to the classification initial model, the classification initial model is trained, and the number of training times of the classification initial model is counted;
[0073] Determine whether the number of training times matches a preset number of iterations. If so, complete the training of the initial classification model, and generate the initial classification model into a classification network model based on the target model parameters obtained during the training.
[0074] If the number of training times does not match the preset number of iterations, the model parameters of the initial classification model are optimized based on a preset optimization mechanism, and based on the optimized model parameters, the step of training the initial classification model is performed.
[0075] The implementation of the text classification terminal based on the classification network model of the present invention is basically the same as the embodiments of the text classification method based on the classification network model described below, and will not be described in detail here.
[0076] The present invention provides a text classification method based on a classification network model. Figure 2 , Figure 2 It is a flowchart of the first embodiment of the text classification method based on the classification network model of the present invention.
[0077] The embodiment of the present invention provides an embodiment of a text classification method based on a classification network model. It should be noted that although a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than that shown here. Specifically, the text classification method based on a classification network model in this embodiment includes:
[0078] Step S10, obtaining multiple types of sample files, and converting each of the sample files into a multi-source information text, wherein the multi-source information text at least includes a body text, a suffix text, a title text, and other information texts;
[0079] The text classification method of this embodiment is applicable to the above-mentioned text classification terminal, which can be a mobile terminal such as a mobile phone, a tablet computer, a laptop computer, or a fixed terminal such as a desktop computer, a smart TV, etc. By deploying a classification network model constructed and trained by a multi-source information fusion and feature extraction classification network on the text classification terminal, the internal texts of companies, enterprises and institutions can be identified to classify and manage the texts. Among them, the divided categories can be set according to needs, such as setting various categories including financial report texts, rules and regulations texts, reward and punishment notification texts, appointment texts, etc.
[0080] Specifically, first obtain various types of sample files from multiple companies, enterprises and institutions. The types include but are not limited to doc, docx, ppt, pptx, xls, xlsx, png, jpg, tiff, dwg, pdf, etc., and the sample files contain readable text content. Thereafter, parse the readable text content of each sample file according to the different information types contained therein, and convert the sample file into a multi-source information text, which is a text file containing readable text. The information type includes at least body information, title information, suffix information, and other text information such as author and time; based on these information types, parse the sample file into a multi-source information text containing at least multi-source body text, suffix text, title text and other information text for later model training.
[0081] Step S20, calling the classification initial model constructed based on the multi-source information fusion and feature extraction classification network, and preprocessing the multi-source information text, generating labeled training data and transmitting it to the classification initial model, training the classification initial model, and obtaining a classification network model;
[0082] Furthermore, a classification initial model is constructed in advance based on multi-source information fusion and feature extraction classification network; multi-source information fusion is a multi-level and multi-faceted processing process of multiple types of information, including detection, combination and estimation of multi-source data to improve the accuracy of state and characteristic estimation; feature extraction classification network, i.e. Transformers, is a network that performs feature extraction and classification, which contains multiple sub-Transformer modules. This embodiment calls the pre-set classification initial model, and passes the labeled training data generated by preprocessing the multi-source information text to the classification initial model to train the classification initial model and obtain a classification network model for text classification.
[0083] Understandably, in order to ensure that the classification network model has an accurate classification effect, it is usually necessary to generate and annotate a large amount of training data to train the initial classification model. However, the annotation of a large amount of training data often consumes a lot of manpower and material resources, the annotation efficiency is low and it is prone to errors. In order to avoid such problems, this embodiment is provided with a mechanism of first pre-training with public data and then adjusting the training with a relatively small amount of annotated training data. Specifically, the step of pre-processing the multi-source information text includes:
[0084] Step a, obtaining a public data set, and transmitting the public data in the public data set to the initial classification model, and pre-training the initial classification model to update the initial classification model.
[0085] Furthermore, a public data set is obtained through a network channel, and the structure of each public data in the public data set is adjusted according to the network structure of the initial classification model. Then, the adjusted public data is transmitted to the initial classification model, the initial model is pre-trained, the model parameters are obtained, and the initial classification model is updated. Thereafter, the updated initial model is continuously trained with labeled training data generated by multi-source information texts to obtain the final classification network model for text classification.
[0086] Step S30, classifying and identifying the text to be classified based on the classification network model, and determining the text category to which the text to be classified belongs.
[0087] Furthermore, when there is a need to classify text, the text to be classified is transmitted to the text classification terminal as the text to be classified. The text classification terminal classifies and identifies the text to be classified based on the trained classification network model to obtain a category identifier; then, according to the text categories corresponding to each category identifier, the text category to which the text to be classified belongs is determined. For example, a company divides its text into 13 categories, namely category A, category B, category C...category M, and the corresponding category identifiers are 1, 2, 3...13; if the category identifier obtained by the classification network model is 3, it can be determined that the text category to which the text to be classified belongs is category C. Thereafter, the classified text is classified and managed according to the management rules corresponding to the text category, such as whether the text category needs to be encrypted, the encryption level, and other factors. In this way, while ensuring the accurate classification of various types of files, the orderly management of various types of files is achieved.
[0088] Please refer to Figure 6 ,exist Figure 6 , which shows a specific process of text recognition in a specific embodiment, wherein a common format file is a text to be classified, which is converted into a readable text containing multi-source information such as the main text, suffix, title, and others, i.e., a multi-source information text, by a conversion module; then, the multi-source information text is preprocessed and encoded to obtain word embedding vectors n1×256, n2×256, n3×256, and ni×256; then, a category embedding vector n matching the number of categories is constructed. class ×256, the word embedding vector and the category embedding vector are formed as the data to be processed and transmitted to the classification network model, in which the feature extraction classification layer Transformers generates the type information vector, and then the classification network model's fully connected layer FC processes the type information vector to obtain the output result of the category to which the common format file belongs.
[0089] The text classification method based on the classification network model of the present invention first obtains multiple types of sample files of multiple enterprises / companies, and converts each sample file into a multi-source information text containing a main text, a suffix text, a title text and other information texts; then obtains the classification initial model constructed based on the multi-source information fusion and feature extraction classification network, pre-processes the converted multi-information text, generates labeled training data and transmits it to the classification initial model, trains the classification initial model, and obtains the classification network model; thereafter, when there is a text classification demand, the trained classification network model can be used to classify and identify the text to be classified, so as to determine the text category to which the text to be classified belongs. Among them, the multi-source information fusion makes great use of the suffix, title and other information other than the main text, and can be fully applicable to different types of writing styles, which is conducive to improving the accuracy of classification; at the same time, the feature extraction classification network can completely rely on the attention mechanism to model the global dependency relationship between input and output, avoiding the problem that the global features of the text are difficult to extract, and is also conducive to improving the accuracy of classification. Therefore, the present invention trains the initial classification model constructed by the multi-source information fusion and feature extraction classification network through the multi-source information text, generates a classification network model to classify the text, and realizes the accurate classification of the text.
[0090] For further information, please refer to Figure 3 Based on the first embodiment of the text classification method based on the classification network model of the present invention, a second embodiment of the text classification method based on the classification network model of the present invention is proposed.
[0091] The difference between the second embodiment of the text classification method based on the classification network model and the first embodiment of the text classification method based on the classification network model is that the preprocessing includes word segmentation and cleaning, and the step of preprocessing the multi-source information text to generate labeled training data and transmit it to the classification initial model includes:
[0092] Step S21, performing word segmentation processing on each of the multi-source information texts according to a preset word segmentation rule to obtain a word segmentation group corresponding to each of the multi-source information texts, and filtering and cleaning the words in each of the word segmentation groups to obtain a selected word segmentation group;
[0093] Step S22, encoding each of the selected word groups based on a preset length to generate a word embedding vector corresponding to each of the multi-source information texts;
[0094] Step S23, constructing a category embedding vector corresponding to a preset number of categories, and after setting the dimension of the type embedding vector to the dimension of the word embedding vector, generating the word embedding vector and the category embedding vector as labeled training data and transmitting them to the classification initial model.
[0095] In this embodiment, after preprocessing each piece of multi-source text information, encoding is performed to form a word embedding vector and a category embedding vector representing the category, which together form labeled training data, and the initial classification model updated with public data is trained.
[0096] Specifically, the preprocessing at least includes word segmentation processing and cleaning processing. For the word segmentation processing, according to the preset word segmentation rules, each multi-source information text is segmented, and the sentences corresponding to each multi-source information text are divided into multiple words, forming a word segmentation group for each multi-source information text. At the same time, each word segmentation group is filtered and cleaned, and meaningless words such as conjunctions and modal particles like "and", "with", "of", "ah", etc. are removed to obtain a selected word segmentation group. Among them, one multi-source information text corresponds to one selected word segmentation group, and the semantics of each word in the selected word segmentation group reflects the text meaning of the multi-source information text.
[0097] Furthermore, each selected word segmentation group is constructed into a word library, and a preset length is set in advance according to requirements. Words of the preset length are selected from each selected word segmentation group for encoding operations, and the word vectors of each selected word segmentation group are obtained as the word embedding vectors corresponding to each multi-source information text. As Figure 6 shown in the specific embodiment, the word embedding vector is 256-dimensional and can be expressed as n t ×256.
[0098] Even further, the number of categories divided in advance according to requirements is used as the preset number of categories, and the same number of category embedding vectors as the preset number of categories are constructed, named classification Tokens. Specifically, it can be expressed as: Moreover, the category embedding vector is a learnable embedding vector. During the training process, it is first initialized, and as the training progresses, learning is achieved to learn the weight encoding that specifically reflects the belonging category. Among them, the dimension of the category embedding vector can be first set to be the same as the dimension of the word embedding vector, and then the word embedding vector and the type embedding vector are generated into the training data to be labeled. Furthermore, labels are added to the training data to be labeled to generate labeled training data and transmitted to the initial classification model to train the initial classification model.
[0099] In this embodiment, through the preprocessing of word segmentation and cleaning of multi-source information texts, the generated selected word segmentation groups more accurately reflect the content of the multi-source information texts and the types of the texts from which the multi-source information texts are derived. Furthermore, it ensures the accuracy of training the initial classification model based on the training data to be labeled formed by the selected word segmentation groups, which is beneficial to training and generating a classification network model to accurately classify texts.
[0100] Furthermore, please refer to Figure 4Based on the first or second embodiment of the text classification method based on the classification network model of the present invention, a third embodiment of the text classification method based on the classification network model of the present invention is proposed.
[0101] The difference between the third embodiment of the text classification method based on the classification network model and the first or second embodiment of the text classification method based on the classification network model is that the steps of generating labeled training data and transmitting it to the classification initial model, training the classification initial model, and obtaining the classification network model include:
[0102] Step S24, after generating labeled training data, transmitting the training data to the initial classification model, training the initial classification model, and calculating the cross entropy classification loss function value of the initial classification model;
[0103] This embodiment uses the cross entropy classification loss function to determine whether the trained initial classification model has reached the convergence condition for the end of training. Specifically, after the multi-source information text is generated as labeled training data, the training data is transmitted to the initial classification model, processed by the initial classification model, and the classification model is trained. In addition, after each training, the function value of the cross entropy classification loss function of the initial classification model is calculated, and the classification accuracy is evaluated by the cross entropy classification loss function value; wherein, the smaller the function value, the more accurate the classification, and vice versa.
[0104] Furthermore, the step of training the initial classification model and calculating the cross entropy classification loss function value of the initial classification model includes:
[0105] Step S241, processing the training data based on the classification initial model, and generating a type information vector at the feature extraction classification layer of the classification initial model;
[0106] Step S242, transmitting the type information vector to the fully connected layer of the classification initial model, and the fully connected layer processes the type information vector to obtain the training result of the classification initial model;
[0107] Step S243, calculating the cross entropy classification loss function of the initial classification model based on the training result to obtain the cross entropy classification loss function value.
[0108] Furthermore, for the classification initial model based on multi-source information fusion and feature extraction classification network, the classification initial model includes feature extraction classification layer Transformers. The classification initial model processes the training data, and generates a type information vector in the feature extraction classification layer Transformers. The type information vector contains information representing the category, and its dimension is the same as the dimension of the word embedding vector.
[0109] Furthermore, the type information vector is transmitted to the fully connected layer of the classification initial model, which processes it and outputs a one-dimensional processing result, such as Figure 7 The output processing result is the training result obtained by the classification initial model for the classification data classification processing. The cross entropy classification function of the classification initial model is calculated based on the training result to obtain the cross entropy classification loss function value that characterizes the accuracy of the classification processing.
[0110] Step S25, judging whether the initial classification model has reached the convergence condition according to the cross entropy classification loss function value, if the convergence condition is reached, completing the training of the initial classification model, and generating the initial classification model into a classification network model based on the target model parameters obtained through the training.
[0111] Furthermore, a preset value characterizing the convergence of the model is pre-set, and the preset value is used as a convergence condition. By comparing the size relationship between the cross entropy classification loss function value and the preset value, it is determined whether the initial classification model has reached the convergence condition. Specifically, if it is determined by comparison that the cross entropy classification loss function value is less than the preset value, it means that the classification accuracy of the initial classification model is high, and it is determined that the convergence condition has been reached. At this time, the training of the initial classification model is completed, and the model parameters obtained by the training are fixed as the target model parameters for processing text classification, and the initial classification model is generated as a classification network model, and the classification network model performs text classification based on the target model parameters.
[0112] Furthermore, in the case where the initial classification model does not meet the convergence condition, the step of judging whether the initial classification model meets the convergence condition according to the cross entropy classification loss function value includes:
[0113] Step S251, if the initial classification model does not meet the convergence condition, optimizing the model parameters of the initial classification model based on a preset optimization mechanism;
[0114] Step S252, based on the optimized model parameters, executing the step of training the initial classification model until the initial classification model reaches a convergence condition.
[0115] Furthermore, if it is determined by comparison that the cross entropy classification loss function value is not less than the preset value, it means that the classification accuracy of the initial classification model is not high, and it is determined that the initial classification model has not reached the convergence condition. At this time, the model parameters of the initial classification model are optimized according to the preset optimization mechanism pre-set in the initial classification model. In addition, the initial classification model is controlled to classify the training data based on the optimized model parameters to train the initial classification model, and calculate the new cross entropy classification loss function value and compare it with the preset value. In this way, once the cross entropy classification loss function value is less than the preset value, it is determined that the initial classification model has reached the convergence condition, and the initial classification model is generated as a classification network model for classifying text.
[0116] This embodiment determines whether the initial classification model reaches the convergence condition by calculating the cross entropy classification loss function value. Only after the convergence condition is reached, the initial classification model is generated as a classification network model for text classification, which is conducive to ensuring that the generated classification network model accurately classifies the text.
[0117] For further information, please refer to Figure 5 Based on the first, second or third embodiment of the text classification method based on the classification network model of the present invention, a fourth embodiment of the text classification method based on the classification network model of the present invention is proposed.
[0118] The fourth embodiment of the text classification method based on the classification network model is different from the first, second or third embodiment of the text classification method based on the classification network model in that the steps of generating labeled training data and transmitting it to the classification initial model, training the classification initial model, and obtaining the classification network model include:
[0119] Step S26, after generating the labeled training data, transmitting the training data to the classification initial model, training the classification initial model, and counting the number of training times for the classification initial model;
[0120] Step S27, determining whether the number of training times matches the preset number of iterations. If so, the training of the initial classification model is completed, and based on the target model parameters obtained through the training, the initial classification model is generated as a classification network model.
[0121] Step S28, if the number of training times does not match the preset number of iterations, the model parameters of the classification initial model are optimized based on a preset optimization mechanism, and based on the optimized model parameters, the step of training the classification initial model is executed.
[0122] This embodiment sets the maximum number of iterations to determine whether the trained initial classification model has reached the training end condition. Specifically, the maximum number of iterations is used as the preset number of iterations, and after the multi-source information text is generated as labeled training data, the training data is transmitted to the initial classification model, processed by the initial classification model, the classification model is trained, and the number of trainings performed on the initial classification model is counted. Thereafter, the counted number of trainings is compared with the preset number of iteration trainings, and by determining whether the number of trainings is greater than or equal to the preset number of iterations, it is determined whether the number of trainings matches the preset number of iterations. If the number of trainings is greater than or equal to the preset number of iterations, the two are determined to match, otherwise the two are determined to not match.
[0123] Furthermore, for the case where the number of training times matches the preset number of iterations, the training of the initial classification model is completed, and the model parameters of the trained initial classification model are fixed as the target model parameters for processing text classification, and the initial classification model is generated as a classification network model, and the classification network model performs text classification based on the target model parameters.
[0124] Furthermore, in the case where the number of training times is less than the preset number of iterations and the two do not match, the model parameters of the classification initial model are also optimized according to the preset optimization mechanism pre-set in the classification initial model. In addition, the classification initial model is controlled to classify the training data based on the optimized model parameters to train the classification initial model, and the number of training times is accumulated and compared with the preset number of iterations. This cycle is repeated. Once the number of training times is greater than or equal to the preset number of iterations, it is determined that the two match, and the classification initial model is generated as a classification network model to classify the text.
[0125] This embodiment determines whether the initial classification model meets the conditions for the end of training by counting the number of iterative training times. Only after the conditions for the end of training are met, the initial classification model is generated as a classification network model for text classification, ensuring sufficient training of the classification network model, which is conducive to accurate classification of text through the generated classification network model.
[0126] The embodiment of the present invention further provides a text classification device based on a classification network model, and the text classification device based on the classification network model includes:
[0127] A conversion module, used for acquiring multiple types of sample files, and converting each of the sample files into multi-source information texts, wherein the multi-source information texts at least include body texts, suffix texts, title texts and other information texts;
[0128] A calling module is used to call the classification initial model constructed based on the multi-source information fusion and feature extraction classification network, and pre-process the multi-source information text to generate labeled training data and transmit it to the classification initial model, train the classification initial model, and obtain a classification network model;
[0129] The determination module is used to classify and identify the text to be classified based on the classification network model, and determine the text category to which the text to be classified belongs.
[0130] Furthermore, the preprocessing includes word segmentation and cleaning, and the calling module is also used for:
[0131] Performing word segmentation processing on each of the multi-source information texts according to a preset word segmentation rule to obtain a word segmentation group corresponding to each of the multi-source information texts, and filtering and cleaning the words in each of the word segmentation groups to obtain a selected word segmentation group;
[0132] Encoding each of the selected word groups based on a preset length to generate a word embedding vector corresponding to each of the multi-source information texts;
[0133] A category embedding vector corresponding to a preset number of categories is constructed, and after the dimension of the category embedding vector is set to the dimension of the word embedding vector, the word embedding vector and the category embedding vector are generated as labeled training data and transmitted to the classification initial model.
[0134] Furthermore, the calling module is also used for:
[0135] After generating labeled training data, transmitting the training data to the initial classification model, training the initial classification model, and calculating the cross entropy classification loss function value of the initial classification model;
[0136] According to the cross entropy classification loss function value, it is judged whether the initial classification model reaches the convergence condition. If the convergence condition is reached, the training of the initial classification model is completed, and based on the target model parameters obtained by the training, the initial classification model is generated as a classification network model.
[0137] Furthermore, the calling module is also used for:
[0138] If the initial classification model does not meet the convergence condition, optimizing the model parameters of the initial classification model based on a preset optimization mechanism;
[0139] Based on the optimized model parameters, the step of training the initial classification model is performed until the initial classification model reaches a convergence condition.
[0140] Furthermore, the calling module is also used for:
[0141] A public data set is obtained, and public data in the public data set is transmitted to the initial classification model, and the initial classification model is pre-trained to update the initial classification model.
[0142] Furthermore, the calling module is also used for:
[0143] After generating labeled training data, the training data is transmitted to the classification initial model, the classification initial model is trained, and the number of training times of the classification initial model is counted;
[0144] Determine whether the number of training times matches a preset number of iterations. If so, complete the training of the initial classification model, and generate the initial classification model into a classification network model based on the target model parameters obtained during the training.
[0145] If the number of training times does not match the preset number of iterations, the model parameters of the initial classification model are optimized based on a preset optimization mechanism, and based on the optimized model parameters, the step of training the initial classification model is performed.
[0146] The specific implementation of the text classification device based on the classification network model of the present invention is basically the same as the various embodiments of the text classification method based on the classification network model mentioned above, and will not be repeated here.
[0147] The embodiment of the present invention further provides a readable storage medium having a control program stored thereon, and when the control program is executed by the processor, the steps of the text classification method based on the classification network model are implemented.
[0148] The readable storage medium of the present invention may be a computer-readable storage medium, and its specific implementation is substantially the same as the above-mentioned embodiments of the text classification method based on the classification network model, and will not be described in detail here.
[0149] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the enlightenment of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the present invention and the claims. All equivalent structures or equivalent process changes made using the contents of the specification and drawings of the present invention, or directly or indirectly used in other related technical fields, are protected by the present invention.
Claims
1. A text classification method based on classification network model, It is characterized in that The text classification method based on the classification network model includes: Acquire multiple types of sample files, and convert each of the sample files into multi-source information texts, wherein the types of the sample files include doc, docx, ppt, pptx, xls, xlsx, png, jpg, tiff, dwg, and pdf, and the multi-source information texts are text files containing readable texts, and the multi-source information texts at least include body texts, suffix texts, title texts, and other information texts; Calling the classification initial model constructed based on the multi-source information fusion and feature extraction classification network, preprocessing the multi-source information text, generating labeled training data and transmitting it to the classification initial model, training the classification initial model, and obtaining a classification network model; Classify and identify the text to be classified based on the classification network model, and determine the text category to which the text to be classified belongs; The preprocessing includes word segmentation and cleaning, and the step of preprocessing the multi-source information text to generate labeled training data and transmit it to the classification initial model includes: Performing word segmentation processing on each of the multi-source information texts according to a preset word segmentation rule to obtain a word segmentation group corresponding to each of the multi-source information texts, and filtering and cleaning the words in each of the word segmentation groups to obtain a selected word segmentation group; Encoding each of the selected word groups based on a preset length to generate a word embedding vector corresponding to each of the multi-source information texts; Constructing a category embedding vector corresponding to a preset number of categories, and after setting the dimension of the category embedding vector to the dimension of the word embedding vector, generating the word embedding vector and the category embedding vector as labeled training data and transmitting them to the classification initial model; The steps of generating labeled training data and transmitting it to the classification initial model, training the classification initial model, and obtaining the classification network model include: After generating labeled training data, transmitting the training data to the initial classification model, training the initial classification model, and calculating the cross entropy classification loss function value of the initial classification model; According to the cross entropy classification loss function value, it is judged whether the initial classification model reaches the convergence condition. If the convergence condition is reached, the training of the initial classification model is completed, and based on the target model parameters obtained by the training, the initial classification model is generated as a classification network model.
2. The text classification method based on the classification network model as claimed in claim 1, It is characterized in that After the step of judging whether the initial classification model has reached the convergence condition according to the cross entropy classification loss function value, the following steps are included: If the initial classification model does not meet the convergence condition, optimizing the model parameters of the initial classification model based on a preset optimization mechanism; Based on the optimized model parameters, the step of training the initial classification model is performed until the initial classification model reaches a convergence condition.
3. The text classification method based on the classification network model as claimed in claim 1, It is characterized in that The step of training the initial classification model and calculating the cross entropy classification loss function value of the initial classification model includes: Processing the training data based on the classification initial model, generating a type information vector at the feature extraction classification layer of the classification initial model; The type information vector is transmitted to the fully connected layer of the classification initial model, and the fully connected layer processes the type information vector to obtain the training result of the classification initial model; The cross entropy classification loss function of the initial classification model is calculated based on the training result to obtain the cross entropy classification loss function value.
4. A text classification method based on a classification network model as claimed in any one of claims 1 to 3, It is characterized in that The step of preprocessing the multi-source information texts includes: A public data set is obtained, and public data in the public data set is transmitted to the initial classification model, and the initial classification model is pre-trained to update the initial classification model.
5. The text classification method based on the classification network model as claimed in claim 1, It is characterized in that The steps of generating labeled training data and transmitting it to the classification initial model, training the classification initial model, and obtaining the classification network model include: After generating labeled training data, the training data is transmitted to the classification initial model, the classification initial model is trained, and the number of training times of the classification initial model is counted; Determine whether the number of training times matches a preset number of iterations. If so, complete the training of the initial classification model, and generate the initial classification model into a classification network model based on the target model parameters obtained during the training. If the number of training times does not match the preset number of iterations, the model parameters of the initial classification model are optimized based on a preset optimization mechanism, and based on the optimized model parameters, the step of training the initial classification model is performed.
6. A text classification device based on a classification network model, It is characterized in that The text classification device based on the classification network model includes: a conversion module, used for acquiring sample files of various types, and converting each of the sample files into multi-source information texts, wherein the types of the sample files include doc, docx, ppt, pptx, xls, xlsx, png, jpg, tiff, dwg, and pdf, and the multi-source information texts are text files containing readable texts, and the multi-source information texts at least include body texts, suffix texts, title texts, and other information texts; A calling module is used to call the classification initial model constructed based on the multi-source information fusion and feature extraction classification network, and pre-process the multi-source information text to generate labeled training data and transmit it to the classification initial model, train the classification initial model, and obtain a classification network model; A determination module, used for classifying and identifying the text to be classified based on the classification network model, and determining the text category to which the text to be classified belongs; Wherein, the preprocessing includes word segmentation and cleaning, and the calling module is further used to: perform word segmentation processing on each of the multi-source information texts according to a preset word segmentation rule to obtain a word segmentation group corresponding to each of the multi-source information texts, and filter and clean the words in each of the word segmentation groups to obtain a selected word segmentation group; encode each of the selected word segmentation groups based on a preset length to generate a word embedding vector corresponding to each of the multi-source information texts; construct a category embedding vector corresponding to a preset number of categories, and after setting the dimension of the category embedding vector to the dimension of the word embedding vector, generate the word embedding vector and the category embedding vector as labeled training data and transmit them to the classification initial model; The calling module is also used to: after generating labeled training data, transfer the training data to the initial classification model, train the initial classification model, and calculate the cross-entropy classification loss function value of the initial classification model; according to the cross-entropy classification loss function value, determine whether the initial classification model reaches the convergence condition, if the convergence condition is reached, complete the training of the initial classification model, and generate the initial classification model into a classification network model based on the target model parameters obtained through training.
7. A text classification terminal based on a classification network model, It is characterized in that The text classification terminal based on the classification network model includes a memory, a processor, and a control program stored in the memory and executable on the processor. When the control program is executed by the processor, the steps of the text classification method based on the classification network model as described in any one of claims 1 to 5 are implemented.
8. A readable storage medium, It is characterized in that The readable storage medium stores a control program, and when the control program is executed by the processor, the steps of the text classification method based on the classification network model as described in any one of claims 1 to 5 are implemented.
