Document classification method and device, electronic equipment and medium
Through the dual-channel processing of pre-trained language model and interactive attention network, combined with the template processing of large language model, the problems of high cost and low accuracy of document classification methods in the existing technology are solved, and efficient and accurate document automation is achieved Classification.
Patent Information
- Application Number
- CN202411174870.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-26
- Publication Date
- 2024-11-29
AI Technical Summary
The document classification method in the prior art relies on manual prediction or simple linear regression analysis, which is cost-effective and inefficient, and cannot effectively handle complex nonlinear relationships and has low accuracy.
Pre-trained language model and interactive attention network are used to convert text features into word vectors, and templated through large language models, and document categories are obtained in combination with the largest voting mechanism to realize automatic document classification.
It improves the efficiency and accuracy of document classification, overcomes the high labor cost and the limitations of traditional linear regression prediction methods in nonlinear relationship processing and feature capture.
Smart Images

Figure CN119046464A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of artificial intelligence technology, and in particular to a document classification method, device, electronic device and medium. Background Art
[0002] Service document classification is a fine-grained classification of mobile customer service and user daily service benchmark documents and management regulations. Efficient service document classification has become a hot issue for communication operators. It is the basis for building intelligent customer service, but it is also a challenging task.
[0003] At present, document classification methods in related technologies are usually manual prediction or simple linear regression analysis based on factors such as author reputation, publisher popularity, document type, etc. However, manual prediction methods are costly and inefficient; simple linear regression analysis cannot take into account complex nonlinear relationships and has low accuracy. Summary of the invention
[0004] The present disclosure provides a document classification method, device, electronic device and medium to solve the problems in the related art.
[0005] The first aspect of the present disclosure proposes a document classification method, which includes: using a pre-trained language model to convert each word in each text feature of a normalized document after normalization processing into a word vector; using an interactive attention network to use the word vector as input to obtain a first document category of the normalized document; using a large language model to template the normalized document according to a preset number of preset prompt templates to obtain a preset number of prompt templates after template processing, and using the preset number of prompt templates to obtain a preset number of second document categories; using the document category that is the same as the first document category and the preset number of second document categories and has the largest number as the target document category of the normalized document.
[0006] In some embodiments of the present disclosure, using an interactive attention network and taking word vectors as input, obtaining a first document category of a normalized document includes: determining the overall feature of each text feature using a long short-term memory network in the interactive attention network and each word vector of each text feature; determining the interactive feature of each text feature using the attention mechanism in the interactive attention network, the overall feature of each text feature, and the overall feature of a preset interactive feature of each text feature; splicing the interactive features of each text feature to obtain a spliced feature; and inputting the spliced feature into an activation function in the interactive attention network to obtain a first document category corresponding to the normalized document.
[0007] In some embodiments of the present disclosure, the overall characteristics of each text feature are determined by utilizing the long short-term memory network in the interactive attention network and each word vector of each text feature, including: inputting each word vector in each text feature into the long short-term memory network corresponding to each text feature in time sequence to obtain the hidden state of each text feature at each time; determining the average value of the hidden state of each text feature based on the hidden state of each text feature at each time, and taking the average value of the hidden state of each text feature as the overall characteristic of each text feature.
[0008] In some embodiments of the present disclosure, a large language model is used to perform template processing on normalized documents according to a preset number of preset prompt templates to obtain a preset number of prompt templates after template processing, and a preset number of second document categories are obtained by using the preset number of prompt templates, including: using the large language model to fill the normalized documents with content according to the language structure of a preset number of preset prompt templates to obtain prompt templates corresponding to a preset number of normalized documents, and encapsulating the prompt templates corresponding to the preset number of normalized documents into a prompt list; using the large language model and the prompt list to obtain a preset number of second document categories.
[0009] In some embodiments of the present disclosure, using a large language model and a prompt list to obtain a preset number of second document categories includes: inputting the prompt list into a service website of a registered account in the large language model to obtain a preset number of second document categories.
[0010] In some embodiments of the present disclosure, using a large language model and a prompt list to obtain a preset number of second document categories includes: generating an interface key according to an interface key interface corresponding to a service interface of the large language model; using a package management tool in the large language model to install a software package that interacts with the service interface, and adding an interface key to the software package; using the software package and the prompt list to which the interface key is added to generate a preset number of second document categories, and calling text completion information to print the prompt list and the preset number of second document categories.
[0011] A second aspect of the present disclosure provides a document classification device, the device comprising:
[0012] A vector conversion unit, used for converting each word in each text feature of the normalized document into a word vector by using a pre-trained language model;
[0013] A first classification unit, configured to use an interactive attention network and take the word vector as input to obtain a first document category of a normalized document;
[0014] A second classification unit is used to use the large language model to perform template processing on the normalized document according to a preset number of preset prompt templates to obtain a preset number of prompt templates after template processing, and obtain a preset number of second document categories using the preset number of prompt templates;
[0015] The prediction unit is used to use the first document category and the document category with the same document category and the largest number among the preset number of second document categories as the target document category of the normalized document.
[0016] The third aspect embodiment of the present disclosure proposes an electronic device, comprising: a processor and a memory for storing a computer program that can be run on the processor, wherein the processor, when used to run the computer program, executes the method described in the first aspect embodiment of the present disclosure.
[0017] The fourth aspect embodiment of the present disclosure proposes a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to execute the method described in the first aspect embodiment of the present disclosure.
[0018] The fifth aspect embodiment of the present disclosure provides a computer program product, including a computer program, which implements the method described in the first aspect embodiment of the present disclosure when executed by a processor.
[0019] In summary, according to the document classification method proposed in the present disclosure, by utilizing a pre-trained language model, each word in each text feature of the normalized document after normalization is converted into a word vector; utilizing an interactive attention network, taking the word vector as input, a first document category of the normalized document is obtained; utilizing a large language model, the normalized document is templated according to a preset number of preset prompt templates to obtain a preset number of prompt templates after template processing, and utilizing the preset number of prompt templates to obtain a preset number of second document categories; the document category with the same document category and the largest number in the first document category and the preset number of second document categories is used as the target document category of the normalized document, automatic document classification is realized, the first document category and the second document category are obtained through dual-channel local prediction, and the maximum voting mechanism is adopted to obtain the final target document category, which effectively improves the efficiency and accuracy of document classification, and overcomes the high labor cost and the limitations of traditional linear regression prediction methods in nonlinear relationship processing and feature capture.
[0020] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS
[0021] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations on the present disclosure.
[0022] Figure 1 A flowchart of a document classification method provided by an embodiment of the present disclosure;
[0023] Figure 2 A flowchart of a document classification method provided by an embodiment of the present disclosure;
[0024] Figure 3 A flowchart of a document classification method provided by an embodiment of the present disclosure;
[0025] Figure 4 A general architecture diagram of a specific document classification provided by an embodiment of the present disclosure;
[0026] Figure 5 A schematic diagram of the structure of a document classification device provided in an embodiment of the present disclosure;
[0027] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION
[0028] The embodiments of the present disclosure are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions.
[0029] Service document classification is a fine-grained classification of mobile customer service and user daily service benchmark documents and management regulations. Efficient service document classification and classification has become a hot issue for communication operators. It is the basis for building intelligent customer service, but also a challenging task.
[0030] At present, the document classification methods of related technologies are usually manual prediction or simple linear regression analysis based on factors such as the reputation of the author, the popularity of the publisher, the type of document, etc.
[0031] The service document classification method based on manual prediction in related technologies requires customer service practitioners to have deep professional knowledge and experience in knowledge base sorting in order to accurately predict different knowledge documents. However, the accumulation of professional experience takes a long time and is difficult to quantify and standardize. This method has poor scalability and repeatability, and high labor costs. Manual prediction methods require a lot of manpower, and organizations with limited resources cannot afford the high costs. Although the simple linear regression method solves the problems of professional limitations and labor costs in manual prediction, it lacks consideration of complex nonlinear relationships and massive data characteristics, and the prediction accuracy is not high.
[0032] In order to solve the above problems, the present disclosure aims to utilize large language model technology to propose a document automatic classification method based on large language model and deep interactive attention network. The method obtains respective local prediction results, i.e., the first document category and the second document category, through the dual channels of large language model and interactive attention network, and adopts the maximum voting mechanism to merge the local prediction results to obtain the target document category for automatic document classification, thereby avoiding the labor cost problem caused by manual prediction and the lack of consideration of complex nonlinear relationships and massive data features caused by linear regression prediction, thereby improving the efficiency and accuracy of document classification prediction.
[0033] The document classification method provided in this application is described in detail below with reference to the accompanying drawings.
[0034] Figure 1 A flowchart of a document classification method provided by an embodiment of the present disclosure. Figure 1 As shown, the document classification method includes steps 101-104.
[0035] Step 101, using a pre-trained language model, convert each word in each text feature of the normalized document into a word vector.
[0036] In the embodiments of the present disclosure, the normalized document of the present disclosure refers to the knowledge document in the data set that has been normalized, mainly including the title, keywords, introduction and author information of the knowledge. Normalization is the process of ensuring the consistency of the data format of the input knowledge document and reducing noise and outliers.
[0037] The present disclosure adopts a dual-channel design scheme, processing the normalized document through a large language model channel and an interactive attention network channel respectively. Before the normalized document is processed through the interactive attention network, each text feature in the normalized document needs to be vectorized using a pre-trained language model. Text features refer to the title, author, keywords, content introduction, etc. in the document.
[0038] The pre-trained language model can specifically be a model such as BERT, RoBERTa, etc. This disclosure takes the BERT model as the pre-trained language model and the title as the text feature as an example. The specific process of vectorizing each text feature in the normalized document is as follows:
[0039] Assume that the title of the document has n words, represented by T = {t 1 ,t 2 ,...,t n}. Given that the pre-trained language model BERT has the advantages of efficient feature extraction capability and no need for word segmentation when modeling Chinese, this disclosure first uses the pre-trained language model BERT to represent each word in the title as a word vector (also known as word embedding), formalized as:
[0040]
[0041] is the word vector of the jth word in the title. Among them, the keywords, content introduction and author information of the document are similar and are also vectorized by the pre-trained language model BERT, that is, and They are respectively used as the input of the interactive attention network channel.
[0042] Step 102, using the interactive attention network, taking the word vector as input, and obtaining the first document category of the normalized document.
[0043] In an embodiment of the present disclosure, considering that the title, author, keywords and content introduction of knowledge are interrelated and intertwined, and jointly affect the recommendation degree of knowledge documents, the present disclosure adopts an interactive attention network based on long short-term memory network and attention mechanism.
[0044] The interactive attention network disclosed in the present invention uses the BERT representation of the input layer, that is, the word vector obtained in step 101 as input, and adopts a long short-term memory network to model the context information in each text feature (such as title, author, keywords and content introduction), and then performs feature splicing after fusing the hidden state of each text feature through the attention mechanism to obtain a comprehensive feature representation of the normalized document, that is, the spliced feature, and then uses the activation function in the interactive attention network to obtain the final first document category.
[0045] Among them, in the document classification task, the document categories can include three categories: key recommendation, general recommendation, and outdated content not recommended. The first document category can be any one of the three categories of key recommendation, general recommendation, and outdated content not recommended, which is mainly based on actual conditions and is not limited in the embodiments of the present disclosure.
[0046] Step 103, using the large language model, the normalized document is templated according to a preset number of preset prompt templates to obtain a preset number of prompt templates after template processing, and using the preset number of prompt templates to obtain a preset number of second document categories.
[0047] In the embodiment of the present disclosure, the large language model has excellent human language understanding and generation capabilities. Since the large language model can only accept text input at present, the normalized document does not need to be vectorized, but is directly input into the large language model.
[0048] Among them, in the prompt project, the preset prompt template is used to formalize the sentences in the standardized document into a natural language template, and convert the document classification task into a fill-in-the-blank generation task. The process of generating the preset prompt template disclosed in the present invention involves defining the template structure, determining the content to be filled, and designing the prompt guidance, so as to finally form a text template containing a complete structure and prompts, that is, a preset prompt template.
[0049] Then, through the understanding and generation capabilities of the large language model, combined with a preset number of preset prompt templates, a preset number of second document categories corresponding to the normalized document are determined.
[0050] It should be noted that the preset number of preset prompt templates is set according to actual needs and is not limited in the embodiments of the present disclosure. The present disclosure takes the preset number of 3 as an example. The second document category can be any one of the three categories of key recommendation, general recommendation, and non-recommendation of outdated content, which is based on actual conditions and is not limited in the embodiments of the present disclosure.
[0051] Step 104: The first document category and the document category with the largest number of the same document category among the preset number of second document categories are used as the target document category for the standardized document.
[0052] In an embodiment of the present disclosure, the present disclosure may determine the target document category of the normalized document through a maximum voting mechanism (also referred to as a majority voting mechanism).
[0053] The maximum voting mechanism is a simple ensemble learning technique in which the decision result is determined by the candidate option that obtains the most votes, allowing the predictions of multiple basic learners to be combined. The present disclosure can obtain a preset number of second document categories through the large language model channel, and can obtain 1 first document category by calling the interactive attention network channel. Assuming that the answers are concentrated in three categories of {key recommendation, general recommendation, non-important content not recommended}, the one with the largest number of corresponding numbers is identified as the final target document category.
[0054] In summary, according to the document classification method proposed in the present disclosure, by utilizing a pre-trained language model, each word in each text feature of the normalized document after normalization is converted into a word vector; utilizing an interactive attention network, taking the word vector as input, a first document category of the normalized document is obtained; utilizing a large language model, the normalized document is templated according to a preset number of preset prompt templates to obtain a preset number of prompt templates after template processing, and utilizing the preset number of prompt templates to obtain a preset number of second document categories; the document category with the same document category and the largest number in the first document category and the preset number of second document categories is used as the target document category of the normalized document, automatic document classification is realized, the first document category and the second document category are obtained through dual-channel local prediction, and the maximum voting mechanism is adopted to obtain the final target document category, which effectively improves the efficiency and accuracy of document classification, and overcomes the high labor cost and the limitations of traditional linear regression prediction methods in nonlinear relationship processing and feature capture.
[0055] As a possible implementation, Figure 2 The flowchart of a document classification method shown in the figure, based on the above embodiment, uses an interactive attention network, takes a word vector as input, and obtains a specific process of the first document category of a normalized document, including the following steps:
[0056] Step 201, using the long short-term memory network in the interactive attention network and each word vector of each text feature, determine the overall feature of each text feature.
[0057] In an embodiment of the present disclosure, each word vector in each text feature is input into the long short-term memory network corresponding to each text feature in sequence according to the time sequence to obtain the hidden state of each text feature at each time; based on the hidden state of each text feature at each time, the average value of the hidden state of each text feature is determined, and the average value of the hidden state of each text feature is used as the overall feature of each text feature.
[0058] Specifically, each word vector of each text feature, namely the title word vector sequence T, keyword vector sequence G, introduction word vector sequence J and author word vector sequence Z, is specifically As the input from time 1 to n, it is input into 4 independent LSTM units in sequence. The title, keywords, introduction and author hidden state at each moment are obtained through the forget gate, input gate, output gate and memory unit:
[0059]
[0060] The average value will be calculated based on the output state of the four LSTM networks at each moment. The average value is calculated using the avg function to obtain the overall characteristics of the title, keywords, introduction and author information, namely:
[0061]
[0062] Step 202, using the attention mechanism in the interactive attention network, the overall features of each text feature, and the overall features of the preset interactive features of each text feature, to determine the interactive features of each text feature.
[0063] In an embodiment of the present disclosure, in order to model the interactions and associations between various field contents, the present disclosure utilizes an attention mechanism to fuse the overall features of each text feature with the overall features of the preset interaction features of each text feature.
[0064] Specifically, the present disclosure combines the overall characteristics of the title, keywords, introduction and author information Selectively merge with the hidden state of other content. Taking the title as an example, the present disclosure uses keywords, introduction, and author information as the preset interactive features of the title, and obtains the interactive features of the title through the attention mechanism. In other words, the present disclosure uses the overall features of keywords, introduction, and author information as the preset interactive features of the title. As a query, hide the title content at each moment As the key and value, the interactive features of the title are obtained through the attention mechanism:
[0065]
[0066] Among them, Q∈{G,J,Z}, k∈[1,2,...,n], h T After attention fusion, the interactive features of the title are obtained. Similarly, the overall features of the title, introduction and author information are combined As a query, hide the keyword content at every moment As the key and value, the interactive features of keywords are obtained through the attention mechanism, denoted as h G Then the interactive features of the introduction and the author can be obtained through the above formula, denoted as h J ,h Z .
[0067] Step 203, concatenate the interactive features of each text feature to obtain a concatenated feature.
[0068] In an embodiment of the present disclosure, the present disclosure splices the interactive features of each text feature obtained above to obtain a representation B of the normalized document, that is, a spliced feature of the normalized document.
[0069] Step 204: input the concatenated features into the activation function in the interactive attention network to obtain the first document category corresponding to the normalized document.
[0070] In an embodiment of the present disclosure, the concatenated feature B is input into the activation function (softmax function) in the interactive attention network to obtain the predicted first document category:
[0071] B=[h T ;h G ;h J ;h Z ]
[0072]
[0073] Among them, c represents the number of predicted document categories, yc represents the vector representation of the first document category, b is the correlation coefficient, and W is the preset weight of the softmax function, which has three categories: {key recommendation, general recommendation, and non-important content not recommended}. The one with the highest value is the prediction result, that is, the first document category.
[0074] In summary, this paper introduces the LSTM network and attention mechanism, combined with the BERT pre-trained language model, and innovatively constructs an interactive attention network, which aims to accurately capture the internal connections and characteristics of knowledge documents by deeply integrating multi-dimensional information such as the title, author, keywords and content summary of the knowledge document, thereby efficiently generating the first document category and significantly improving the accuracy of document classification.
[0075] As a possible implementation, Figure 3 The flowchart of a document classification method shown in the figure, based on the above embodiment, uses a large language model to perform template processing on the normalized document according to a preset number of preset prompt templates, obtains a preset number of prompt templates after template processing, and uses the preset number of prompt templates to obtain a preset number of second document categories, including the following steps:
[0076] Step 301, using a large language model to fill the normalized document with content according to the language structure of a preset number of preset prompt templates, obtain prompt templates corresponding to a preset number of normalized documents, and encapsulate the prompt templates corresponding to the preset number of normalized documents into a prompt list.
[0077] In an embodiment of the present disclosure, the model structure of the large language model of the present disclosure is based on the Transformer model, which includes multiple Encoder and Decoder layers. Each Encoder and Decoder layer includes multiple self-attention mechanisms and fully connected neural networks, and the depth of the model is achieved through multi-layer stacking. The generation method is based on the idea of conditional generation, that is, given some input text, predicting and generating the next sentence. The generation process is a process of encoding the input text and then decoding to generate the next sentence. During the generation process, the model takes into account the context of the previous text and predicts the content of the next sentence based on information such as the frequency and position of words and phrases appearing in the previous text. In the present disclosure, the large language model takes ChatGLM4 as an example.
[0078] The large language model channel of this disclosure contains three core modules, namely, prompt template construction, ChatGLM4 API call and document classification module. Specifically, firstly, according to the i-th standardized document (namely, title, keywords, introduction and author information), three preset prompt templates are manually designed, denoted as P = {P 1 ,P 2 ,P 3 Then, the preset prompt template is used to generate three prompt templates corresponding to the normalized document, and the generated three prompt templates are used as the input of ChatGLM4 API respectively. Calling ChatGLM4 interface will generate three prediction results, that is, three second document lists, as the output result R={R 1 ,R 2 ,R 3}.
[0079] For prompt template construction, in the prompt project, the input sentence is formalized into a natural language template, and the document classification task is converted into a fill-in-the-blank generation task. The template is the background description of the service knowledge document classification task, and ChatGLM4 will generate the corresponding answer based on the current context.
[0080] The process of generating a prompt template can be divided into the following four steps:
[0081] (1) Define the language structure of the preset prompt template of the document, such as which elements should be included in the template, the relationship between the elements, etc. The present disclosure will use the hard prompt template structure to create a cloze question.
[0082] (2) Determine the content to be filled in the preset prompt template according to the language result of the preset prompt template, which may include keywords, entities, tags, etc. This disclosure mainly focuses on document classification tasks. The preset prompt template content includes elements such as text, prompt words and [MASK], among which the knowledge content will be spliced with the text information as auxiliary and supplementary information. However, in order to construct different types of templates, the positions of images and texts may change.
[0083] (3) Design prompts for filling in content. For each content that needs to be filled in, a corresponding prompt needs to be designed to guide the model to generate appropriate label words. This disclosure focuses on document classification tasks. The prompt is to guide the large language model to judge sentences related to knowledge categories, such as "Therefore, the first-level category of this knowledge is [MASK]".
[0084] (4) According to the above steps, the final prompt template is generated, that is, the complete text containing the structure and filler content prompts. This prompt template will be used as the input of the pre-trained language model.
[0085] This disclosure designs three typical preset prompt templates based on different knowledge base information, as described below:
[0086] P 1 :{Given a piece of knowledge, its title: [Title T i ], Author: [Author Information Z i ]. Based on your own basic ability of large language model, please predict which category this book belongs to: key recommendation, general recommendation, outdated content not recommended?}
[0087] P 2 :{Given a piece of knowledge, its title: [Title T i ], Keywords: [Keyword G i ], Author: [Author Information Z i ]. Based on your own large language model basic capabilities, please predict which of the following categories this knowledge belongs to: key recommendation, general recommendation, outdated content not recommended?}
[0088] P 3 :{Given a piece of knowledge, its title: [Title T i ], Keywords: [Keyword G i ], the first paragraph of the document is: [IntroductionJ i ], issuing unit information: [Unit information Z i Based on your own large language model basic capabilities, please predict whether this knowledge is most likely to be: key recommendation, general recommendation, or outdated content not recommended?}
[0089] Step 302: Obtain a preset number of second document categories using the large language model and the prompt list.
[0090] In the embodiment of the present disclosure, using the large prediction model and the prompt list, obtaining a preset number of second document categories includes two methods: inputting the prompt list to the service website of the registered account in the large language model to obtain a preset number of second document categories. Alternatively, according to the interface key interface corresponding to the service interface of the large language model, an interface key is generated; using the package management tool in the large language model, a software package that interacts with the service interface is installed, and an interface key is added to the software package; using the software package and the prompt list after adding the interface key, a preset number of second document categories is generated, and the text completion information is called to print the prompt list and the preset number of second document categories.
[0091] Specifically, the present disclosure provides two ways to use ChatGLM4. The first is to register an account and enter the ChatGLM4 experience website, then directly enter the user prompt template in the input box. ChatGLM4 will quickly generate answers based on the user prompt input and display them on the interface; the second is to jump to the API key interface (interface key interface) according to the interface (service interface) published by OpenAI, click Create new secret key to generate API Keys (interface key), and then use the Pip tool (package management tool) to install the Python package (software package) of the OpenAI API, add the API key to openai.api_key, and get a preset number of second document categories. Secondly, call completion.choices[0].message (text completion information) to print out the user's own prompt input (i.e., prompt list) and the answer generated by ChatGLM4 (i.e., a preset number of second document lists). In order to input three prompt templates at one time, the present disclosure can encapsulate these three prompt templates into a prompts list (prompt list), then use a loop to iterate each prompt template, and call the generate_text function to generate text, so that three prompt templates can be input at one time, the corresponding answers can be obtained, and each prompt template and the corresponding generated text can be printed on the console. In this way, the ChatGLM4API call can be completed, and the second document category of the document classification prediction corresponding to the three prompt templates can be obtained.
[0092] The present disclosure directly generates three service document classification results according to three prompt templates by calling ChatGLM4 API, and uses them as the output result R={R 1 ,R 2 ,R 3}. From then on, the large language model channel output ends.
[0093] Among them, the specific process of the large language model classifying according to the prompt template can be summarized as the following steps:
[0094] Adding prompt information: Build a prompt template for the document category, and match the original text such as service management regulations with the additionally added prompt information.
[0095] The specific templates are as follows:
[0096] P 1 :{Given a piece of knowledge, its title: [Title T i ], Author: [Author Information Z i ]. Based on your own basic ability of large language model, please predict which category this book belongs to: key recommendation, general recommendation, outdated content not recommended?}
[0097] P 2 :{Given a piece of knowledge, its title: [Title T i ], Keywords: [Keyword G i ], Author: [Author Information Z i ]. Based on your own large language model basic capabilities, please predict which of the following categories this knowledge belongs to: key recommendation, general recommendation, outdated content not recommended?}
[0098] P 3 :{Given a piece of knowledge, its title: [Title T i ], Keywords: [Keyword G i ], the first paragraph of the document is: [IntroductionJ i ], issuing unit information: [Unit information Z i Based on your own large language model basic capabilities, please predict whether this knowledge is most likely to be: key recommendation, general recommendation, or outdated content not recommended?}
[0099] (2) Prompt input: Encapsulate the determined prompt template into a prompt list and input it into the Large Language Model API interface. ChatGLM4 API has learned the probability distribution of language on large-scale text data and can generate text related to the input.
[0100] (3) Answer search: The large language model ChatGLM4 fills in the information to be filled in the template in a probabilistic manner based on the input prompt template and original text, and predicts the recommended answer of the corresponding document.
[0101] (4) Result output: Finally, the large language model will output the classification results and print them in the log of the large model service. The result is that three second document categories corresponding to the three prompt templates are obtained.
[0102] In summary, the document classification method disclosed in the present invention, by utilizing a large language model, fills the content of the normalized document according to a preset template structure, generates a corresponding number of prompt templates, and integrates them into a prompt list. Subsequently, based on these prompt lists, the large language model further generates a preset number of second document categories. The whole process combines the idea of prompt-based learning, and guides the model to perform small sample or even zero sample learning by defining specific prompt templates to adapt to new classification tasks. The model is allowed to use the knowledge learned on large-scale text data to adapt to new scenarios and tasks through a small amount of annotated data or no annotated data, thereby achieving efficient and structured document classification.
[0103] based on Figures 1 to 3 The embodiment shown, as Figure 4 The present disclosure provides a general architecture diagram of a specific document classification.
[0104] In the embodiments of the present disclosure, referring to Figure 4 The classification model LIAN that integrates the large language model and the interactive attention network is mainly divided into three parts: the input layer, the middle layer (including the large language model channel and the interactive attention network channel) and the output layer.
[0105] The i-th normalized document D in the dataset = {T i ,G i ,J i ,Z i}, where T i Indicates the title, G i Indicates keywords, J i Indicates the introduction content, Z i Indicates author information. There are three categories of books: "Highly Recommended", "Generally Recommended" and "Outdated Content Not Recommended".
[0106] The present invention uses a document classification model that integrates the large language model ChatGLM4 and the interactive attention network, referred to as LIAN, to normalize the document D = {T i ,G i ,J i ,Z i} respectively through the large language model ChatGLM4 channel and the interactive attention network channel to obtain their respective prediction results P L (Second Document Category) and P A (First document category), and then the document classification is performed through the maximum voting mechanism, which can be formalized as: [key recommendation, general recommendation, outdated content is not recommended] = LIAN (D).
[0107] A large language model refers to a pre-trained language model based on a deep neural network, usually using a Transformer model, which can better handle long sequences and long-distance dependencies, has strong language understanding and generation capabilities, and has been widely used in tasks such as text generation and document classification. In addition, given that the title, keywords, content summary and author information of the knowledge base are interrelated and jointly determine the importance of knowledge content, this paper proposes a deep interactive attention network based on the long short-term memory network LSTM to model the semantic dependencies between different field information and obtain a more accurate representation of book features. Therefore, the present disclosure will integrate the large language model with the interactive attention network to achieve more accurate document classification.
[0108] Among them, this disclosure adopts cross entropy as the loss function of the model, which is defined as:
[0109]
[0110] in, represents the true category, Represents the predicted category, i represents the number of book documents, c represents the number of categories, and λ is the regularization coefficient. The present disclosure uses a back propagation algorithm to update the gradient and network parameters. In order to avoid overfitting in training, the present disclosure adopts a random pruning strategy.
[0111] The specific process includes:
[0112] (1) Calculate the loss
[0113] The cross entropy loss function is used to calculate the loss between the model's output and the true label. For the service document classification problem, cross entropy loss is a commonly used loss function.
[0114] (2) Back Propagation
[0115] The gradient is calculated based on the loss function, and the error is passed layer by layer from the output layer to the input layer through the chain rule. In LSTM, the gradient of the hidden state, the cell state, and each gate (input gate, forget gate, output gate) is calculated.
[0116] (3) Parameter update
[0117] Use an optimization algorithm (Adam is used in this article) to update the weights and biases of the model according to the calculated gradient. By continuously iterating and optimizing the parameters, the loss of the model on the training set is gradually reduced.
[0118] (4) Random pruning
[0119] During the model training process, a random pruning strategy is introduced to prevent overfitting. Pruning in model training is achieved by randomly setting a part of the connections (or neurons) to zero during the training process.
[0120] Pruning is performed based on the weights of connections or neurons. Connections or neurons with smaller weights are more likely to be pruned.
[0121] Use some pruning rate, such as pruning 1% of connections or neurons every training cycle.
[0122] When continuing to train the pruned model, the way to calculate gradients and update parameters needs to be adjusted accordingly.
[0123] (5) Iterative training, evaluation, and verification
[0124] Evaluate the performance of the model on the validation set to monitor the model training process. If the performance of the model on the validation set begins to decline (i.e., overfitting occurs), consider strategies such as stopping training early, reducing the learning rate, and increasing the regularization term.
[0125] Repeat the iterative optimization of the model until the performance of the model on the validation set reaches a satisfactory level or reaches the preset number of training rounds. The model is then saved in the Internet-based help desk knowledge base system.
[0126] In summary, the present invention uses large-scale service knowledge data to build a document classification model that integrates a large language model and a deep interactive attention network, so as to improve the prediction accuracy of the model and achieve accurate prediction of the service document type. The large language model and the interactive attention network in the document classification model are used to obtain their respective local predictions, and the maximum voting mechanism is used to merge the local predictions to obtain the service document classification results, so as to achieve automatic prediction and consider multiple relationships between data, and improve the prediction accuracy and efficiency.
[0127] The data used for the empirical test in this disclosure comes from the service document information resource data in a mobile customer service knowledge base. Through the "Recommended Ranking" page on the homepage of the knowledge base, according to the top ten rankings of 20 first-level service knowledge management categories from 2023 to 2024, a total of 20,100 knowledge documents are recommended as key recommendations. Through the "Knowledge Browsing"-"Keyword Browsing" page, search and browse according to 20 first-level categories, select service documents with less than or equal to 50 clicks, and regard them as outdated documents, totaling 29,800. From the remaining book samples, 50,000 service documents are randomly selected as general recommendations. Among them, each piece of knowledge will collect field information such as its title, keywords, content summary (introduction), author, etc. The entire experimental data set contains a total of 99,900 service documents. The data set is divided into training set, validation set and test set in a ratio of 6:2:2. Table 1 shows the statistical characteristics of the data set samples.
[0128] Table 1 Dataset sample statistics
[0129] category Training set Validation set Test Set Total number of samples Recommended 12060 4020 4020 20100 General Recommendations 30000 10000 10000 50000 Outdated Documentation 17880 5960 5960 29800 total 59440 19980 19980 99900
[0130] Experimental setup:
[0131] (1) Evaluation indicators. In view of the need to uniformly measure the performance of each method, this paper will calculate the precision (P), recall (R), and F1 value (the larger the value, the better the prediction effect) as the basis for analysis. At the same time, the accuracy (Acc) and F1 will also be used for subsequent analysis such as model ablation experiments.
[0132] (2) Hyperparameter setting. This paper uses PyTorch to build an interactive attention network model, and the hardware environment is a GeForce RTX 3080Ti GPU. The book word vector selection is initialized by BERT representation, and the word vector dimension is 768. The pruning rate is set to 0.5, the batch size is set to 64, the learning rate is set to 0.001, the regularization coefficient is 0.001, and the number of iterations is 100 rounds.
[0133] Controlled trial setup:
[0134] The present invention uses cutting-edge machine learning methods to achieve automatic classification of service documents in order to compare with the fusion of large language models and interactive attention networks.
[0135] (1) Machine learning: Support vector machine (SVM) and random forest (RF) algorithms were selected as two comparison targets. The classification process is: use GloVe word embedding to represent the title, keywords, introduction, and author vectors, concatenate them and input them into the machine learning classifier. After training and verification, the prediction results are output.
[0136] (2) Deep learning: Convolutional neural network CNN, recurrent neural network RNN, long short-term memory network LSTM and LSTM model based on attention mechanism (abbreviated as Attention-LSTM), convolutional-recurrent neural network RCNN, and graph convolutional neural network GCN are cutting-edge deep learning algorithms that will be compared. Specifically, the GloVe word embedding technology is used to extract title, keywords, introduction and author features, and the four are spliced into knowledge features and input into the deep neural network. After training convergence, the book category label is obtained through the softmax function. In addition, this study also uses the "BERT-Base, Chinese" pre-trained language model with 110 million parameters to extract and classify the key knowledge information in the knowledge base.
[0137] To ensure the comparability of the experiments, the parameter settings of the deep learning control experiments are consistent with the model constructed in this article, which will not be repeated here.
[0138] Experimental results analysis:
[0139] Table 2 Control experiment results
[0140]
[0141] The results of the control experiment are shown in Table 2. For the service document classification task, it can be observed that the SVM algorithm has the worst experimental results, with an F1 result of only 74.3%. The random forest algorithm achieved better results than the SVM, with an F1 score of 75.7%. As the two most widely used machine learning algorithms, random forest is more suitable for large-scale classification tasks, while SVM performs poorly in large-scale data due to the relatively complex Gaussian kernel transformation. In addition, the two machine learning algorithms are significantly weaker than other deep learning algorithms, such as convolutional neural networks and recurrent neural networks. This shows that the performance of machine learning algorithms is not as good as that of deep learning algorithms. The reason is that deep learning can extract more effective features with its deep abstract architecture. In addition, large-scale data is more suitable for deep learning algorithms, so that they can learn the optimal parameters through sufficient training. However, machine learning algorithms have small parameters, and large-scale data affects their running time. Therefore, when facing large-scale data processing tasks, model selection should be more cautious.
[0142] In deep learning algorithms, convolutional neural networks and recurrent neural networks have achieved the same level of experimental results. Combining the architectural advantages of the two, convolutional recurrent neural networks (CRNNs) have achieved significant performance improvements. The long short-term memory network (LSTM) has achieved good improvements in various indicators because it can model long-term word order information, breaking through 80% for the first time. The LSTM network based on the attention mechanism has surpassed the standard long short-term memory network because it can weigh the importance of each word. The graph convolutional neural network can regard words as vertices and the dependencies between words as edges to construct a graph network. Given its ability to aggregate information from neighboring words, GCN has also achieved good results, better than convolutional neural networks and recurrent neural networks, but weaker than Attention-LSTM. The main reason is that the attention mechanism can selectively select important information. With the popularity of large-scale language models, the pre-trained language model BERT has significantly surpassed mainstream neural networks such as Attention-LSTM, LSTM, GCN, and RCNN in F1 score, reaching 84.8% with the powerful feature extraction capability of the deep encoder. The reason is that the BERT model includes deep bidirectional encoding and self-attention mechanism to capture long-distance dependencies within sentences, and its number of parameters reaches 110 million.
[0143] The prediction model LIAN proposed in this paper, which integrates a large language model and an interactive attention network, significantly surpasses all control methods and achieves the best prediction performance in all indicators, with an F1 score of 88.5%, which is 4.36% higher than the best baseline model BERT. There are two reasons for this: (1) This paper first uses ChatGPT, a large language model containing hundreds of billions of parameters, which already has outstanding basic capabilities; (2) The interactive attention channel can interactively integrate and update the title, keywords, content introduction and author information to obtain more efficient features.
[0144] Corresponding to the methods provided in the above-mentioned embodiments, the present disclosure also provides a document classification device. Since the device provided in the embodiment of the present disclosure corresponds to the methods provided in the above-mentioned embodiments, the implementation method of the method is also applicable to the device provided in this embodiment and will not be described in detail in this embodiment.
[0145] Figure 5 FIG. 5 is a schematic diagram of a document classification device 500 provided in an embodiment of the present disclosure. Figure 5 As shown, the document classification device includes:
[0146] A vector conversion unit 510 is used to convert each word in each text feature of the normalized document after normalization processing into a word vector using a pre-trained language model;
[0147] A first classification unit 520 is used to use an interactive attention network to take the word vector as input to obtain a first document category of the normalized document;
[0148] The second classification unit 530 is used to use the large language model to perform template processing on the normalized document according to a preset number of preset prompt templates to obtain a preset number of prompt templates after template processing, and use the preset number of prompt templates to obtain a preset number of second document categories;
[0149] The prediction unit 540 is configured to use the first document category and the document category with the largest number of the same document category among the preset number of second document categories as the target document category for the normalized document.
[0150] In some embodiments of the present disclosure, the first classification unit 520 is used to: determine the overall features of each text feature using the long short-term memory network in the interactive attention network and each word vector of each text feature; determine the interactive features of each text feature using the attention mechanism in the interactive attention network, the overall features of each text feature and the overall features of the preset interactive features of each text feature; splice the interactive features of each text feature to obtain a spliced feature; input the spliced feature into the activation function in the interactive attention network to obtain a first document category corresponding to the normalized document.
[0151] In some embodiments of the present disclosure, the first classification unit 520 is used to: input each word vector in each text feature into the long short-term memory network corresponding to each text feature in time sequence to obtain the hidden state of each text feature at each time; determine the average value of the hidden state of each text feature according to the hidden state of each text feature at each time, and take the average value of the hidden state of each text feature as the overall feature of each text feature.
[0152] In some embodiments of the present disclosure, the second classification unit 530 is used to: use a large language model to fill the content of the normalized document according to the language structure of a preset number of preset prompt templates, obtain prompt templates corresponding to a preset number of normalized documents, and encapsulate the prompt templates corresponding to the preset number of normalized documents into a prompt list; use the large language model and the prompt list to obtain a preset number of second document categories.
[0153] In some embodiments of the present disclosure, the second classification unit 530 is used to: input the prompt list to the service website of the registered account in the large language model to obtain a preset number of second document categories.
[0154] In some embodiments of the present disclosure, the second classification unit 530 is used to: generate an interface key according to an interface key interface corresponding to the service interface of the large language model; use a package management tool in the large language model to install a software package that interacts with the service interface, and add an interface key to the software package; use the software package after adding the interface key and the prompt list to generate a preset number of second document categories, and call text completion information to print the prompt list and a preset number of second document categories.
[0155] In summary, through the document classification device, by using the pre-trained language model, each word in each text feature of the normalized document after normalization is converted into a word vector; using the interactive attention network, the word vector is used as input to obtain the first document category of the normalized document; using the large language model, the normalized document is templated according to a preset number of preset prompt templates to obtain a preset number of prompt templates after template processing, and using the preset number of prompt templates to obtain a preset number of second document categories; the document category with the same document category and the largest number in the first document category and the preset number of second document categories is used as the target document category of the normalized document, to realize automatic document classification, obtain the first document category and the second document category through dual-channel local prediction, and adopt the maximum voting mechanism to obtain the final target document category, which effectively improves the efficiency and accuracy of document classification, and overcomes the high labor cost and the limitations of traditional linear regression prediction methods in nonlinear relationship processing and feature capture.
[0156] In the embodiments provided in the present application, the methods and devices provided in the embodiments of the present application are introduced. In order to implement the functions in the methods provided in the embodiments of the present application, the electronic device may include a hardware structure and a software module, and implement the functions in the form of a hardware structure, a software module, or a hardware structure plus a software module. A function of the functions may be executed in the form of a hardware structure, a software module, or a hardware structure plus a software module.
[0157] Figure 6 is a block diagram of an electronic device 600 for implementing the document classification method according to an exemplary embodiment. For example, the electronic device 600 may be a mobile phone, a computer, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.
[0158] Reference Figure 6 The electronic device 600 may include a communication interface 601, which can interact with other devices; a processor 602, which is connected to the communication interface 601 to interact with other devices and is used to execute the method provided by one or more of the above technical solutions when running a computer program; and a memory 603, on which the computer program is stored. Specifically, the specific processing process of the processor 602 can refer to the document classification method described in the above embodiment of the present disclosure.
[0159] Of course, in actual application, the various components in the electronic device 600 are coupled together through the bus system 604. It can be understood that the bus system 604 is used to realize the connection and communication between these components. In addition to the data bus, the bus system 604 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, Figure 6 Various buses are labeled as bus system 604 .
[0160] The memory 603 in the embodiment of the present application is used to store various types of data to support the operation of the electronic device 600. Examples of such data include: any computer program used to operate on the electronic device 600.
[0161] The method disclosed in the above embodiment of the present application can be applied to the processor 602, or implemented by the processor 602. The processor 602 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 602. The above processor 602 may be a general processor, a digital signal processor (DSP, Digital Signal Processor), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The processor 602 can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiments of the present application. The general processor may be a microprocessor or any conventional processor, etc. In combination with the steps of the method disclosed in the embodiment of the present application, it can be directly embodied as a hardware decoding processor to execute, or it can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium, which is located in the memory 603, and the processor 602 reads the information in the memory 603 and completes the steps of the above method in combination with its hardware.
[0162] In an exemplary embodiment, the electronic device 600 can be implemented by one or more application specific integrated circuits (ASIC), DSP, programmable logic device (PLD), complex programmable logic device (CPLD), field programmable gate array (FPGA), general processor, controller, microcontroller (MCU), microprocessor, or other electronic components to execute the aforementioned method.
[0163] The embodiments of the present disclosure further provide a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to execute the document classification method described in the above embodiments of the present disclosure.
[0164] The embodiments of the present disclosure further provide a computer program product, including a computer program, which executes the document classification method described in the above embodiments of the present disclosure when a processor is used to execute the computer program.
[0165] An embodiment of the present disclosure also proposes a chip, which includes one or more interface circuits and one or more processors; the interface circuit is used to receive signals from a memory of an electronic device and send signals to the processor, the signals include computer instructions stored in the memory, and when the processor executes the computer instructions, the electronic device executes the document classification method described in the above embodiments of the present disclosure.
[0166] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.
[0167] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples" or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiments or examples are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.
[0168] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present invention includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present invention belong.
[0169] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processing module, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (control method), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or otherwise processing in a suitable manner if necessary, and then stored in a computer memory.
[0170] It should be understood that the various parts of the embodiments of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.
[0171] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.
[0172] In addition, each functional unit in each embodiment of the present invention may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium. The above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.
[0173] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.
Claims
1. A document classification method, characterized in that: The method comprises: Using the pre-trained language model, each word in each text feature of the normalized document is converted into a word vector; Using an interactive attention network, taking the word vector as input, to obtain a first document category of the normalized document; Using the large language model, the normalized document is templated according to a preset number of preset prompt templates to obtain a preset number of prompt templates after the template processing, and using the preset number of prompt templates to obtain the preset number of second document categories; The first document category and the document category with the same document category and the largest number among the preset number of second document categories are used as the target document category for the normalized document.
2. The method according to claim 1, characterized in that The method of using the interactive attention network and taking the word vector as input to obtain the first document category of the normalized document includes: Determine the overall feature of each text feature by using the long short-term memory network in the interactive attention network and each word vector of each text feature; Determine the interactive feature of each text feature by using the attention mechanism in the interactive attention network, the overall feature of each text feature, and the overall feature of the preset interactive feature of each text feature; Splicing the interactive features of each text feature to obtain a spliced feature; The concatenated features are input into an activation function in the interactive attention network to obtain a first document category corresponding to the normalized document.
3. The method according to claim 2, characterized in that Determining the overall feature of each text feature by using the long short-term memory network in the interactive attention network and each word vector of each text feature includes: Inputting each word vector in each text feature into the long short-term memory network corresponding to each text feature in order of time, respectively, to obtain the hidden state of each text feature at each time; According to the hidden state of each text feature at each moment, the average value of the hidden state of each text feature is determined respectively, and the average value of the hidden state of each text feature is used as the overall feature of each text feature.
4. The method according to claim 1, characterized in that: The using of the large language model to perform template processing on the normalized document according to a preset number of preset prompt templates to obtain a preset number of prompt templates after the template processing, and using the preset number of prompt templates to obtain the preset number of second document categories includes: Using the large language model, the normalized document is filled with content according to the language structure of a preset number of preset prompt templates, to obtain prompt templates corresponding to the preset number of normalized documents, and the prompt templates corresponding to the preset number of normalized documents are encapsulated into a prompt list; A preset number of second document categories are obtained by using the large language model and the prompt list.
5. The method according to claim 4, characterized in that The obtaining of a preset number of second document categories by using the large language model and the prompt list includes: The prompt list is input into the service website of the registered account in the large language model to obtain the preset number of second document categories.
6. The method according to claim 1, characterized in that The obtaining of a preset number of second document categories by using the large language model and the prompt list includes: Generate an interface key according to the interface key interface corresponding to the service interface of the large language model; Using a package management tool in the large language model, installing a software package that interacts with the service interface, and adding the interface key to the software package; The software package to which the interface key is added and the prompt list are used to generate the preset number of second document categories, and the text completion information is called to print the prompt list and the preset number of second document categories.
7. A document classification device, characterized in that: The device comprises: A vector conversion unit, used for converting each word in each text feature of the normalized document into a word vector by using a pre-trained language model; A first classification unit is used to use an interactive attention network to take the word vector as input to obtain a first document category of the normalized document; A second classification unit is used to use a large language model to perform template processing on the normalized document according to a preset number of preset prompt templates to obtain a preset number of prompt templates after the template processing, and use the preset number of prompt templates to obtain the preset number of second document categories; The prediction unit is used to use the first document category and the document category with the same document category and the largest number among the preset number of second document categories as the target document category for the normalized document.
8. An electronic device, characterized in that: include: a processor and a memory for storing a computer program capable of being executed on the processor, Wherein, when the processor is used to run the computer program, it executes the method described in any one of claims 1 to 6.
9. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 6.
10. A computer program product, characterized in that A computer program is included which, when executed by a processor, implements the method according to any one of claims 1 to 6.
