Document classification method and device based on large model, intelligent agent and electronic equipment

Abstract

The invention provides a document classification method and device based on a large model, an intelligent agent and electronic equipment, and relates to the technical field of artificial intelligence, in particular to the technical fields of natural language processing, large models and the like. The method can be applied to scenes of enterprise internal document management based on artificial intelligence, patient record management of medical institutions, news classification, social media topic classification, spam filtering, medical literature classification, legal document classification and the like. According to the specific implementation scheme, an input document is obtained; determining the document type of the input document, wherein the document type is divided into a streaming document and a format document; generating corresponding target prompt word information according to the document type; document classification is conducted on the input document based on the target prompt word information through the multi-modal large model, the category of the input document is obtained, and the category of the input document refers to the semantic category of the input document. The prediction performance of the document classification system can be improved, and the prediction time can be shortened.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, in particular to technical fields such as natural language processing and large models, and more particularly to a document classification method, device, intelligent body and electronic equipment based on large models, which can be applied to scenarios such as internal enterprise document management based on artificial intelligence, patient record management in medical institutions, news classification, social media topic classification, spam filtering, medical literature classification, and legal document classification. Background Art

[0002] Document classification is the process of automatically assigning input document samples to predefined categories. The output is typically a semantic label (or semantic category) for the document, such as "contract document," "bid document," "form document," or "invoice document."

[0003] However, there is currently a lack of effective means to achieve document classification. Summary of the Invention

[0004] The present disclosure provides a document classification method, device, intelligent agent and electronic device based on a large model.

[0005] In a first aspect, an embodiment of the present disclosure provides a document classification method based on a large model, comprising:

[0006] Get the input document;

[0007] Determining the document type of the input document, wherein the document type is divided into a flow document and a layout document;

[0008] Generate corresponding target prompt word information according to the input document and the document type;

[0009] The input document is classified based on the target prompt word information by a multimodal large model to obtain the category to which the input document belongs. The category to which the input document belongs refers to the semantic category to which the input document belongs.

[0010] In a second aspect, an embodiment of the present disclosure provides a document classification device based on a large model, comprising:

[0011] A first acquisition module is used to acquire an input document;

[0012] A first determining module is used to determine the document type of the input document, wherein the document type is divided into a flow document and a layout document;

[0013] A generating module, configured to generate corresponding target prompt word information according to the input document and the document type;

[0014] A classification module is used to classify the input document based on the target prompt word information through a multimodal large model to obtain the category to which the input document belongs, where the category to which the input document belongs refers to the semantic category to which the input document belongs.

[0015] In a third aspect, an embodiment of the present disclosure provides an intelligent agent, including:

[0016] An input module, used for receiving input information;

[0017] a processing module, configured to determine a target task based on the input information received by the input module, determine a large model based on the target task, and execute the method described in the first aspect by calling the large model to obtain output information;

[0018] The output module is used to output the output information obtained by the processing module.

[0019] In a fourth aspect, an embodiment of the present disclosure provides an electronic device, including:

[0020] at least one processor; and

[0021] a memory communicatively connected to the at least one processor; wherein,

[0022] The memory stores instructions that can be executed by the at least one processor. The instructions are executed by the at least one processor to enable the at least one processor to perform the method described in the first aspect.

[0023] In a fifth aspect, an embodiment of the present disclosure provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable the computer to execute the method described in the first aspect above.

[0024] In a sixth aspect, an embodiment of the present disclosure provides a computer program product, comprising a computer program, wherein the computer program implements the steps of the method described in the first aspect when executed by a processor.

[0025] According to the technical solutions disclosed herein, by considering the different characteristics of layout documents and streaming documents from a document type perspective and developing targeted document classification strategies, the predictive performance of the document classification system can be improved and prediction time can be reduced. By implementing the embodiments disclosed herein, relatively satisfactory results can be achieved without the need for model training, addressing the problem that traditional document classification methods rely on large amounts of manually annotated data and model training.

[0026] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The accompanying drawings are provided to facilitate a better understanding of the present invention and do not constitute a limitation of the present disclosure.

[0028] Figure 1 A flowchart of a document classification method based on a large model provided in an embodiment of the present disclosure;

[0029] Figure 2 A flowchart of a document classification method based on a large model provided in an embodiment of the present disclosure;

[0030] Figure 3 A flowchart of a document classification method based on a large model provided in an embodiment of the present disclosure;

[0031] Figure 4 A flowchart of a document classification method based on a large model provided in an embodiment of the present disclosure;

[0032] Figure 5 A block diagram of a document classification device based on a large model provided in an embodiment of the present disclosure;

[0033] Figure 6 A block diagram of a document classification device based on a large model provided in an embodiment of the present disclosure;

[0034] Figure 7 A block diagram of a document classification device based on a large model provided in an embodiment of the present disclosure;

[0035] Figure 8 A block diagram of a document classification device based on a large model provided in an embodiment of the present disclosure;

[0036] Figure 9 is a block diagram of an electronic device according to an embodiment of the present disclosure;

[0037] Figure 10 A block diagram of an intelligent agent provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0038] The following description of exemplary embodiments of the present disclosure is made in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding. These details should be considered as merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0039] The embodiments of the present disclosure relate to artificial intelligence technology fields such as natural language processing and large models.

[0040] Artificial Intelligence, commonly known as AI, is a new technology science that studies, develops theories, methods, technologies and application systems for simulating, extending and expanding human intelligence.

[0041] Natural Language Processing (NLP) is an important direction in the field of computer science and artificial intelligence. It studies various theories and methods that can realize effective communication between people and computers using natural language. It is a discipline that uses computer technology to analyze, understand and process natural language, that is, it uses computers as a powerful tool for language research, and under the support of computers, it conducts quantitative research on language information and provides language descriptions that can be used by people and computers.

[0042] Large Language Model (LLM) refers to a deep learning model trained using a large amount of text data, which can generate natural language text or understand the meaning of language text. Large language models can handle a variety of natural language tasks such as text classification, question answering, and dialogue, and are an important way to artificial intelligence. That is, large language models usually refer to large-scale generative language models. By constructing prompts, you can quickly construct a dialogue tool that can solve natural language processing tasks based on LLM. This method does not require labeled data or model training, and can quickly obtain an NLP dialogue application.

[0043] Agent refers to an agent that can perceive the environment and take action to achieve a specific goal. It can be software, hardware or a system, with autonomy, adaptability and interaction ability. The agent perceives changes in the environment (such as through sensors or data input), makes judgments and decisions based on its learned knowledge and algorithms, and then performs actions to influence the environment or achieve predetermined goals.

[0044] It should be noted that in the technical solutions of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved in the technical solutions comply with relevant laws and regulations and do not violate public order and good customs.

[0045] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data for analysis, stored data, displayed data, etc.) and signals involved in the present disclosure are authorized by the user or fully authorized by all parties, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0046] It should also be noted that in the embodiments of the present disclosure, certain software, components, models and other existing solutions in the industry may be mentioned. They should be regarded as exemplary and their purpose is only to illustrate the feasibility of implementing the technical solution of the present disclosure, but it does not mean that the applicant has or will necessarily use the solution.

[0047] It is worth noting that document classification, as an upstream task of information extraction, has a significant impact on the performance of the entire document processing system. Exemplary products or projects applying the present disclosure may include, but are not limited to, the following 1) to 4):

[0048] 1) Enterprise Document Archiving Management: Daily business operations involve a large number of documents such as contracts, reports, and meeting minutes. Document classification technology enables automatic archiving, classified storage, and rapid retrieval of documents, significantly improving office efficiency and reducing labor costs.

[0049] 2) Financial risk control and credit review: Banks or internet financial platforms collect a large number of documents submitted by users during the credit review process, including ID cards, payslips, and credit reports. Document classification models can automatically determine file types, assist in the document review process, and expedite approvals.

[0050] 3) Legal Document Classification and Retrieval: In the legal services industry, there are numerous types of documents, including contracts, judgments, indictments, and evidence materials. Leveraging a classification system can effectively improve the efficiency of legal document retrieval and comparison, supporting lawyer document assistance tools or intelligent review systems.

[0051] 4) Medical Document Informationization: Hospitals and medical institutions generate a large number of medical documents daily, including examination reports, prescriptions, and medical records. Classification systems can categorize these documents into different departments or disease types, facilitating the construction of electronic health record (EHR) systems and supporting subsequent analysis and diagnosis.

[0052] The following describes the document classification method, device, intelligent agent and electronic device based on the large model according to the embodiments of the present disclosure with reference to the accompanying drawings.

[0053] It should be noted that the executor of the document classification method based on a large model in the embodiment of the present disclosure can be a document classification device based on a large model, which can be implemented by software and / or hardware, and can be configured in an electronic device, which can include but is not limited to a terminal, a server, etc.

[0054] It is worth noting that the document classification method based on a large model of the embodiment of the present disclosure can be implemented by an autonomous agent (AutoAgent) based on a large language model, which can classify input documents based on a multimodal large language model.

[0055] Figure 1 Flowchart of the document classification method based on the large model provided in the embodiment of the present disclosure. Figure 1 As shown, the document classification method based on the large model may include but is not limited to the following steps.

[0056] In step 101, an input document is obtained.

[0057] In some embodiments, the format of the input document may include but is not limited to at least one of the following: .doc or .docx format, .txt format, .pdf format, .jpg format, .ofd format, .xls format, etc.

[0058] In step 102, the document type of the input document is determined. The document types are divided into flow documents and layout documents.

[0059] It should be noted that documents can be divided into layout documents and flow documents from the perspective of document type. In the embodiment of the present disclosure, when obtaining an input document, the document type of the input document can be determined to facilitate the subsequent generation of different target prompt word information based on different document types.

[0060] In some embodiments, the above-mentioned streaming document may refer to a document whose content is mainly in natural language, whose structure flows linearly, and whose layout is adaptively adjusted with the display device. The above-mentioned layout document may refer to a document with a clear visual layout structure, a fixed content position, and information organization usually in units of pages. Exemplarily, common file extensions (also called suffixes or extended file names) of streaming documents may include but are not limited to: .doc, .docx, .txt, .xls, .xlsx, etc. Exemplarily, common file extensions of layout documents may include but are not limited to: .pdf, .ofd, .jpeg, .png, .bmp, etc.

[0061] In an embodiment of the present disclosure, the document type of the input document may be determined based on the file extension of the input document.

[0062] In step 103, corresponding target prompt word information is generated according to the input document and its document type.

[0063] In an embodiment of the present disclosure, target prompt word information corresponding to different document types can be generated. For example, if the document type is a streaming document, for example, the input document is a streaming document, corresponding streaming prompt word information can be generated based on the input document and its document type as the target prompt word information. For example, if the document type is a layout document, for example, the input document is a layout document, corresponding layout prompt word information can be generated based on the input document and its document type as the target prompt word information. The target prompt word information can be used to instruct the multimodal large model to classify the input document.

[0064] In step 104, the input document is classified based on the target prompt word information through the multimodal large model to obtain the category to which the input document belongs.

[0065] In some embodiments, the multimodal big model may include a streaming document big model and a layout document big model. The streaming document big model may be used to classify streaming documents; and the layout document big model may be used to classify layout documents.

[0066] In an embodiment of the present disclosure, a corresponding large model can be called based on different document types. The input document can be classified by the large model based on the target prompt word information corresponding to the document type, so that the category to which the input document belongs can be obtained. The category to which the input document belongs can refer to the semantic category (also called semantic label) to which the input document belongs, such as "contract documents", "bid documents", "form documents", "bill documents", etc.

[0067] For example, if the input document is a streaming document, the streaming document macromodel can be used to classify the input document based on target prompt word information (e.g., streaming prompt word information generated based on the input document) to obtain the category to which the input document belongs. If the input document is a layout document, the layout document macromodel can be used to classify the input document based on target prompt word information (e.g., layout prompt word information generated based on the input document) to obtain the category to which the input document belongs.

[0068] In the above-mentioned embodiments, by considering the different characteristics of layout documents and flow documents from the perspective of document type and formulating targeted document classification strategies, the prediction performance of the document classification system can be improved and prediction time can be reduced. By implementing the disclosed embodiments, relatively satisfactory results can be achieved without the need for model training, thus resolving the problem that traditional document classification methods rely on large amounts of manually annotated data and model training.

[0069] Figure 2 Flowchart of the document classification method based on the large model provided in the embodiment of the present disclosure. Figure 2As shown, the document classification method based on the large model may include but is not limited to the following steps.

[0070] In step 201, an input document is obtained.

[0071] Optionally, step 201 may be implemented in any of the implementation methods in the embodiments of the present disclosure. The embodiments of the present disclosure do not limit this and will not be described in detail.

[0072] In step 202, the document type of the input document is determined. The document types are divided into flow documents and layout documents.

[0073] Optionally, step 202 may be implemented in any of the implementation methods in the embodiments of the present disclosure, which is not limited in the embodiments of the present disclosure and will not be described in detail.

[0074] In step 203, the key part of the input document is obtained.

[0075] In the embodiment of the present disclosure, the key part can be used to measure the semantic category to which the input document belongs. That is, the semantic category to which the input document belongs, that is, which semantic tag file it belongs to, can be determined by extracting the feature content in the key part.

[0076] It should be noted that the way to obtain the key parts of the input document will be different if the document type of the input document is different. It is worth noting that for the streaming document classification scenario, a streaming document is input, and the text content of the document is first obtained through the document parsing service. The inventors of the present disclosure experimented with using document content of different lengths to enter the multimodal large model for document classification. The experiment showed that when the length is set to the above-mentioned preset length (such as 3000 words), the prediction performance reaches the best. The image information on the homepage in the layout document can usually effectively reflect the category of the document.

[0077] Based on this, in some embodiments, when the document type is a streaming document, based on a preset length, the beginning content of the document intercepted from the input document can be used as the key part; or, when the document type is a layout document, the first page of the input document can be converted into an image as the key part. For example, for a streaming document, the first 3,000 words of the beginning content of the streaming document can be intercepted as the key part of the streaming document. For a layout document, the first page of the layout document (also called the first page) can be converted into an image, and the image can be used as the key part of the layout document.

[0078] In step 204, target prompt word information corresponding to the document type is generated based on the key part and pre-configured classification categories and content features.

[0079] In the embodiment of the present disclosure, in the process of designing target prompt word information, in addition to the task definition, the present disclosure introduces prior information such as key parts, classification categories and content features of the input document.

[0080] In embodiments of the present disclosure, the pre-configured classification categories and content features described above may be pre-defined rules. In some embodiments, the classification categories may include multiple semantic categories, such as, but not limited to, contracts, bids, forms, and bills. The content features may refer to content features included in files of various semantic categories, and these content features may serve as a key basis for identifying and defining the semantic category to which a corresponding document belongs.

[0081] For example, for documents classified as contracts, content features may include, but are not limited to, the contract title, information about Party A and Party B, contract number, and contract terms. For documents classified as bid documents, content features may include, but are not limited to, the project name, bidder qualifications, and project overview. For documents classified as forms, content features may include, but are not limited to, a summary and keywords. For documents classified as bills, content features may include, but are not limited to, a title such as "receipt," "check," or "draft," accompanied by a unique number.

[0082] In an embodiment of the present disclosure, target prompt word information corresponding to the document type of the input document may be generated based on key parts of the input document and pre-configured classification categories and content features in combination with prompt word templates.

[0083] For example, for a streaming document, the prompt word template may include the following:

[0084] You are a professional document classification assistant. Your task is to analyze the beginning and file extension of a document and accurately classify it into predefined categories.

[0085] [Input information]

[0086] The beginning of the document:

[0087] <doc_front_pages>

[0088] {DOC_FRONT_PAGES}

[0089] < / doc_front_pages>

[0090] [Classification categories and content characteristics]

[0091] 1. Contract documents

[0092] -Content features: contract title, Party A and Party B information, contract number, contract terms

[0093] 2. Bidding documents

[0094] -Content characteristics: bidding project name, bidder qualifications, project overview

[0095] ……”.

[0096] For another example, for a layout document, the prompt word template may include the following content:

[0097] You are a professional document classification assistant. Your task is to analyze the document content and accurately classify it into predefined categories.

[0098] [Input information]

[0099] picture:

[0100] [Classification categories and content characteristics]

[0101] 1. Contract documents

[0102] -Content features: contract title, Party A and Party B information, contract number, contract terms

[0103] 2. Bidding documents

[0104] -Content characteristics: bidding project name, bidder qualifications, project overview

[0105] ……”.

[0106] Based on the document type and key portion of the input document, as well as pre-configured classification categories and content features, combined with a prompt word template, target prompt word information corresponding to the document type can be generated. For a stream document, the key portion can be the beginning of the document, cut from the stream document to a preset length; for a layout document, the key portion can be an image converted from the first page of the layout document.

[0107] In step 205, the input document is classified based on the target prompt word information through the multimodal large model to obtain the category to which the input document belongs.

[0108] In some embodiments, the multimodal macro model can be divided into a streaming document macro model and a layout document macro model. In some embodiments, if the document type is a streaming document, the streaming document macro model can be used to classify the input document based on the target prompt word information; or, if the document type is a layout document, the layout document macro model can be used to classify the input document based on the target prompt word information.

[0109] In some embodiments, the streaming document model can be used to identify the document type of the streaming document, and can be, for example, an Ernie-speed language model, but is not limited thereto. In some embodiments, the layout document model can be a visual language multimodal language model, and can be, for example, an InternVL 2.5 model, but is not limited thereto.

[0110] In the embodiments of the present disclosure, since the target prompt word information incorporates prior information such as the key parts, classification categories, and content features of the input document, the multimodal large model can match the content features of the key parts based on the multiple classification categories and content features of each classification category in the target prompt word information when classifying the input document. Based on the matching results, the category to which the input document belongs can be predicted. For example, assuming that the content features of the key parts of the input document include "contract title" and "Party A and Party B information," the category to which the input document belongs can be determined to be "contract." For another example, assuming that the content features of the key parts of the input document include "bidding project name" and "bidder qualifications," the category to which the input document belongs can be determined to be "bid document."

[0111] In the above embodiment, the corresponding prompt word information is automatically generated by inputting the document content and document type. When generating the prompt word information, prior information such as the classification category and content characteristics is built into the prompt word information. This allows the document classification to be achieved directly by utilizing the thinking ability of the large language model, and can achieve relatively satisfactory results without the need for model training. This can solve the problem that traditional document classification methods rely on large amounts of manually annotated data and model training. In addition, by extracting part of the content of the input document (i.e., the key part of the content) for the construction of the prompt word information, the present disclosure can significantly save document parsing time, further reduce the prediction time of the document classification system, and further improve prediction efficiency.

[0112] Figure 3 Flowchart of the document classification method based on the large model provided in the embodiment of the present disclosure. Figure 3 As shown, the document classification method based on the large model may include but is not limited to the following steps.

[0113] In step 301, an input document is obtained.

[0114] Optionally, step 301 may be implemented in any of the implementation methods in the embodiments of the present disclosure, which is not limited in the embodiments of the present disclosure and will not be described in detail.

[0115] In step 302, the document type of the input document is determined. The document types are divided into flow documents and layout documents.

[0116] Optionally, step 302 may be implemented in any of the implementations in the embodiments of the present disclosure, which is not limited in the embodiments of the present disclosure and will not be described in detail.

[0117] In step 303, the key part of the input document is obtained.

[0118] Optionally, step 303 may be implemented in any of the implementation methods in the embodiments of the present disclosure, which is not limited in the embodiments of the present disclosure and will not be described in detail.

[0119] In step 304, the file extension of the input document is determined.

[0120] In step 305 , common file extensions of the classification category in documents belonging to the document type are obtained.

[0121] It is worth noting that for documents of different document types, the common file extensions under the same classification category will also be different. For example, for streaming scenarios, the common file extensions of documents classified as contracts may include, but are not limited to, .pdf, .doc, .docx, etc.; for documents classified as tender documents, the common file extensions of tender documents may include, but are not limited to, .pdf, .doc, .docx, etc. For layout scenarios, the common file extensions of documents classified as contracts may include, but are not limited to, .pdf; for documents classified as tender documents, the common file extensions of tender documents may include, but are not limited to, .pdf.

[0122] It should be noted that, in some embodiments, the above steps 303, 304, and 305 can be executed in an interchangeable order or simultaneously.

[0123] In step 306, target prompt word information corresponding to the document type is generated based on the key part, the file extension, classification category and content characteristics of the input document, and common file extensions of the classification category in documents belonging to the document type.

[0124] In order to further improve the effectiveness of the prompt words, optionally, in an embodiment of the present disclosure, in the process of designing the target prompt word information, in addition to the task definition, the present disclosure introduces prior information such as the key parts of the input document, the classification category and content features, the file extension of the input document, and the common file extensions of the classification category.

[0125] In an embodiment of the present disclosure, target prompt word information corresponding to the document type of the input document can be generated based on the key parts of the input document, the file extension of the input document, the pre-configured classification category and content characteristics, and the common file extension of the classification category, combined with the prompt word template.

[0126] To force the large model to think, in some embodiments, optionally, thought constraint information can be obtained and written into the target prompt information. This thought constraint information can be used to stimulate the multimodal large model's thinking process. In embodiments of the present disclosure, this thought constraint information can include, but is not limited to, at least one of the following: thought precautions and classification reasons.

[0127] Exemplarily, the thinking constraint information may include thinking precautions information and classification reasons. For example, the thinking precautions information may include the following:

[0128] "Notice:

[0129] 1. The file extension is only used as an auxiliary basis for judgment, and classification is mainly based on the document content.

[0130] 2. If the content features are not obvious enough, choose a category with lower confidence

[0131] ……”.

[0132] It should be noted that introducing classification reasons in the target cue word information can significantly improve the prediction accuracy because it can stimulate the model's thinking process and prompt the model to make more accurate classifications in complex tasks.

[0133] In step 307, the input document is classified based on the target prompt word information using the multimodal large model to obtain the category to which the input document belongs.

[0134] Optionally, step 307 may be implemented in any of the implementation methods in the embodiments of the present disclosure, which is not limited in the embodiments of the present disclosure and will not be described in detail.

[0135] Optionally, in some embodiments, the target prompt word information may include output requirement information, which may be used to instruct the multimodal large model to output according to the expected output format. For example, the output requirement in the target prompt word information may be expressed as follows:

[0136] "[Output requirements]

[0137] Please output in the following format:

[0138] <Classification results>

[0139] {{

[0140] \"category\":"Category",

[0141] \"confidence\":"High / Medium / Low",

[0142] \"reason\":"Reason for classification, including content characteristics and file format considerations"

[0143] }}

[0144] < / 分类结果> ”.

[0145] In an embodiment of the present disclosure, the multimodal large model files and classifies the input documents based on the target prompt word information, and can output them according to the expected output format in the target prompt word information, so as to facilitate finding the classification results from the output content, and then facilitate accurate extraction of the "category" field from the classification results, and then output the "category" as output information, such as outputting it to the user for viewing.

[0146] Optionally, in some embodiments, it is determined that the output format of the classification result of the multimodal large model for the input document does not conform to the expected output format; the output format of the classification result is post-processed and corrected using a preset regular expression. It can be understood that in most cases, the large language model has a strong ability to follow instructions and will output according to the output format given in the target prompt word information. However, due to the uncertainty of the output of the large language model, it may output a situation that does not conform to the expected output format. When it is determined that the output format of the classification result of the multimodal large model for the input document does not conform to the expected output format, the output format of the classification result can be post-processed and corrected using a preset regular expression to normalize the predicted classification result to a predetermined category.

[0147] Optionally, in some embodiments, the classification result output by the multimodal large model according to the output requirements can be obtained; and the category to which the input document belongs can be obtained from the classification result. That is, the multimodal large model classifies the input document based on the target prompt word information, and can output the classification result according to the output requirements in the target prompt word information, thereby obtaining the classification result output by the multimodal large model. The classification result may include the category to which the input document belongs, so the category to which the input document belongs can be obtained from the classification result. For example, taking the example of the output requirements in the target prompt word information given above as an example, the category to which the input document belongs can be obtained based on the category field in the classification result.

[0148] Optionally, in some embodiments, the category to which the input document belongs can be output as output information. In other words, the method of the disclosed embodiment does not need to present the target prompt word generation process and the multimodal large model thinking process to the user. Instead, the final document category is provided to the user. In other words, the user can provide an input document and obtain the category to which the input document belongs.

[0149] In order to facilitate those skilled in the art to understand the present disclosure, Figure 4 The disclosed solution is described.

[0150] like Figure 4 As shown, after obtaining the input document, it can be determined whether the document is a streaming document or a layout document. In the case of a streaming document, the document can be subjected to a document parsing service to obtain the beginning content of the first 3,000 words of the document. Based on prior information such as the beginning content, the document suffix, the classification category and content characteristics, and the common suffixes of the category in the streaming document, streaming prompt information (streaming prompt) is generated. In addition, the streaming prompt can include output requirements, and the classification result in the output requirements requires the classification reason to be given. For streaming scenarios, the streaming document large model can adopt the Ernie-speed model. The Ernie-speed model is used to classify the document based on the streaming prompt, and the classification result output by the Ernie-speed model can be obtained. The classification result is then post-processed and corrected according to the expected output format, so that the category to which the document belongs can be obtained from the classification result and output to the user as output information.

[0151] In the case where the document is a layout document, the document can be converted into a document format, and the homepage of the layout document can be converted into a picture. Based on the picture, the document suffix, the classification category and content characteristics, the common suffix of the category in the streaming document and other prior information, the layout prompt information (layout prompt) is generated. In addition, the layout prompt can include output requirements, and the classification result in the output requirements requires the classification reason to be given. For the layout scenario, the layout document large model can adopt the InternVL 2.5 model. The document is classified based on the layout prompt by the InternVL 2.5 model, and the classification result output by the InternVL 2.5 model can be obtained. The classification result is post-processed and corrected according to the expected output format, so that the category to which the document belongs can be obtained from the classification result and output to the user as output information.

[0152] The inventors conducted experiments on the technical solutions involved in this disclosure. The experimental process is as follows:

[0153] Automatic data set annotation: We collected over 800 documents of different categories, each of which is about 30 pages long. The predefined categories are ['bid', 'general document', 'certificate', 'bill', 'contract', 'form', 'general image']. GPT-4 is used to automatically generate document classification results.

[0154] Plain Text LLM & Multimodal Large Model Experiment: This paper compared the classification performance of the Plain Text LLM and the Multimodal LLM. The multimodal large model proposed in this paper, which targets layout and flow document features, uses the Ernie-speed + InternVL 2.5 model. The experimental results are shown in Table 1 below:

[0155] Table 1

[0156]

[0157] As can be seen, in the case of the plain text LLM, the macro f1 (macro average F1, an indicator for evaluating the performance of multi-classification tasks) is 0.8044, while in the case of the Ernie-speed+InternVL 2.5 model proposed in this disclosure, the macro f1 is 0.9440 (an improvement of +14%). The flow scenario takes 6.6700s (an improvement of +66%), and the layout scenario takes 4.0106s (an improvement of +79%). It can be seen that the method proposed in this disclosure has achieved significant improvements in prediction performance.

[0158] Figure 5 This is a block diagram of a document classification device based on a large model provided in an embodiment of the present disclosure. Figure 5 As shown, the document classification device based on the large model may include: a first acquisition module 501, a first determination module 502, a generation module 503 and a classification module 504.

[0159] The first acquisition module 501 is used to acquire an input document.

[0160] The first determining module 502 is used to determine the document type of the input document. The document types are divided into flow documents and layout documents.

[0161] The generating module 502 is used to generate corresponding target prompt word information according to the input document and document type.

[0162] The classification module 504 is used to classify the input document based on the target prompt word information using the multimodal large model to obtain the category to which the input document belongs. The category to which the input document belongs refers to the semantic category to which the input document belongs.

[0163] In some embodiments, the generation module 502 is used to: obtain key parts in the input document, which are used to measure the semantic category to which the input document belongs; and generate target prompt word information corresponding to the document type based on the key parts and pre-configured classification categories and content features.

[0164] In some embodiments, the generation module 502 is used to: when the document type is a streaming document, based on a preset length, use the beginning content of the document intercepted from the input document as the key part; or, when the document type is a layout document, convert the first page of the input document into a picture as the key part.

[0165] Optionally, in some embodiments, Figure 6 As shown, in Figure 5 Based on the above, the document classification device based on the large model may further include: a second determination module 605 and a second acquisition module 606. The second determination module 605 is used to determine the file extension of the input document; the second acquisition module 606 is used to acquire common file extensions of the classification category in documents belonging to the document type.

[0166] In some embodiments, the generating module 602 is used to generate target prompt word information corresponding to the document type based on the key part, the file extension of the input document, the classification category and content characteristics, and the common file extensions of the classification category in the documents belonging to the document type. Figure 6 601-604 and Figure 5 501-504 have the same function and structure.

[0167] Optionally, in some embodiments, Figure 7 As shown, in Figure 6 On the basis shown in the figure, the document classification device based on the large model may further include: a third acquisition module 707. The third acquisition module 707 is used to obtain thinking constraint information. Among them, the generation module 702 is also used to: write the thinking constraint information into the target prompt word information, and the thinking constraint information is used to stimulate the thinking process of the multimodal large model. Among them, the thinking constraint information includes at least one of the following: thinking precautions information, classification reasons. Among them, Figure 7 701-706 and Figure 6 601-606 have the same function and structure.

[0168] In some embodiments, the multimodal large model is divided into a streaming document large model and a layout document large model; the classification module is configured to: if the document type is a streaming document, classify the input document based on the target prompt word information using the streaming document large model; or, if the document type is a layout document, classify the input document based on the target prompt word information using the layout document large model. The layout document large model is a visual language multimodal large language model.

[0169] In some embodiments, the target prompt word information includes output requirement information, and the output requirement information is used to instruct the multimodal large model to output according to the expected output format.

[0170] In some embodiments, as Figure 8 As shown, in Figure 7 On the basis of the above, the document classification device based on the large model may further include: a third determination module 808 and a post-processing module 809. The third determination module 808 is used to determine whether the output format of the classification result of the multimodal large model for the input document does not conform to the expected output format; the post-processing module 809 is used to post-process and correct the output format of the classification result using a preset regular expression. Figure 8 801-807 and Figure 7 701-707 have the same function and structure.

[0171] In some embodiments, the classification module is used to: obtain the classification results output by the multimodal large model according to the output requirements; and obtain the category to which the input document belongs from the classification results.

[0172] In some embodiments, the large model-based document classification device may further include an output module configured to output the category of the input document as output information.

[0173] Regarding the apparatus in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0174] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device and a readable storage medium.

[0175] like Figure 9 , is a block diagram of an electronic device according to an embodiment of the present disclosure. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0176] like Figure 9As shown, the electronic device includes: one or more processors 901, a memory 902, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. The various components are connected to each other using different buses and can be installed on a common mainboard or installed in other ways as needed. The processor can process instructions executed in the electronic device, including instructions stored in or on the memory to display graphical information of a GUI on an external input / output device (such as a display device coupled to the interface). In other embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple electronic devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 9 A processor 901 is taken as an example.

[0177] Memory 902 is a non-transitory computer-readable storage medium provided in the present disclosure. The memory stores instructions executable by at least one processor, causing the at least one processor to execute the large model-based document classification method provided in the present disclosure. The non-transitory computer-readable storage medium of the present disclosure stores computer instructions for causing a computer to execute the large model-based document classification method provided in the present disclosure.

[0178] The memory 902 is a non-transient computer-readable storage medium that can be used to store non-transient software programs, non-transient computer executable programs and modules, such as the program instructions / modules corresponding to the document classification method based on the large model in the embodiment of the present disclosure (for example, the attached Figure 5 The processor 901 executes the non-transient software programs, instructions, and modules stored in the memory 902 to execute various functional applications and data processing of the server, thereby implementing the large model-based document classification method in the above method embodiment.

[0179] The memory 902 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and applications required for at least one function; the data storage area may store data created based on the use of the electronic device, etc. In addition, the memory 902 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory 902 may optionally include a memory remotely located relative to the processor 901, and these remote memories may be connected to the electronic device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0180] The electronic device may further include: an input device 903 and an output device 904. The processor 901, the memory 902, the input device 903 and the output device 904 may be connected via a bus or other means. Figure 9 The bus connection is taken as an example.

[0181] The input device 903 can receive input digital or character information and generate key signal input related to user settings and function control of the electronic device, such as input devices such as a touch screen, a keypad, a mouse, a trackpad, a touch pad, an indicator stick, one or more mouse buttons, a trackball, and a joystick. The output device 904 may include a display device, an auxiliary lighting device (e.g., an LED), and a tactile feedback device (e.g., a vibration motor). The display device may include, but is not limited to, a liquid crystal display (LCD), a light emitting diode (LED) display, and a plasma display. In some embodiments, the display device may be a touch screen.

[0182] Various implementations of the systems and techniques described herein can be realized in digital electronic circuit systems, integrated circuit systems, dedicated ASICs (application specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0183] These computer programs (also referred to as programs, software, software applications, or code) include machine instructions for a programmable processor and can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and / or device (e.g., a magnetic disk, an optical disk, a memory, a programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0184] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0185] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), the Internet, and a blockchain network.

[0186] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact via a communication network. This client-server relationship is established by computer programs running on the respective computers and establishing a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, a host product within a cloud computing service ecosystem that addresses the management difficulties and limited business scalability of traditional physical hosts and VPS services ("Virtual Private Servers" or simply "VPS"). The server may also be a server in a distributed system or a server integrated with blockchain.

[0187] Figure 10 This is a block diagram of an intelligent agent provided by an embodiment of the present disclosure. Figure 10As shown, the agent may include an input module 1001, a processing module 1002, and an output module 1003. The input module 1001 is configured to receive input information; the processing module 1002 is configured to determine a target task based on the input information received by the input module 1001, determine a large model based on the target task, and execute the large model-based document classification method in the above method embodiment by calling the large model to obtain output information; and the output module 1003 is configured to output the output information obtained by the processing module 1002.

[0188] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed herein can be achieved, and this disclosure is not limited thereto.

[0189] The above specific embodiments do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the scope of protection of this disclosure.

Claims

1. A document classification method based on a large model, comprising: Get the input document; Determining the document type of the input document, wherein the document type is divided into a flow document and a layout document; Generate corresponding target prompt word information according to the input document and the document type; The input document is classified based on the target prompt word information by a multimodal large model to obtain the category to which the input document belongs. The category to which the input document belongs refers to the semantic category to which the input document belongs.

2. The method according to claim 1, wherein Generating corresponding target prompt word information according to the input document and the document type includes: Obtaining a key portion of the input document, where the key portion is used to measure the semantic category to which the input document belongs; Target prompt word information corresponding to the document type is generated according to the key part and pre-configured classification categories and content features.

3. The method of claim 2, further comprising: Determining the file extension of the input document; Gets common file extensions of the classification category in documents belonging to the document type.

4. The method according to claim 3, wherein: Generating target prompt word information corresponding to the document type based on the key part and pre-configured classification categories and content features includes: Target prompt word information corresponding to the document type is generated according to the key part, the file extension of the input document, the classification category and content characteristics, and the common file extension of the classification category in the documents belonging to the document type.

5. The method according to claim 2, wherein: The obtaining of the key part of the input document includes: In the case where the document type is the streaming document, based on a preset length, the beginning content of the document intercepted from the input document is used as the key part; or In the case that the document type is the layout document, the first page of the input document is converted into a picture as the key part.

6. The method according to any one of claims 1 to 5, further comprising: Obtain thinking constraint information; The thinking constraint information is written into the target prompt word information, and the thinking constraint information is used to stimulate the thinking process of the multimodal large model.

7. The method according to claim 6, wherein: The thinking constraint information includes at least one of the following: thinking precautions information, classification reasons.

8. The method of claim 1, wherein: The multimodal large model is divided into a streaming document large model and a layout document large model; the document classification of the input document based on the target prompt word information by the multimodal large model includes: In the case where the document type is the streaming document, the input document is classified based on the target prompt word information by using the streaming document macro model; or In the case where the document type is the layout document, the input document is classified based on the target prompt word information using the layout document macro model.

9. The method of claim 8, wherein: The layout document large model is a visual language multimodal large language model.

10. The method of claim 1, wherein: The target prompt word information includes output requirement information, and the output requirement information is used to instruct the multimodal large model to output according to the expected output format.

11. The method of claim 10, further comprising: Determining that an output format of a classification result of the multimodal large model for the input document does not conform to the expected output format; The output format of the classification result is post-processed and corrected using a preset regular expression.

12. The method according to claim 10 or 11, wherein: The obtaining of the category to which the input document belongs includes: Obtaining the classification result output by the multimodal large model according to the output requirement; The category to which the input document belongs is obtained from the classification result.

13. The method of claim 1 , further comprising: The category to which the input document belongs is output as output information.

14. A document classification device based on a large model, comprising: A first acquisition module is used to acquire an input document; A first determining module is used to determine the document type of the input document, wherein the document type is divided into a flow document and a layout document; A generating module, configured to generate corresponding target prompt word information according to the input document and the document type; A classification module is used to classify the input document based on the target prompt word information through a multimodal large model to obtain the category to which the input document belongs, where the category to which the input document belongs refers to the semantic category to which the input document belongs.

15. An intelligent agent comprising: An input module, used for receiving input information; a processing module, configured to determine a target task based on the input information received by the input module, determine a large model based on the target task, and obtain output information by executing the method according to any one of claims 1 to 13 by calling the large model; The output module is used to output the output information obtained by the processing module.

16. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 13.

17. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 13.

18. A computer program product comprising a computer program, wherein When the computer program is executed by a processor, the computer program implements the steps of the method according to any one of claims 1 to 13.

Images