Information extraction method and device, equipment and medium

Abstract

The invention relates to an artificial intelligence technology, and provides an information extraction method and device, equipment and a medium, on one hand, extracted text information is fused with document layout information obtained after layout analysis, and image restoration can be performed on a result after text recognition so as to improve the accuracy of model recognition; on one hand, the target large language model is called from the pre-constructed model pool according to the information extraction demand data, and the large language model matched with the actual demand is called, so that the model deployment cost can be reduced while the task requirement is met; on the other hand, the target prompt is generated according to the fusion information and the key information description, the target large language model is called to process the target document based on the target prompt and the key information extraction strategy, the model does not need to be retrained for different tasks, and therefore generalization and universality can be improved while information extraction can be efficiently executed.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to an information extraction method, device, equipment and medium. Background Art

[0002] Key Information Extraction (KIE) refers to extracting key information from text or images. There are many practical application scenarios for key information extraction tasks for document images, such as ID card information extraction and invoice recognition.

[0003] Taking banks as an example, many business scenarios involve the need to review documents. For example, the review of documents such as airway bills, bills of lading, and letters of credit is an indispensable and important link throughout international trade business. However, manual review often has many pain points, such as high labor costs, low labor efficiency, and high experience dependence. In this case, combining key information extraction technology based on AI (Artificial Intelligence) technology, rule comparison, and other methods to realize the review of various documents in different scenarios can save a lot of manpower and time, and provide more possibilities for the development and intelligent transformation of the financial industry. Therefore, key information extraction technology has important application value.

[0004] However, the existing technology mainly uses GCN (Graph Convolutional Network), LayoutXLM, Donut, etc. as the base model, and uses the key information extraction task as the downstream task for iterative training, but this method still has the following shortcomings:

[0005] 1. Requires a large amount of data sets for training;

[0006] 2. It does not generalize well to data sets that are very different from the training set;

[0007] 3. It is easy to have "hallucination" problems, and unreliable answers "out of thin air" appear for samples that have never been seen;

[0008] 4. The time cost is high.

[0009] In view of the above defects, how to extract information efficiently and with high generalization has become an urgent problem to be solved. Summary of the invention

[0010] The embodiments of the present application provide an information extraction method, apparatus, device and medium, aiming to solve the problems of low efficiency and low generalization of information extraction.

[0011] In a first aspect, an embodiment of the present application provides an information extraction method, which includes:

[0012] In response to an information extraction instruction for a target document, performing text recognition on the target document to obtain text information of the target document;

[0013] Performing layout analysis on the target document to obtain document layout information of the target document;

[0014] Merging the text information with the document layout information to obtain fused information;

[0015] Parsing the information extraction instruction to obtain information extraction requirement data, generating key information description and key information extraction strategy according to the information extraction requirement data, and retrieving a target large language model from a pre-built model pool according to the information extraction requirement data;

[0016] Generate a target prompt according to the fusion information and the key information description;

[0017] Based on the target prompt and the key information extraction strategy, the target large language model is called to process the target document to obtain the information extraction result of the target document.

[0018] In a second aspect, an embodiment of the present application provides an information extraction device, comprising:

[0019] A recognition unit, configured to perform text recognition on the target document in response to an information extraction instruction for the target document, and obtain text information of the target document;

[0020] An analysis unit, used for performing layout analysis on the target document to obtain document layout information of the target document;

[0021] A fusion unit, used for fusing the text information with the document layout information to obtain fusion information;

[0022] A parsing unit, used to parse the information extraction instruction to obtain information extraction requirement data, generate key information description and key information extraction strategy according to the information extraction requirement data, and retrieve a target large language model from a pre-built model pool according to the information extraction requirement data;

[0023] A generating unit, configured to generate a target prompt according to the fusion information and the key information description;

[0024] A processing unit is used to call the target large language model to process the target document based on the target prompt and the key information extraction strategy to obtain the information extraction result of the target document.

[0025] In a third aspect, an embodiment of the present application further provides a computer device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the information extraction method described in the first aspect when executing the computer program.

[0026] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor executes the information extraction method described in the first aspect above.

[0027] The embodiments of the present application provide an information extraction method, apparatus, device and medium. On the one hand, the extracted text information is fused with the document layout information obtained after layout analysis, and the image restoration of the text recognition result can be performed to improve the accuracy of model recognition; on the other hand, the target large language model is called from the pre-built model pool according to the information extraction requirement data. By calling the large language model that matches the actual needs, the model deployment cost can be reduced while meeting the task requirements; on the other hand, the target prompt is generated according to the fused information and the key information description, and the target large language model is called based on the target prompt and the key information extraction strategy to process the target document. There is no need to retrain the model for different tasks, so the generalization and versatility can be improved while efficiently executing information extraction. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0029] Figure 1 A schematic diagram of an application scenario of the information extraction method provided in an embodiment of the present application;

[0030] Figure 2 A schematic diagram of a flow chart of an information extraction method provided in an embodiment of the present application;

[0031] Figure 3 A schematic block diagram of an information extraction device provided in an embodiment of the present application;

[0032] Figure 4 A schematic block diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0033] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0034] It should be understood that when used in this specification and the appended claims, the terms "include" and "comprises" indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0035] It should also be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0036] It should be further understood that the term “and / or” used in the specification and appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0037] See also Figure 1 and Figure 2 , Figure 1 A schematic diagram of an application scenario of the information extraction method provided in an embodiment of the present application; Figure 2 This is a flow chart of an information extraction method provided in an embodiment of the present application. The information extraction method is applied in a server and is executed by application software installed in the server.

[0038] like Figure 2 As shown, the method includes steps S101 to S106.

[0039] S101. In response to an information extraction instruction for a target document, perform text recognition on the target document to obtain text information of the target document.

[0040] In this embodiment, the technical solution is described with the server as the execution subject. The user terminal (such as a smart phone, tablet computer and other intelligent terminals) used by the user can interact with the server for data. Specifically, the server provides an information extraction platform, and the user can log in to the information extraction platform using the user terminal. The user interaction interface of the information extraction platform is displayed on the terminal interface of the user terminal, and there is at least one image upload interface in the user interaction interface. When the user selects an image as the image to be identified and uploads it to the server through the image upload interface, subsequent information extraction can be performed in the server.

[0041] In this embodiment, the target document may include, but is not limited to: invoice, bill of lading, contract, air waybill, etc.

[0042] In this embodiment, when it is detected that an image has been uploaded to the server, it can be determined that the information extraction instruction has been received.

[0043] In this embodiment, performing text recognition on the target document to obtain text information of the target document includes:

[0044] Optical Character Recognition (OCR) is used to perform text recognition on the target document to obtain text information of the target document.

[0045] Of course, in other embodiments, statistical methods, structural methods, image processing technology, machine learning, and deep learning technology may also be used to perform text recognition on the target document, and the specific selection may be made based on the technical conditions in the actual scenario.

[0046] In this embodiment, after the text recognition is performed, text rearrangement, layout restoration, and layout analysis may be performed to standardize the text form.

[0047] S102: Perform layout analysis on the target document to obtain document layout information of the target document.

[0048] In this embodiment, performing layout analysis on the target document to obtain document layout information of the target document includes:

[0049] Performing region segmentation on the target document to obtain sub-regions of each content type;

[0050] The association between the sub-regions is analyzed to obtain document layout information of the target document.

[0051] Of course, in other embodiments, the layout analysis of the target document may also be performed by means of grid design, logical layout analysis, etc., and this application does not limit the layout analysis method used.

[0052] Through the above embodiment, the document layout information of the target document can be analyzed and obtained for use in subsequent document restoration.

[0053] S103: Fusing the text information with the document layout information to obtain fused information.

[0054] In this embodiment, the fusion of the text information and the document layout information to obtain fusion information includes:

[0055] Obtaining the location coordinates of the text information;

[0056] Generate table information according to the document layout information;

[0057] The text information is restored to a picture layout according to the table information and the position coordinates to obtain the fusion information.

[0058] For example, for invoice X, you can use <box> 、< / box> is the location coordinate identifier, and the fusion information is obtained: [invoice X <box> [150,39,255,60]\< / box> \nNo0384230 <box> [332,25,545,53]\< / box> 】; then add 、 As identifiers of table information, and as cell identifiers, the layout of the restored image can be further standardized so that the text can be accurately restored to the image layout.

[0059] In the above embodiment, by fusing the text information with the document layout information, the restoration of the image can be made more accurate, thereby making the subsequent model analysis more accurate.

[0060] S104, parsing the information extraction instruction to obtain information extraction requirement data, generating key information description and key information extraction strategy according to the information extraction requirement data, and retrieving a target large language model from a pre-built model pool according to the information extraction requirement data.

[0061] In this embodiment, generating key information description and key information extraction strategy according to the information extraction requirement data includes:

[0062] Obtaining key information interpretation and rule restrictions from the information extraction requirement data;

[0063] Generate the key information description according to the key information interpretation and the rule restrictions;

[0064] A key information grouping strategy, a standard format strategy and an early termination strategy are obtained from the information extraction requirement data as the key information extraction strategy.

[0065] Among them, the key information interpretation is used to explain the key information to assist the model to better understand the task and accurately extract the key information. For example: when extracting information from an invoice, if you want to extract the lowercase amount in the invoice, the key information interpretation can be configured as: "Extract the total amount of all goods in the invoice (lowercase), only extract the value, not the unit". If the key information to be extracted is relatively simple, you do not need to explain the interpretation. For example, for a name, you only need to fill in the name of the key information to be extracted. In addition, if the key information to be extracted is relatively complex and has many restrictions or rule processing, it can be further explained by giving examples.

[0066] The key information grouping strategy is used to limit the grouping method.

[0067] The standard format strategy is used to limit the format of the output information extraction results. For example, since most downstream tasks use json as the data format, the information extraction results can be limited to be output in json format to facilitate efficient execution of downstream tasks.

[0068] The early termination strategy is used to limit the model to stop outputting results at a reasonable position to avoid generating redundant text and causing unparseable situations.

[0069] In this embodiment, the step of extracting a target large language model from a pre-built model pool according to the information extraction requirement data includes:

[0070] Extracting required data based on the information to determine the model scale and model parameters;

[0071] Matching in the model pool according to the model scale and the model parameters;

[0072] The matched model is determined as the target large language model.

[0073] Through the above embodiments, it is possible to specifically retrieve a suitable large language model from the model pool according to the performance requirements of the model, so as to reduce the model deployment cost and unnecessary operating cost while meeting the task requirements.

[0074] In this embodiment, before extracting the target large language model from the pre-built model pool according to the information extraction requirement data, the method further includes:

[0075] Get the model size and model parameters of each large language model;

[0076] Label each large language model according to its model size and model parameters;

[0077] Each labeled large language model is stored in the model pool.

[0078] Through the above embodiments, large language models of various sizes can be integrated into a unified model pool, thereby facilitating subsequent calls and improving task execution efficiency.

[0079] S105: Generate a target prompt according to the fusion information and the key information description.

[0080] In this embodiment, generating a target prompt according to the fusion information and the key information description includes:

[0081] Get pre-configured information extraction template;

[0082] Adding the fused information to the information extraction template to obtain fused features;

[0083] Obtaining an initial prompt of the target large language model;

[0084] The fusion feature and the key information description are added to the initial prompt to obtain the target prompt.

[0085] Through the above embodiments, a variety of information can be integrated into the target prompt, thereby guiding the model to output results that better meet the requirements.

[0086] S106. Based on the target prompt and the key information extraction strategy, the target large language model is called to process the target document to obtain an information extraction result of the target document.

[0087] In this embodiment, based on the target prompt and the key information extraction strategy, calling the target large language model to process the target document to obtain the information extraction result of the target document includes:

[0088] Determining multiple element types to be extracted according to the key information extraction strategy;

[0089] Classifying the multiple element types according to element lengths to obtain multiple element groups;

[0090] According to the target prompt, the target large language model is controlled to extract information based on each element group respectively, so as to obtain a sub-information extraction result corresponding to each element group;

[0091] The sub-information extraction results corresponding to each element group are combined to obtain the information extraction result of the target document.

[0092] For example: In actual application scenarios, it is common for each type of document to have more than 20 key information elements that need to be paid attention to. In some cases, it even reaches 50 key information. For such information-intensive documents, it is not necessarily appropriate to treat all key information as a key information extraction task. Generally speaking, large language models understand short texts better than long texts, so it is possible to consider dividing multiple key information elements into multiple groups of tasks for separate extraction. It should also be noted that considering that the large model has the ability to understand contextual semantics, how to group is also an issue that needs attention, such as the present embodiment that groups according to the type of key information elements. Still taking the invoice as an example, the invoice code, invoice number and other key information elements can be grouped into one group, while the purchaser's name, purchaser's address and telephone number and other key information elements can be grouped into one group.

[0093] Through the above embodiments, the amount of information processed by the model each time can be reduced, thereby optimizing the model processing effect.

[0094] In this embodiment, the method further includes:

[0095] When calling the target large language model to process the target document, determining a termination token parameter according to the key information extraction strategy;

[0096] Controlling the target large language model to detect the termination token parameter in real time when outputting results;

[0097] When the termination token parameter is detected, the target large language model is controlled to stop outputting.

[0098] For example: In some cases, after the generative large model generates the extraction results, it will further explain and illustrate them, but the result required by the task is only the previous part of the parseable content that conforms to the format. Therefore, if the format is fixed, this embodiment sets the termination token parameter during model reasoning (that is, the termination timing, the termination token parameter is related to the data format of the final output, and is part of the format (when it is returned in the format of {key1:value1,key2:value2}), the termination parameter can be set to}, requiring the large model to terminate the output when generating}), the termination parameter can be displayed or not), so as to avoid the generation of redundant text that causes unparseable situations.

[0099] This embodiment combines a large language model to extract information, without training, and can obtain better results by using the powerful prior knowledge of the generative large model. In addition, this embodiment has generalization. Since the ability of the native model is maintained, it can still have a good effect on unseen samples of the same type or mixed noise data, while the trained model is difficult to obtain ideal results for examples outside the sample, and may even produce unreliable answers. In addition, this embodiment requires less data volume. Conventional model training requires the preparation of a large amount of diverse data to avoid model overfitting, while this embodiment only needs to extract the characteristics of key information elements on a small amount of data to achieve better results. This embodiment also has lower requirements for technicians, and only needs to complete the prompt of the input large model to obtain the result of the extraction task. This embodiment also has a certain degree of flexibility. The generative large model has a strong semantic understanding ability. For some key information elements with many restrictions, such as when the extracted key information element results need to be calculated, LayoutXLM, Donut and other models are difficult to do, and this embodiment can achieve the extraction of such key information elements, which is a major breakthrough. This embodiment also has a certain degree of versatility. All key information extraction tasks in similar scenarios can use the same set of prompt templates. It only needs to change the fusion information and key information description. There is no need to train a model in each scenario to complete downstream tasks. This embodiment also has a low time cost and is simple to operate, which can effectively save training time.

[0100] It can be seen from the above technical solutions that, on the one hand, the present application fuses the extracted text information with the document layout information obtained after layout analysis, and can perform image restoration on the results after text recognition to improve the accuracy of model recognition; on the one hand, the present application calls the target large language model from the pre-built model pool according to the information extraction requirement data, and by calling the large language model that matches the actual needs, it is possible to reduce the model deployment cost while meeting the task requirements; on the other hand, the present application generates a target prompt based on the fused information and key information description, and calls the target large language model to process the target document based on the target prompt and the key information extraction strategy, without the need to retrain the model for different tasks, so it is possible to improve generalization and versatility while efficiently performing information extraction.

[0101] The present application also provides an information extraction device, which is used to execute any embodiment of the above-mentioned information extraction method. Figure 3 , Figure 3 It is a schematic block diagram of the information extraction device 100 provided in an embodiment of the present application.

[0102] Among them, Figure 3As shown, the information extraction device 100 includes an identification unit 101, an analysis unit 102, a fusion unit 103, a parsing unit 104, a generation unit 105, and a processing unit 106.

[0103] The recognition unit 101 is used to perform text recognition on the target document in response to an information extraction instruction for the target document to obtain text information of the target document.

[0104] In this embodiment, the technical solution is described with the server as the execution subject. The user terminal (such as smart terminals such as smart phones and tablet computers) used by the user can interact with the server for data. Specifically, the server provides an information extraction platform, and the user can log in to the information extraction platform using the user terminal. The user interaction interface of the information extraction platform is displayed on the terminal interface of the user terminal, and there is at least one image upload interface in the user interaction interface. When the user selects an image as the image to be identified and uploads it to the server through the image upload interface, subsequent information extraction can be performed in the server.

[0105] In this embodiment, the target document may include, but is not limited to: invoice, bill of lading, contract, air waybill, etc.

[0106] In this embodiment, when it is detected that an image has been uploaded to the server, it can be determined that the information extraction instruction has been received.

[0107] In this embodiment, the recognition unit 101 performs text recognition on the target document, and obtains text information of the target document including:

[0108] Optical Character Recognition (OCR) is used to perform text recognition on the target document to obtain text information of the target document.

[0109] Of course, in other embodiments, statistical methods, structural methods, image processing technology, machine learning, and deep learning technology may also be used to perform text recognition on the target document, and the specific selection may be made based on the technical conditions in the actual scenario.

[0110] In this embodiment, after the text recognition is performed, text rearrangement, layout restoration, and layout analysis may also be performed to standardize the text form.

[0111] The analyzing unit 102 is used to perform layout analysis on the target document to obtain document layout information of the target document.

[0112] In this embodiment, the analyzing unit 102 performs layout analysis on the target document, and the document layout information of the target document obtained includes:

[0113] Performing region segmentation on the target document to obtain sub-regions of each content type;

[0114] The association between the sub-regions is analyzed to obtain document layout information of the target document.

[0115] Of course, in other embodiments, the layout analysis of the target document may also be performed by means of grid design, logical layout analysis, etc., and this application does not limit the layout analysis method used.

[0116] Through the above embodiment, the document layout information of the target document can be analyzed and obtained for use in subsequent document restoration.

[0117] The fusion unit 103 is used to fuse the text information with the document layout information to obtain fusion information.

[0118] In this embodiment, the fusion unit 103 fuses the text information with the document layout information to obtain fusion information including:

[0119] Obtaining the location coordinates of the text information;

[0120] Generate table information according to the document layout information;

[0121] The text information is restored to a picture layout according to the table information and the position coordinates to obtain the fusion information.

[0122] For example, for invoice X, you can use <box> 、< / box> is the location coordinate identifier, and the fusion information is obtained: [invoice X <box> [150,39,255,60]\< / box> \nNo0384230 <box> [332,25,545,53]\< / box> 】; then add 、 As identifiers of table information, and as cell identifiers, the layout of the restored image can be further standardized so that the text can be accurately restored to the image layout.

[0123] In the above embodiment, by fusing the text information with the document layout information, the restoration of the image can be made more accurate, thereby making the subsequent model analysis more accurate.

[0124] The parsing unit 104 is used to parse the information extraction instruction to obtain information extraction requirement data, generate key information description and key information extraction strategy according to the information extraction requirement data, and retrieve the target large language model from a pre-built model pool according to the information extraction requirement data.

[0125] In this embodiment, generating key information description and key information extraction strategy according to the information extraction requirement data includes:

[0126] Obtaining key information interpretation and rule restrictions from the information extraction requirement data;

[0127] Generate the key information description according to the key information interpretation and the rule restrictions;

[0128] A key information grouping strategy, a standard format strategy and an early termination strategy are obtained from the information extraction requirement data as the key information extraction strategy.

[0129] Among them, the key information interpretation is used to explain the key information to assist the model to better understand the task and accurately extract the key information. For example: when extracting information from an invoice, if you want to extract the lowercase amount in the invoice, the key information interpretation can be configured as: "Extract the total amount of all goods in the invoice (lowercase), only extract the value, not the unit". If the key information to be extracted is relatively simple, you do not need to explain the interpretation. For example, for a name, you only need to fill in the name of the key information to be extracted. In addition, if the key information to be extracted is relatively complex and has many restrictions or rule processing, it can be further explained by giving examples.

[0130] The key information grouping strategy is used to limit the grouping method.

[0131] The standard format strategy is used to limit the format of the output information extraction results. For example, since most downstream tasks use json as the data format, the information extraction results can be limited to be output in json format to facilitate efficient execution of downstream tasks.

[0132] The early termination strategy is used to limit the model to stop outputting results at a reasonable position to avoid generating redundant text and causing unparseable situations.

[0133] In this embodiment, the step of extracting a target large language model from a pre-built model pool according to the information extraction requirement data includes:

[0134] Extracting required data based on the information to determine the model scale and model parameters;

[0135] Matching in the model pool according to the model scale and the model parameters;

[0136] The matched model is determined as the target large language model.

[0137] Through the above embodiments, it is possible to specifically retrieve a suitable large language model from the model pool according to the performance requirements of the model, so as to reduce the model deployment cost and unnecessary operating cost while meeting the task requirements.

[0138] In this embodiment, before retrieving the target large language model from the pre-built model pool according to the information extraction requirement data, the model scale and model parameters of each large language model are obtained;

[0139] Label each large language model according to its model size and model parameters;

[0140] Each labeled large language model is stored in the model pool.

[0141] Through the above embodiments, large language models of various sizes can be integrated into a unified model pool, thereby facilitating subsequent calls and improving task execution efficiency.

[0142] The generating unit 105 is used to generate a target prompt according to the fusion information and the key information description.

[0143] In this embodiment, the generating unit 105 generates a target prompt according to the fusion information and the key information description, including:

[0144] Get pre-configured information extraction template;

[0145] Adding the fused information to the information extraction template to obtain fused features;

[0146] Obtaining an initial prompt of the target large language model;

[0147] The fusion feature and the key information description are added to the initial prompt to obtain the target prompt.

[0148] Through the above embodiments, a variety of information can be integrated into the target prompt, thereby guiding the model to output results that better meet the requirements.

[0149] The processing unit 106 is used to call the target large language model to process the target document based on the target prompt and the key information extraction strategy to obtain the information extraction result of the target document.

[0150] In this embodiment, the processing unit 106 calls the target large language model to process the target document based on the target prompt and the key information extraction strategy, and obtains the information extraction result of the target document including:

[0151] Determining multiple element types to be extracted according to the key information extraction strategy;

[0152] Classifying the multiple element types according to element lengths to obtain multiple element groups;

[0153] According to the target prompt, the target large language model is controlled to extract information based on each element group respectively, so as to obtain a sub-information extraction result corresponding to each element group;

[0154] The sub-information extraction results corresponding to each element group are combined to obtain the information extraction result of the target document.

[0155] For example: In actual application scenarios, it is common for each type of document to have more than 20 key information elements that need to be paid attention to. In some cases, it even reaches 50 key information. For such information-intensive documents, it is not necessarily appropriate to treat all key information as a key information extraction task. Generally speaking, large language models understand short texts better than long texts, so it is possible to consider dividing multiple key information elements into multiple groups of tasks for separate extraction. It should also be noted that considering that the large model has the ability to understand contextual semantics, how to group is also an issue that needs attention, such as the present embodiment that groups according to the type of key information elements. Still taking the invoice as an example, the invoice code, invoice number and other key information elements can be grouped into one group, while the purchaser's name, purchaser's address and telephone number and other key information elements can be grouped into one group.

[0156] Through the above embodiments, the amount of information processed by the model each time can be reduced, thereby optimizing the model processing effect.

[0157] In this embodiment, when the target large language model is called to process the target document, a termination token parameter is determined according to the key information extraction strategy;

[0158] Controlling the target large language model to detect the termination token parameter in real time when outputting results;

[0159] When the termination token parameter is detected, the target large language model is controlled to stop outputting.

[0160] For example: In some cases, after the generative large model generates the extraction results, it will further explain and illustrate them, but the result required by the task is only the previous part of the parseable content that conforms to the format. Therefore, if the format is fixed, this embodiment sets the termination token parameter during model reasoning (that is, the termination timing, the termination token parameter is related to the data format of the final output, and is part of the format (when it is returned in the format of {key1:value1,key2:value2}), the termination parameter can be set to}, requiring the large model to terminate the output when generating}), the termination parameter can be displayed or not), so as to avoid the generation of redundant text that causes unparseable situations.

[0161] This embodiment combines a large language model to extract information, without training, and can obtain better results by using the powerful prior knowledge of the generative large model. In addition, this embodiment has generalization. Since the ability of the native model is maintained, it can still have a good effect on unseen samples of the same type or mixed noise data, while the trained model is difficult to obtain ideal results for examples outside the sample, and may even produce unreliable answers. In addition, this embodiment requires less data volume. Conventional model training requires the preparation of a large amount of diverse data to avoid model overfitting, while this embodiment only needs to extract the characteristics of key information elements on a small amount of data to achieve better results. This embodiment also has lower requirements for technicians, and only needs to complete the prompt of the input large model to obtain the result of the extraction task. This embodiment also has a certain degree of flexibility. The generative large model has a strong semantic understanding ability. For some key information elements with many restrictions, such as when the extracted key information element results need to be calculated, LayoutXLM, Donut and other models are difficult to do, and this embodiment can achieve the extraction of such key information elements, which is a major breakthrough. This embodiment also has a certain degree of versatility. All key information extraction tasks in similar scenarios can use the same set of prompt templates. It only needs to change the fusion information and key information description. There is no need to train a model in each scenario to complete downstream tasks. This embodiment also has a low time cost and is simple to operate, which can effectively save training time.

[0162] It can be seen from the above technical solutions that, on the one hand, the present application fuses the extracted text information with the document layout information obtained after layout analysis, and can perform image restoration on the results after text recognition to improve the accuracy of model recognition; on the one hand, the present application calls the target large language model from the pre-built model pool according to the information extraction requirement data, and by calling the large language model that matches the actual needs, it is possible to reduce the model deployment cost while meeting the task requirements; on the other hand, the present application generates a target prompt based on the fused information and key information description, and calls the target large language model to process the target document based on the target prompt and the key information extraction strategy, without the need to retrain the model for different tasks, so it is possible to improve generalization and versatility while efficiently performing information extraction.

[0163] The above-mentioned information extraction device can be implemented in the form of a computer program. The computer program can be Figure 4 Runs on the computer device shown.

[0164] See also Figure 4 , Figure 44 is a schematic block diagram of a computer device provided in an embodiment of the present application. The computer device 400 is a server, or a server cluster. The server can be an independent server, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0165] See also Figure 4 The computer device 400 includes a processor 402 , a memory and a network interface 405 connected via a device bus 401 , wherein the memory may include a storage medium 403 and an internal memory 404 .

[0166] The storage medium 403 may store an operating system 4031 and a computer program 4032. When the computer program 4032 is executed, the processor 402 may execute the information extraction method.

[0167] The processor 402 is used to provide computing and control capabilities to support the operation of the entire computer device 400 .

[0168] The internal memory 404 provides an environment for the operation of the computer program 4032 in the storage medium 403. When the computer program 4032 is executed by the processor 402, the processor 402 can execute the information extraction method.

[0169] The network interface 405 is used for network communication, such as providing data information transmission, etc. Those skilled in the art will understand that Figure 4 The structure shown in the figure is merely a block diagram of a portion of the structure related to the solution of the present application, and does not constitute a limitation on the computer device 400 to which the solution of the present application is applied. The specific computer device 400 may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0170] The processor 402 is used to run a computer program 4032 stored in the memory to implement the information extraction method disclosed in the embodiment of the present application.

[0171] Those skilled in the art will understand that Figure 4 The embodiments of the computer device shown in the figure do not constitute a limitation on the specific composition of the computer device. In other embodiments, the computer device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently. For example, in some embodiments, the computer device may only include a memory and a processor. In such embodiments, the structure and function of the memory and the processor are the same as those of the embodiment of the present invention. Figure 4The embodiments shown are consistent and will not be described again here.

[0172] It should be understood that in the embodiment of the present application, the processor 402 may be a central processing unit (CPU), and the processor 402 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0173] In another embodiment of the present application, a computer-readable storage medium is provided. The computer-readable storage medium may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. The computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the information extraction method disclosed in the embodiment of the present application is implemented.

[0174] It should be noted that the data involved in this case were all obtained legally.

[0175] It will be clear to those skilled in the art that, for the convenience and simplicity of description, the specific working process of the above-described equipment, devices and units can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. It will be appreciated by those of ordinary skill in the art that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented with electronic hardware, computer software or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to the function. Whether these functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of this application.

[0176] In the several embodiments provided in the present application, it should be understood that the disclosed equipment, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, and units with the same function may be combined into one unit. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interfaces, devices or units, or may be electrical, mechanical or other forms of connection.

[0177] The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiments of the present application.

[0178] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0179] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a background server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a disk or an optical disk.

[0180] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. An information extraction method, characterized in that: The information extraction method comprises: In response to an information extraction instruction for a target document, performing text recognition on the target document to obtain text information of the target document; Performing layout analysis on the target document to obtain document layout information of the target document; Merging the text information with the document layout information to obtain fused information; Parsing the information extraction instruction to obtain information extraction requirement data, generating key information description and key information extraction strategy according to the information extraction requirement data, and retrieving a target large language model from a pre-built model pool according to the information extraction requirement data; Generate a target prompt according to the fusion information and the key information description; Based on the target prompt and the key information extraction strategy, the target large language model is called to process the target document to obtain the information extraction result of the target document.

2. The information extraction method according to claim 1, characterized in that: The fusing of the text information and the document layout information to obtain the fused information includes: Obtaining the location coordinates of the text information; Generate table information according to the document layout information; The text information is restored to a picture layout according to the table information and the position coordinates to obtain the fusion information.

3. The information extraction method according to claim 1, characterized in that: The generating of key information description and key information extraction strategy according to the information extraction requirement data includes: Obtaining key information interpretation and rule restrictions from the information extraction requirement data; Generate the key information description according to the key information interpretation and the rule restrictions; A key information grouping strategy, a standard format strategy and an early termination strategy are obtained from the information extraction requirement data as the key information extraction strategy.

4. The information extraction method according to claim 1, characterized in that: The extracting the target large language model from the pre-built model pool according to the information extraction requirement data comprises: Extracting required data based on the information to determine the model scale and model parameters; Matching in the model pool according to the model scale and the model parameters; The matched model is determined as the target large language model.

5. The information extraction method according to claim 1, characterized in that: Generating a target prompt according to the fusion information and the key information description includes: Get pre-configured information extraction template; Adding the fused information to the information extraction template to obtain fused features; Obtaining an initial prompt of the target large language model; The fusion feature and the key information description are added to the initial prompt to obtain the target prompt.

6. The information extraction method according to claim 1, characterized in that: The process of calling the target large language model to process the target document based on the target prompt and the key information extraction strategy to obtain the information extraction result of the target document includes: Determining multiple element types to be extracted according to the key information extraction strategy; Classifying the multiple element types according to element lengths to obtain multiple element groups; According to the target prompt, the target large language model is controlled to extract information based on each element group respectively, so as to obtain a sub-information extraction result corresponding to each element group; The sub-information extraction results corresponding to each element group are combined to obtain the information extraction result of the target document.

7. The information extraction method according to claim 1, characterized in that: The method further comprises: When calling the target large language model to process the target document, determining a termination token parameter according to the key information extraction strategy; Controlling the target large language model to detect the termination token parameter in real time when outputting results; When the termination token parameter is detected, the target large language model is controlled to stop outputting.

8. An information extraction device, characterized in that: The information extraction device comprises: A recognition unit, configured to perform text recognition on the target document in response to an information extraction instruction for the target document, and obtain text information of the target document; An analysis unit, used for performing layout analysis on the target document to obtain document layout information of the target document; A fusion unit, used for fusing the text information with the document layout information to obtain fusion information; A parsing unit, used to parse the information extraction instruction to obtain information extraction requirement data, generate key information description and key information extraction strategy according to the information extraction requirement data, and retrieve a target large language model from a pre-built model pool according to the information extraction requirement data; A generating unit, configured to generate a target prompt according to the fusion information and the key information description; A processing unit is used to call the target large language model to process the target document based on the target prompt and the key information extraction strategy to obtain the information extraction result of the target document.

9. A computer device, characterized in that: The computer device comprises: a memory storing at least one instruction; and A processor executes instructions stored in the memory to implement the information extraction method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one instruction, and the at least one instruction is executed by a processor in a computer device to implement the information extraction method according to any one of claims 1 to 7.

Images