Bid inviting and tendering document automatic information extraction method and system based on large language model

Abstract

The invention provides an automatic bidding and tendering file information extraction method and system based on a large language model, and the method is suitable for various file formats through preprocessing a bidding and tendering file; constructing a data set in the field of bidding and tendering documents, and performing fine adjustment on the BERT model based on the data set to realize rough labeling of the bidding and tendering documents; further performing fine adjustment on the generative large language model, and refining the rough label by using the context understanding capability of the generative large language model; the fine-tuned BERT model and the generative large language model are integrated into a dual-stage information extraction model, and automatic information extraction is realized, so that the accuracy and efficiency of extracting key information from bidding documents are effectively improved. In addition, the key information is output through highlight display and structuralization, so that the extraction result is visually presented to the user, and the practicability of the information is ensured.

Description

Technical Field

[0001] The present invention belongs to the technical field of natural language processing and information extraction, and in particular relates to a method and system for automatic information extraction from bidding documents based on a large language model. Background Art

[0002] With the rapid development of information technology, the document management and processing of the bidding industry has gradually transitioned from traditional paper documents to electronic ones. However, bidding documents are usually complex in structure, long in length, and contain a large number of different types of information, such as project descriptions, bidding requirements, qualification conditions, and bid evaluation criteria. For bidders, it is necessary to quickly understand and respond to these bidding requirements to ensure that their proposals meet the bidding conditions. For the tenderer, it is necessary to efficiently screen out the bidding proposals that meet the requirements from a large number of submitted documents. Therefore, how to quickly and accurately extract key information from bidding documents has become a key issue in the bidding process.

[0003] At present, extracting key information from bidding documents mainly relies on manual operations, which not only consumes a lot of time and human resources, but also easily leads to information omissions or errors, making it difficult to meet the high requirements of efficiency and accuracy in the bidding process. Therefore, in order to help the tenderer and the bidder to extract information from the bidding documents more efficiently and accurately, an automated information extraction method is urgently needed to improve work efficiency and reduce errors.

[0004] In recent years, large language models (LLMs) have made breakthrough progress in the field of natural language processing. Through pre-training of massive corpora, they have powerful semantic understanding and language generation capabilities, and can automatically process and understand various types of unstructured text data, which makes it possible to automatically extract information. However, when this technology is applied to the processing of bidding documents, there are still some limitations: first, the lack of high-quality data sets specifically for the field of bidding documents limits the performance of the model in this field; second, existing methods often fail to combine the characteristics of bidding scenarios and fail to consider the specific structure and expression of different label items in bidding documents (such as bidding requirements, bidding quotation methods, bidding agency service fees, etc.), resulting in poor performance when processing complex documents. Summary of the invention

[0005] Based on this, an embodiment of the present invention provides a method and system for automatic information extraction from bidding documents based on a large language model, aiming to solve the problems in the prior art of manual extraction of key information from bidding documents, which is time-consuming, error-prone and inefficient.

[0006] A first aspect of an embodiment of the present invention provides a method for automatically extracting information from bidding documents based on a large language model, the method comprising: Preprocess the bidding documents to convert the documents in different formats into the target text format; According to the preprocessed bidding documents, a bidding document dataset is constructed, wherein the bidding document dataset includes input text, rough labels and refined labels, wherein the input text represents the original text in the bidding document, the rough labels represent the information category identified from the input text, and the refined labels represent the specific information extracted from the input text based on the rough labels; According to the bidding document dataset, the BERT sequence labeling model is adjusted, and the adjusted BERT sequence labeling model is used to identify rough labels in the input text and output preliminary classification results; According to the bidding document dataset, a generative large language model with context understanding capability is adjusted, and the adjusted large language model is used to refine the rough labels identified by the adjusted BERT sequence labeling model to extract specific information; Integrate the adjusted BERT sequence tagging model and the adjusted large language model as the target model to extract target information; The bidding documents to be processed are input into the target model, the target information is extracted and highlighted, and the target information is saved in a structured format.

[0007] Furthermore, the step of pre-processing the bidding documents includes: Convert bidding documents into text and clean up the format and / or characters in the text; The cleaned text is divided into blocks according to the common structure of the bidding documents, file directory or chapter number.

[0008] Furthermore, the step of adjusting the BERT sequence labeling model according to the bidding document dataset includes: Select the pre-trained BERT model as the base model; Enter text X Decompose into multiple words to generate embedding vectors. The generated embedding vectors E It is word embedding , Position Embedding And snippet embed The result of addition; The bidirectional Transformer architecture is used for encoding, capturing the context information of words from both the forward and backward directions of the input sequence, and generating a context vector for each word. The encoded context vector is input to the classification layer, which performs a linear transformation on the context vector of each word and calculates the label category probability distribution of each word through the Softmax function; Constructing the loss function :

[0009] in, is the one-hot encoding of the true label, The model is i The predicted probability of each word; Loss Function After the calculation is completed, the BERT model parameters are updated through the back-propagation algorithm. The gradient obtained by back-propagation is passed to the AdamW optimizer, and the optimizer updates the BERT model parameters according to the gradient. Through iterative training, the adjusted BERT model is obtained.

[0010] Furthermore, the step of adjusting the generative large language model with context understanding capability according to the bidding document dataset includes: Select a pre-trained large language model with contextual understanding and generation capabilities as the base model and input the text X and rough labels as input; Design prompt words to guide the large language model from the input text X Extract precise detailed information from The large language model uses autoregressive generation to gradually generate the next word in the target sequence and dynamically adjusts the prediction of the next word based on the generated content; Constructing the loss function :

[0011] Where T is the length of the generated sequence, is the tth generated word or token in the generated sequence, Is entering text X and rough labels The probability of generating the next word under the condition.

[0012] Loss Function After the calculation is completed, the large language model parameters are updated through the back-propagation algorithm. The gradient obtained by back-propagation is passed to the AdamW optimizer, and the optimizer updates the large language model parameters according to the gradient.

[0013] Through iterative training, an adjusted large language model is obtained.

[0014] Furthermore, the step of inputting the bidding documents to be processed into the target model, extracting and highlighting the target information, and saving the target information in a structured format includes: Inputting the bidding documents to be processed into the target model and outputting target information; Match the target information with the bidding documents to be processed. After the match is successful, highlight the target information in the bidding documents to be processed; Save the target information in a structured format.

[0015] A second aspect of an embodiment of the present invention provides a system for automatically extracting information from bidding documents based on a large language model, which is used to implement the method for automatically extracting information from bidding documents based on a large language model provided in the first aspect, and the system includes: A preprocessing module is used to preprocess the bidding documents to convert the documents in different formats into the target text format; A data set construction module is used to construct a bidding document data set according to the preprocessed bidding documents, wherein the bidding document data set includes input text, rough labels and refined labels, wherein the input text represents the original text in the bidding document, the rough labels represent the information category identified from the input text, and the refined labels represent the specific information extracted from the input text based on the rough labels; A first adjustment module is used to adjust the BERT sequence labeling model according to the bidding document dataset, and the adjusted BERT sequence labeling model is used to identify rough labels in the input text and output a preliminary classification result; A second adjustment module is used to adjust the generative large language model with context understanding ability according to the bidding document dataset, and the adjusted large language model is used to refine the rough labels identified by the adjusted BERT sequence labeling model to extract specific information; An integration module, used to integrate the adjusted BERT sequence tagging model and the adjusted large language model into a target model for extracting target information; The input module is used to input the bidding documents to be processed into the target model, extract and highlight the target information, and save the target information in a structured format.

[0016] A third aspect of an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for automatic information extraction from bidding documents based on a large language model provided in the first aspect.

[0017] A fourth aspect of an embodiment of the present invention provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method for automatic information extraction from bidding documents based on a large language model provided in the first aspect is implemented.

[0018] The method and system for automatically extracting information from bidding documents based on a large language model provided in the embodiments of the present invention have the following beneficial effects: By preprocessing the bidding documents, it is applicable to a variety of file formats; constructing a dataset in the field of bidding documents, fine-tuning the BERT model based on the dataset, and achieving rough annotation of bidding documents; further fine-tuning the generative large language model, using its contextual understanding ability to refine the rough labels; integrating the fine-tuned BERT model and the generative large language model into a two-stage information extraction model to achieve automated information extraction, thereby effectively improving the accuracy and efficiency of extracting key information from bidding documents. In addition, by highlighting and outputting key information in a structured manner, the extracted results are intuitively presented to users, ensuring the practicality of the information. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 A flowchart of an implementation method for automatic information extraction from bidding documents based on a large language model provided in the first embodiment of the present invention; Figure 2 Construct examples for the prompt words; Figure 3 This is a schematic diagram of the results of information extraction from bidding documents; Figure 4 A structural block diagram of a system for automatically extracting information from bidding documents based on a large language model provided in Embodiment 2 of the present invention; Figure 5 This is a structural block diagram of an electronic device provided in Embodiment 3 of the present invention. DETAILED DESCRIPTION

[0020] In order to more clearly illustrate the technical solutions of the embodiments of this specification, the following is a brief introduction to the drawings required for the description of the embodiments. Obviously, the drawings described below are only some examples or embodiments of this specification. For ordinary technicians in this field, this specification can also be applied to other similar scenarios based on these drawings without creative work. Unless it is obvious from the language environment or otherwise explained, the same reference numerals in the figures represent the same structure or operation.

[0021] The method for automatic information extraction from bidding documents based on a large language model provided in this embodiment can be executed in an intelligent terminal, a computer terminal, a network device, a chip, a chip module or a similar computing device. Mentioning "embodiment" in this application means that the specific features, structures or characteristics described in conjunction with the embodiment may be included in at least one embodiment of the present application. The appearance of this phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described in this application can be combined with other embodiments.

[0022] It should be understood that the "system", "device", "unit" and / or "module" used herein are a method for distinguishing different components, elements, parts, portions or assemblies at different levels. However, if other words can achieve the same purpose, the words can be replaced by other expressions.

[0023] As shown in this specification, unless the context clearly indicates an exception, the words "a", "an", "a kind" and / or "the" do not refer to the singular, but also include the plural. Generally speaking, the terms "include" and "comprise" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.

[0024] Embodiment 1 According to an embodiment of the present invention, a method for automatic information extraction from bidding documents based on a large language model is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0025] In this embodiment 1, a method for automatically extracting information from bidding documents based on a large language model is provided. Figure 1 , Figure 1 A flowchart of an implementation of a method for automatic information extraction from bidding documents based on a large language model provided in Embodiment 1 of the present invention is shown, which specifically includes steps S01 to S06.

[0026] Step S01, pre-processing the bidding documents to convert the documents in different formats into a target text format.

[0027] Understandably, bidding documents are often in complex formats, including tables, special characters, page breaks, etc. If these contents are directly input into the model, it may cause the model to have difficulty understanding or even extract errors. Through preprocessing, these complex formats can be standardized so that the model can more accurately identify key information. Specifically, the steps for preprocessing bidding documents include: Convert the bidding documents into text and clean the format and / or characters in the text. It should be noted that the bidding documents should first be unified into text format, that is, according to the file type, different processing methods are selected to convert them into plain text. If it is a Word file, Python tools (such as python-docx) can be used to convert it into plain text format; if it is a PDF file, a PDF parsing tool (such as PyPDF2) is needed to extract the text content. For PDF files containing complex content such as tables, images, non-standard fonts, etc., OCR (optical character recognition) technology is needed to process the text information of the scanned PDF. Then the format and / or characters in the bidding documents are processed, specifically, including deleting invisible characters, processing special symbols, and converting tables into structured text. For example, delete invisible characters in headers and footers, and standardize line breaks and blanks. If there is a table in the text, it needs to be converted into a structured text format suitable for processing to ensure the readability and consistency of the text content; The cleaned text is divided into blocks according to the common structure of the bidding documents, the file directory or the chapter number. It can be understood that based on the common structure of the bidding documents (such as the tender notice, bidding instructions, bid evaluation method, etc.), or according to the directory or chapter number in the document, the text is divided into multiple logical paragraphs to facilitate the subsequent information extraction.

[0028] Step S02, constructing a bidding document dataset based on the preprocessed bidding documents, wherein the bidding document dataset includes input text, rough labels, and refined labels.

[0029] Due to the lack of Chinese annotated datasets for bidding documents, it is impossible to provide well-annotated training data for large models. In order to clarify which information in bidding documents is critical, it is necessary to first build a dataset of bidding documents. The purpose of building the dataset of bidding documents is to clarify the key information categories in bidding documents so that the model can accurately identify and locate the target information based on predefined labels during training, thereby generating output results with application value in practical applications.

[0030] Construct a bidding document dataset. Each data record includes three fields: input text X , rough label , refine the label . The input text X The original content of the bidding document, type is string; rough label The main information category identified from the input text is a predefined classification label, such as "the time of obtaining the bidding documents", "the project construction unit", "the bidding quotation method", etc.; refine the label It is the specific information further extracted based on the rough label. It is in the form of key-value pairs and includes precise entity information such as time interval, unit name, amount, etc. Therefore, each data instance can be represented as ( X， ， ) triples.

[0031] To further understand the data format of the constructed bidding document dataset, the following example shows the "Time of Obtaining Bidding Documents": < Input text X "Anyone who is interested in participating in the bidding, please log in to the tenderer's bidding transaction platform information system from 23:00:00 on June 18, 2024 to 17:00:00 on June 25, 2024 to obtain the bidding documents." Rough Label :Time to obtain the bidding documents Refine label : {“Start time for obtaining bidding documents”: “23:00:00, June 18, 2024”; “End time for obtaining bidding documents”: “17:00:00, June 25, 2024”}> In this example, for the input text, the rough label For "Time to Obtain Tender Documents", in the detailed label The detailed information of "the start time for obtaining the tender documents" and "the end time for obtaining the tender documents" are listed in the form of key-value pairs.

[0032] Step S03, adjusting the BERT sequence labeling model according to the bidding document dataset, and the adjusted BERT sequence labeling model is used to identify rough labels in the input text and output preliminary classification results.

[0033] It should be noted that the pre-trained BERT model is first selected as the basis. The model has been pre-trained on a large-scale corpus, and the task is usually word prediction or sentence prediction. However, in the fine-tuning stage, the task needs to be converted into a sequence labeling task, that is, a corresponding label (i.e., a rough label) is assigned to each token of the input text. At this time, the parameters of the BERT model will be fine-tuned to adapt to the task requirements.

[0034] Then generate the embedding vector and input the text X Decompose into multiple tokens to generate embedding vectors. The generated embedding vectors E It consists of three parts: word embedding , Position Embedding And snippet embed , the formula is as follows:

[0035] Among them, word embedding The input text is segmented according to BERT's WordPiece tokenizer, and a word embedding vector is assigned to each token. In this process, special tags [CLS] and [SEP] are automatically added before and after the text to indicate the start and end of the text. The sequence of the text after tokenization is , then the corresponding word embedding is ,in, for The word embedding vector of .

[0036] Positional Embedding It generates position embedding based on the total token sequence of the text (including [CLS] and [SEP]) according to its relative position in the text. For a token sequence of length n, its position embedding is ,in, Indicates i The position of the token is embedded.

[0037] Snippet embedding It is used to distinguish different paragraphs or sentences in the input text. If the input text contains only one paragraph, the fragment embeddings of all tokens are the same. If the input contains multiple sentences or paragraphs, different sentences or paragraphs have different fragment embeddings, that is, ,in It is i A fragment of a token is embedded.

[0038] Furthermore, encoding layer encoding: BERT uses a bidirectional Transformer architecture to encode text, which can capture the contextual information of words from both the forward and backward directions of the input sequence. After L layers of Transformer encoding, each token The corresponding context vector is generated , here It represents the last layer (i.e. l layer).

[0039] Furthermore, the classification layer outputs: The context vector output by the encoding layer is passed to the classification layer, which generates a context vector for each token. Perform linear transformation and calculate the probability distribution of label categories through the Softmax function. The specific process is as follows:

[0040] in, and are the learnable parameters of the classification layer, Token The predicted label of .

[0041] In order to measure the gap between the model prediction results and the true labels, the cross entropy loss function is used. The loss function of the BERT model as follows:

[0042] in, is the one-hot encoding of the true label, The model is i The predicted probability of a word.

[0043] During the model training process, the optimizer AdamW is used to update the parameters to ensure that each iteration can effectively reduce the value of the loss function. During the training process, the model will iterate for multiple epochs and update the model parameters in each iteration. To prevent overfitting, an early stopping mechanism is set, that is, when the loss on the validation set no longer decreases, the training is automatically stopped.

[0044] Through the above fine-tuning steps, the obtained BERT model can accurately identify rough labels from bidding documents, providing a good foundation for the subsequent refined label extraction based on the large language model.

[0045] Step S04: According to the bidding document dataset, the generative large language model with context understanding capability is adjusted, and the adjusted large language model is used to refine the rough labels identified by the adjusted BERT sequence labeling model to extract specific information.

[0046] It should be noted that, based on the pre-established bidding document dataset, the generative large language model with context understanding capability is fine-tuned, wherein the input of the large language model is the rough label and the corresponding text fragment recognized by the BERT model obtained in step S03, and the output is the refined label.

[0047] Specifically, first select a pre-trained large language model with context understanding and generation capabilities as the basic model, and then use the annotated text and rough labels as input, where the large language model includes but is not limited to GPT-3, PaLM, LLaMA, etc. At the same time, design a suitable prompt word to guide the model to extract precise and detailed information from the input. The design of the prompt word needs to clearly specify the task requirements so that the model can understand the target information to be extracted from the text. Exemplarily, the format of the prompt word is:

[0048] The content of “…” is set according to the specific type of the rough label. and input text X Afterwards, based on the instructions of the prompt word, the autoregressive generation method is used to gradually generate the next word in the target sequence, and the prediction of the next word is dynamically adjusted according to the generated content, and finally the corresponding refined entity information is generated. Figure 2 Construct examples for the prompt words.

[0049] Among them, the generative large language model optimizes the difference between the generated refined labels and the true labels through cross entropy loss. Its loss function The formula is as follows:

[0050] Where T is the length of the generated sequence, is the tth generated word or token in the generated sequence, Is entering text X and rough labels The probability of generating the next word under the condition.

[0051] Loss Function After the calculation is completed, the model updates its parameters through the back-propagation algorithm. The back-propagation process calculates the partial derivative of each parameter with respect to the loss layer by layer through the chain rule, and propagates the error to each layer in turn. The gradients obtained by back-propagation are passed to the AdamW optimizer, which updates the model parameters based on these gradients. Through multiple iterations of training, the model is able to generate accurate refined entities and understand information such as time and unit names in different contexts.

[0052] Through fine-tuning of the above steps, the generative large language model can effectively extract and refine the rough labels identified by BERT, providing accurate generation capabilities for extracting target entity information from bidding documents.

[0053] Step S05: Integrate the adjusted BERT sequence labeling model and the adjusted large language model into a target model for extracting target information.

[0054] It is understandable that the fine-tuned BERT model and the large language model are integrated to construct a two-stage information extraction model, which is also the target model. In the two-stage information extraction model, in the first stage, the fine-tuned BERT sequence labeling model is used to identify the rough labels in the bidding documents and identify the approximate categories of various key information; in the second stage, the fine-tuned generative large language model is used to further refine the output of the BERT model and extract specific entity information based on the context and rough labels. This two-stage process can more accurately identify and extract key information in bidding documents.

[0055] Step S06, inputting the bidding documents to be processed into the target model, extracting and highlighting the target information, and saving the target information in a structured format.

[0056] It should be noted that the user inputs the target bidding document into the integrated two-stage information extraction model to automatically complete the information extraction task. After the reasoning is completed, the model can not only output the extracted key information, but also highlight this information in the original document or save it in a structured format to facilitate subsequent data processing and analysis.

[0057] Specifically, the extracted refined tag information is matched in the original document. According to the needs of the user interface, the extracted information can be highlighted in different colors, bold fonts or underlined. For PDF or Word files, annotations can be automatically inserted through the program to ensure that users can quickly find key information. At the same time, the structured information is saved in an exportable format, such as Excel, CSV files, to facilitate subsequent data analysis and use.

[0058] Figure 3 This is a schematic diagram of the results of information extraction from bidding documents. The figure shows that after the initial bidding documents are processed, the key information is highlighted, and the extracted tags and related content can also be saved as a file in table form.

[0059] In summary, the automatic information extraction method for bidding documents based on a large language model in the above embodiments of the present invention is applicable to various file formats by preprocessing the bidding documents; constructing a data set in the field of bidding documents, fine-tuning the BERT model based on the data set, and realizing rough labeling of bidding documents; further fine-tuning the generative large language model, and using its context understanding ability to refine the rough labels; integrating the fine-tuned BERT model and the generative large language model into a two-stage information extraction model to realize automated information extraction, thereby effectively improving the accuracy and efficiency of extracting key information from bidding documents. In addition, by highlighting and outputting key information in a structured manner, the extraction results are intuitively presented to the user, ensuring the practicality of the information.

[0060] Embodiment 2 See also Figure 4 , Figure 4 1 is a structural block diagram of a system 200 for automatically extracting information from bidding documents based on a large language model provided in the second embodiment of the present invention. The system 200 for automatically extracting information from bidding documents based on a large language model comprises: a preprocessing module 21, a data set construction module 22, a first adjustment module 23, a second adjustment module 24, an integration module 25 and an input module 26, wherein: A preprocessing module 21 is used to preprocess the bidding documents to convert the documents in different formats into a target text format; The data set construction module 22 is used to construct a bidding document data set according to the preprocessed bidding document, wherein the bidding document data set includes input text, rough labels and refined labels, wherein the input text represents the original text in the bidding document, the rough labels represent the information category identified from the input text, and the refined labels represent the specific information extracted from the input text based on the rough labels; A first adjustment module 23 is used to adjust the BERT sequence labeling model according to the bidding document dataset, and the adjusted BERT sequence labeling model is used to identify rough labels in the input text and output a preliminary classification result. Specifically, a pre-trained BERT model is selected as a basic model; Enter text X Decompose into multiple words to generate embedding vectors. The generated embedding vectors E It is word embedding , Position Embedding And snippet embed The result of addition; The bidirectional Transformer architecture is used for encoding, capturing the context information of words from both the forward and backward directions of the input sequence, and generating a context vector for each word. The encoded context vector is input to the classification layer, which performs a linear transformation on the context vector of each word and calculates the label category probability distribution of each word through the Softmax function; Constructing the loss function :

[0061] in, is the one-hot encoding of the true label, The model is i The predicted probability of each word; Loss Function After the calculation is completed, the BERT model parameters are updated through the back-propagation algorithm. The gradient obtained by back-propagation is passed to the AdamW optimizer, and the optimizer updates the BERT model parameters according to the gradient. Through iterative training, the adjusted BERT model is obtained; The second adjustment module 24 is used to adjust the generative large language model with context understanding ability according to the bidding document dataset. The adjusted large language model is used to refine the rough labels identified by the adjusted BERT sequence labeling model and extract specific information. Specifically, a pre-trained large language model with context understanding and generative ability is selected as the basic model to convert the input text X and rough labels as input; Design prompt words to guide the large language model from the input text X Extract precise detailed information from The large language model uses autoregressive generation to gradually generate the next word in the target sequence and dynamically adjusts the prediction of the next word based on the generated content; Constructing the loss function :

[0062] Where T is the length of the generated sequence, is the tth generated word or token in the generated sequence, Is entering text X and rough labels The probability of generating the next word under the condition.

[0063] Loss Function After the calculation is completed, the large language model parameters are updated through the back-propagation algorithm. The gradient obtained by back-propagation is passed to the AdamW optimizer, and the optimizer updates the large language model parameters according to the gradient.

[0064] Through iterative training, an adjusted large language model is obtained; An integration module 25, used to integrate the adjusted BERT sequence tagging model and the adjusted large language model into a target model for extracting target information; The input module 26 is used to input the bidding documents to be processed into the target model, extract and highlight the target information, and save the target information in a structured format.

[0065] Further, in some optional embodiments of the present invention, the preprocessing module 21 includes: A cleaning unit, used for converting the bidding documents into text and cleaning the format and / or characters in the text; The block division unit is used to divide the cleaned text into blocks, wherein the blocks are divided according to the common structure of the bidding documents, the file directory or the chapter number.

[0066] Further, in some optional embodiments of the present invention, the input module 26 includes: An input unit, used to input the bidding documents to be processed into the target model and output target information; A matching unit, used to match the target information with the bidding document to be processed, and after a successful match, highlight the target information in the bidding document to be processed; The saving unit is used to save the target information in a structured format.

[0067] Embodiment 3 Another aspect of the present invention provides an electronic device, see Figure 5 , shown is an electronic device in Embodiment 3 of the present invention, including a memory 20, a processor 10, and a computer program 30 stored in the memory and executable on the processor. When the processor 10 executes the computer program 30, the automatic information extraction method for bidding documents based on a large language model as described above is implemented.

[0068] In some embodiments, the processor 10 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor or other data processing chip, used to run program codes or process data stored in the memory 20, such as executing access restriction programs.

[0069] Among them, the memory 20 includes at least one type of readable storage medium, and the readable storage medium includes a flash memory, a hard disk, a multimedia card, a card-type memory (for example, an SD or DX memory, etc.), a magnetic memory, a magnetic disk, an optical disk, etc. In some embodiments, the memory 20 can be an internal storage unit of an electronic device, such as a hard disk of the electronic device. In other embodiments, the memory 20 can also be an external storage device of an electronic device, such as a plug-in hard disk equipped on the electronic device, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (FlashCard), etc. Further, the memory 20 can also include both an internal storage unit of the electronic device and an external storage device. The memory 20 can not only be used to store application software and various types of data of the electronic device, but also can be used to temporarily store data that has been output or is to be output.

[0070] It should be pointed out that Figure 5 The structure shown does not constitute a limitation on the electronic device. In other embodiments, the electronic device may include fewer or more components than those shown in the figure, or combine certain components, or arrange the components differently.

[0071] An embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-mentioned method for automatic information extraction from bidding documents based on a large language model.

[0072] Those skilled in the art will appreciate that the logic and / or steps represented in the flowchart or otherwise described herein, for example, may be considered as an ordered list of executable instructions for implementing logical functions, and may be specifically implemented in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For purposes of this specification, "computer-readable medium" may be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0073] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0074] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or a combination thereof: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0075] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0076] The above embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present invention. It should be pointed out that, for a person of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the present invention patent shall be subject to the attached claims.

Claims

1. A method for automatic information extraction from bidding documents based on a large language model, characterized in that: The method comprises: Preprocess the bidding documents to convert the documents in different formats into the target text format; According to the preprocessed bidding documents, a bidding document dataset is constructed, wherein the bidding document dataset includes input text, rough labels and refined labels, wherein the input text represents the original text in the bidding document, the rough labels represent the information category identified from the input text, and the refined labels represent the specific information extracted from the input text based on the rough labels; According to the bidding document dataset, the BERT sequence labeling model is adjusted, and the adjusted BERT sequence labeling model is used to identify rough labels in the input text and output preliminary classification results; According to the bidding document dataset, a generative large language model with context understanding capability is adjusted, and the adjusted large language model is used to refine the rough labels identified by the adjusted BERT sequence labeling model to extract specific information; Integrate the adjusted BERT sequence tagging model and the adjusted large language model as the target model to extract target information; The bidding documents to be processed are input into the target model, the target information is extracted and highlighted, and the target information is saved in a structured format.

2. The method for automatic information extraction from bidding documents based on a large language model according to claim 1 is characterized in that: The step of pre-processing the bidding documents comprises: Convert bidding documents into text and clean up the format and / or characters in the text; The cleaned text is divided into blocks according to the common structure of the bidding documents, file directory or chapter number.

3. The method for automatic information extraction from bidding documents based on a large language model according to claim 2 is characterized in that: The step of adjusting the BERT sequence labeling model according to the bidding document dataset includes: Select the pre-trained BERT model as the base model; Enter text X Decompose into multiple words to generate embedding vectors. The generated embedding vectors E It is word embedding , Position Embedding And snippet embed The result of addition; The bidirectional Transformer architecture is used for encoding, capturing the context information of words from both the forward and backward directions of the input sequence, and generating a context vector for each word. The encoded context vector is input to the classification layer, which performs a linear transformation on the context vector of each word and calculates the label category probability distribution of each word through the Softmax function; Constructing the loss function : in, is the one-hot encoding of the true label, The model is i The predicted probability of each word; Loss Function After the calculation is completed, the BERT model parameters are updated through the back-propagation algorithm. The gradient obtained by back-propagation is passed to the AdamW optimizer, and the optimizer updates the BERT model parameters according to the gradient. Through iterative training, the adjusted BERT model is obtained.

4. The method for automatic information extraction from bidding documents based on a large language model according to claim 3 is characterized in that: The step of adjusting the generative large language model with context understanding capability according to the bidding document dataset comprises: Select a pre-trained large language model with contextual understanding and generation capabilities as the base model and input the text X and rough labels as input; Design prompt words to guide the large language model from the input text X Extract precise detailed information from The large language model uses autoregressive generation to gradually generate the next word in the target sequence and dynamically adjusts the prediction of the next word based on the generated content; Constructing the loss function : Where T is the length of the generated sequence, is the tth generated word or token in the generated sequence, Is entering text X and rough labels The probability of generating the next word under the condition; Loss Function After the calculation is completed, the back propagation algorithm is used to update the parameters of the large language model. The gradient obtained by back propagation is passed to the AdamW optimizer, and the optimizer updates the parameters of the large language model according to the gradient. Through iterative training, an adjusted large language model is obtained.

5. The method for automatic information extraction from bidding documents based on a large language model according to claim 4 is characterized in that: The step of inputting the bidding documents to be processed into the target model, extracting and highlighting the target information, and saving the target information in a structured format comprises: Inputting the bidding documents to be processed into the target model and outputting target information; Match the target information with the bidding documents to be processed. After the match is successful, highlight the target information in the bidding documents to be processed; Save the target information in a structured format.

6. An automatic information extraction system for bidding documents based on a large language model, characterized in that: The system is used to implement the method for automatically extracting information from bidding documents based on a large language model as described in any one of claims 1 to 5, the system comprising: A preprocessing module is used to preprocess the bidding documents to convert the documents in different formats into the target text format; A data set construction module is used to construct a bidding document data set according to the preprocessed bidding documents, wherein the bidding document data set includes input text, rough labels and refined labels, wherein the input text represents the original text in the bidding document, the rough labels represent the information category identified from the input text, and the refined labels represent the specific information extracted from the input text based on the rough labels; A first adjustment module is used to adjust the BERT sequence labeling model according to the bidding document dataset, and the adjusted BERT sequence labeling model is used to identify rough labels in the input text and output a preliminary classification result; A second adjustment module is used to adjust the generative large language model with context understanding ability according to the bidding document dataset, and the adjusted large language model is used to refine the rough labels identified by the adjusted BERT sequence labeling model to extract specific information; An integration module, used to integrate the adjusted BERT sequence tagging model and the adjusted large language model into a target model for extracting target information; The input module is used to input the bidding documents to be processed into the target model, extract and highlight the target information, and save the target information in a structured format.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method for automatic information extraction from bidding documents based on a large language model as described in any one of claims 1 to 5 is implemented.

8. An electronic device, characterized in that: The invention comprises a memory, a processor and a computer program stored in the memory and executable on the processor. When the processor executes the program, the method for automatic information extraction from bidding documents based on a large language model as claimed in any one of claims 1 to 5 is implemented.

Images