Bidding document information extraction method and system based on large language model

Abstract

The present invention discloses a bidding document information extraction method and system based on a large language model, which belongs to the field of information processing technology, including: collecting multiple file samples in the bidding document field, converting the file samples into standard samples based on the file format, combining all the standard samples to generate a standard sample set; constructing a large language model, extracting the first label contained in the standard sample, calculating the importance value corresponding to the first label, adjusting the parameters of the large language model based on the evaluation value, and generating a standard model; the standard model sequentially extracts the label information of the bidding document based on the importance value corresponding to the first label, and combines the missing label with the associated information to generate supplementary information; summarizing the supplementary information and the label information, generating extracted information, analyzing the extracted information based on the bidding theme rule, and outputting the strategy suggestion corresponding to the bidding document. The present invention can improve the accuracy and completeness of the bidding document information extraction.

Description

Technical Field

[0001] The present invention belongs to the field of information processing technology, and in particular relates to a bidding document information extraction method and system based on a large language model. Background Art

[0002] Bidding documents usually contain a large amount of structured and unstructured data. Traditional methods for extracting information from bidding documents mainly rely on manual reading and understanding. This method is not only time-consuming and labor-intensive, but also easily affected by human factors, resulting in inaccurate and incomplete information extraction. With the development of deep learning technology, especially the emergence of large language models (LLMs), progress has been made in the field of natural language processing (NLP). These models can learn rich language representations by pre-training on large-scale text data, thereby achieving significant performance improvements on a variety of NLP tasks. However, although large language models have shown great potential in text understanding, it is still challenging to directly apply them to information extraction from bidding documents. The professionalism and complexity of bidding documents require that the model not only have strong language understanding capabilities, but also have a certain grasp of knowledge in specific fields.

[0003] Similar prior art includes a Chinese patent application with publication number CN117764058A, which discloses a method, device, equipment and medium for extracting fields from bidding documents, and relates to the field of file data processing technology. The method for extracting fields from bidding documents includes: collecting multiple historical bidding documents, taking all fields of preset information categories in each historical bidding document as a training sample, obtaining a training sample set, and preprocessing the training sample set to obtain a fine-tuning sample set; using the training sample set to train a pre-established deep learning model to obtain an initial field extraction model, and using the fine-tuning sample set to tune the initial field extraction model to obtain a target field extraction model; extracting fields from the target bidding document based on the target field extraction model, taking all fields of preset information categories in the extracted target bidding document as target fields, and obtaining all target fields in the target bidding document. This embodiment of the invention can improve the efficiency of field extraction from bidding documents. There is also a Chinese patent application with publication number CN118606425A, which discloses a method for extracting information from bidding documents based on a large language model, aiming to solve the problems of inaccuracy, inefficiency and insufficient automation of data processing in the prior art. First, the method includes widely collecting bidding documents from different industries and fields to ensure the comprehensiveness and diversity of the data set, and then performing thorough data preprocessing to improve data quality and availability. Subsequently, by implementing data enhancement technology, the scale and diversity of the data set are expanded, and meticulous data labeling is performed to ensure accurate extraction of information. In addition, the method also involves designing optimized prompt words to guide the model to more accurately identify and extract key information. Finally, a suitable pre-trained model is selected and fine-tuned using the LoRA method, and it is deployed in actual application scenarios to provide a user-friendly interface. This method not only improves the accuracy and efficiency of information extraction from bidding documents, but also enhances the automation level of the entire processing flow.

[0004] However, the above-mentioned prior art extracts target fields according to preset information categories, which makes the target field extraction model limited. It also requires data annotation to ensure accurate extraction of information, which increases the complexity of information extraction from bidding documents. Summary of the invention

[0005] In response to the above-mentioned technical problems, the present invention provides a method and system for extracting bidding document information based on a large language model to solve the problems in the prior art.

[0006] The present invention proposes a method for extracting information from bidding documents based on a large language model, comprising: S1: Collect multiple file samples in the bidding document field, obtain the file formats of the file samples, convert the file samples into standard samples based on the file formats, and combine all the standard samples to generate a standard sample set; S2: constructing a large language model, wherein the large language model extracts a first label contained in the standard sample based on the standard sample set, calculates an importance value corresponding to the first label based on the standard sample set, divides all the first labels into first-category labels, second-category labels, and third-category labels based on the importance value, respectively extracts first-category character information, second-category character information, and third-category character information corresponding to the first-category labels, the second-category labels, and the third-category labels in any of the standard samples, evaluates the first-category character information, the second-category character information, and the third-category character information, and outputs an evaluation value, and adjusts parameters of the large language model based on the evaluation value to generate a standard model; S3: Preprocessing the bidding document and inputting it into the standard model, extracting the label information of the bidding document in sequence based on the importance value corresponding to the first label, marking the missing label of the bidding document if there is missing information, judging whether the missing label has associated information based on the first type of character information and the second type of character information, and if so, combining the missing label with the associated information to generate supplementary information; S4: Summarize the supplementary information and the label information based on the label category, generate extracted information, set bidding subject rules, analyze the extracted information based on the bidding subject rules, and output the strategy recommendations corresponding to the bidding document.

[0007] Furthermore, converting the file sample into a standard sample based on the file format comprises the following steps: A text extraction method is set based on the file format, content information of the file sample is extracted based on the text extraction method, and the text position of the content information is marked, a standard format is set, and the content information and the text position are converted based on the standard format to generate the standard sample.

[0008] Furthermore, S2 further includes the following steps: Setting requirement information based on the bid document field, inputting the requirement information and the standard sample set into the large language model, and generating a training script; The large language model identifies the standard sample set based on the training script, retrieves the representation information type in any standard sample in the standard sample set based on the demand information, and extracts a predetermined keyword corresponding to the representation information type; Calculating the semantic similarity between any two predetermined keywords, if the semantic similarity is greater than a first preset value, semantically extending the two predetermined keywords to generate an extended keyword, otherwise, retaining the two predetermined keywords, and setting the extended keyword and the two predetermined keywords as the first label; The large language model generates an extension instruction based on the first label, and extracts character information corresponding to the first label in any of the standard samples based on the extension instruction, and based on the label category corresponding to the first label, sets the character information belonging to the first category of labels as the first category of character information, sets the character information belonging to the second category of labels as the second category of character information, and sets the character information belonging to the third category of character labels as the third category of character information.

[0009] Furthermore, the dividing all the first tags into first-category tags, second-category tags, and third-category tags based on the importance values ​​comprises the following steps: The number of the standard samples in the standard sample set is set as a first value, the number of the standard samples with the first label in the standard sample set is set as a second value, and the ratio of the second value to the first value is set as the importance value corresponding to the first label; A division threshold interval is set based on the importance value, and the first tag whose importance value is greater than the threshold interval is classified as the first category tag, the first tag whose importance value belongs to the threshold interval is classified as the second category tag, and the first tag whose importance value is less than the threshold interval is classified as the third category tag.

[0010] Further, the evaluation value is output based on the following steps: Respectively extracting preset proportions of the first category of character information, the second category of character information, and the third category of character information and setting them as a test information set, marking a correct label corresponding to the test information set in the standard sample set, extracting the first label corresponding to the test information set and setting it as an extracted label, obtaining a proportion distribution of the first category of labels, the second category of labels, and the third category of labels in the extracted labels, and setting a weight coefficient based on the proportion distribution; A confusion matrix is ​​constructed based on the correct labels and the extracted labels, the accuracy of the classification corresponding to any test information in the test information set is output based on the confusion matrix, and the product of the accuracy and the weight coefficient is accumulated to generate the evaluation value.

[0011] Furthermore, the generating of the standard model comprises: If the evaluation value is less than a second preset value, the difference between the evaluation value and the second preset value is calculated, and the learning rate corresponding to the parameter in the large language model is adjusted based on the difference. This step is repeated until the evaluation value is greater than or equal to the second preset value, and the adjusted large language model is set as the standard model. Otherwise, the trained large language model is set as the standard model.

[0012] Furthermore, the generating of the extraction information comprises the following steps: The standard model extracts file information from the bidding document based on the first label. If the file information corresponds to the first label one-to-one, it is determined that the bidding document does not contain the missing information, and the file information is set as the label information; otherwise, the first label that does not contain the file information is set as the missing label. If the missing tag belongs to the first category of tags or the second category of tags, multiple entity information is extracted from the first category of character information and the second category of character information. If the entity information exists in the bidding document, the first category of character information or the second category of character information is semantically expanded based on the entity information to generate the supplementary information. Otherwise, the missing tag is removed. The first tags are arranged in sequence based on the importance values ​​to generate the tag category, and the supplementary information and the tag information are combined in sequence based on the tag category to generate the extracted information.

[0013] Furthermore, the outputting of the strategy suggestion corresponding to the bidding document comprises the following steps: Extracting a business set in the first tag based on the attribute type of the bidding document, creating a business structure based on the business set, and setting the bidding subject rule based on the business structure; The business information included in the business set is extracted from the extracted information, the business information is compared and verified with the bidding subject rules, and the verification result is output, all the verification results are combined to generate an analysis report, and the analysis report is set as the policy recommendation.

[0014] The present invention also provides a bidding document information extraction system based on a large language model, which is used to implement the above-mentioned bidding document information extraction method based on a large language model. The system mainly includes: A sample collection module collects multiple file samples in the bidding document field, obtains the file format of the file samples, converts the file samples into standard samples based on the file format, and combines all the standard samples to generate a standard sample set; A model generation module is configured to construct a large language model, wherein the large language model extracts a first label contained in the standard sample based on the standard sample set, calculates an importance value corresponding to the first label based on the standard sample set, divides all the first labels into first-category labels, second-category labels, and third-category labels based on the importance value, respectively extracts first-category character information, second-category character information, and third-category character information corresponding to the first-category labels, the second-category labels, and the third-category labels in any of the standard samples, evaluates the first-category character information, the second-category character information, and the third-category character information, and outputs an evaluation value, and adjusts parameters of the large language model based on the evaluation value to generate a standard model; an information extraction module, which pre-processes the bidding document and then inputs it into the standard model, extracts the label information of the bidding document in sequence based on the importance value corresponding to the first label, marks the missing label of the bidding document if there is missing information, determines whether there is associated information for the missing label based on the first type of character information and the second type of character information, and if so, combines the missing label with the associated information to generate supplementary information; The document analysis module aggregates the supplementary information and the label information based on label categories, generates extracted information, sets bidding subject rules, analyzes the extracted information based on the bidding subject rules, and outputs strategy recommendations corresponding to the bidding document.

[0015] Compared with the prior art, the beneficial effects of the present invention are at least as follows: The present invention first pre-processes the file samples based on the file format to generate standard samples, which can improve the training speed of the large language model using the standard sample set. Then, by setting the demand information and the training script, the large language model automatically extracts multiple first tags, and constructs an extended instruction through the first tag to extract the corresponding character information, so that the large language model can extract information more accurately. Finally, the standard model is generated by training the large language model, and the bidding documents are analyzed to generate extraction information and strategy suggestions, which can improve the accuracy and completeness of the information extraction of the bidding documents.

[0016] The present invention also outputs the evaluation value of the large language model through the label category of the first label, and adjusts the learning rate of the large language model according to the evaluation value, so as to improve the accuracy of the standard model. The present invention also generates supplementary information by retrieving related information to supplement the missing information of the bidding document, so as to improve the integrity of the bidding document. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 A flowchart of the steps of a bidding document information extraction method based on a large language model of the present invention; Figure 2 A schematic diagram of constructing a confusion matrix in the present invention; Figure 3 This is a structural diagram of a bidding document information extraction system based on a large language model according to the present invention. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0019] It is understood that the terms "first", "second", etc. used in this application may be used herein to describe various elements, but unless otherwise specified, these elements are not limited by these terms. These terms are only used to distinguish a first element from another element. For example, without departing from the scope of this application, a first xx script may be referred to as a second xx script, and similarly, a second xx script may be referred to as a first xx script.

[0020] like Figure 1 As shown, a method for extracting information from bidding documents based on a large language model includes: S1: Collect multiple file samples in the bidding document field, obtain the file format of the file samples, convert the file samples into standard samples based on the file format, and combine all standard samples to generate a standard sample set.

[0021] Specifically, in this embodiment, the bid document field refers to a series of document management fields related to the bidding and tendering process, and the file sample refers to the historical formal documents in the bidding and tendering process, for example, the bid document submitted in the past. The file format refers to the file format type corresponding to the file sample, including but not limited to PDF, Word documents and Excel tables. The standard sample refers to the sample information generated after preprocessing the file sample, and the standard sample set refers to the set of all standard samples.

[0022] S2: Construct a large language model. The large language model extracts the first label contained in the standard sample based on the standard sample set, calculates the importance value corresponding to the first label based on the standard sample set, divides all the first labels into first-category labels, second-category labels and third-category labels based on the importance value, respectively extracts the first-category character information, second-category character information and third-category character information corresponding to the first-category labels, second-category labels and third-category labels in any standard sample, evaluates the first-category character information, the second-category character information and the third-category character information, and outputs the evaluation value. Based on the evaluation value, the parameters of the large language model are adjusted to generate a standard model.

[0023] Specifically, in this embodiment, the large language model refers to a natural language processing model with a large number of parameters trained by deep learning technology, which can be trained to perform a variety of natural language processing tasks, such as text generation, text classification, semantic search and data mining analysis. The present invention uses a large language model to train and learn the standard sample set, and the first label refers to a label character generated by summarizing multiple elements extracted from the standard sample, including but not limited to the project name, project information, bidder information and technical specifications in the standard file. The importance value is a numerical value that measures the importance of the first label according to the frequency of occurrence of the first label in each standard sample. The first category of labels refers to a set of first labels with higher importance values, the second category of labels refers to a set of first labels with medium importance values, and the third category of labels refers to a set of first labels with lower importance values. After learning the standard sample set using the large language model, the characters contained in the first label in any standard sample are extracted, and then the extracted characters can be classified in sequence according to the category of the first label, that is, the first category of character information, the second category of character information and the third category of character information.

[0024] The evaluation value refers to the accuracy of the first, second and third character information extracted by the large language model. The evaluation value is used to measure the learning effect of the large language model, and the parameters of the large language model are adjusted according to the size of the evaluation value, so that the large language model has higher accuracy. Thus, a standard model is generated, that is, the standard model refers to the model generated after the large language model is trained and adjusted.

[0025] S3: After preprocessing the bidding documents, they are input into the standard model. The label information of the bidding documents is extracted in sequence based on the importance value corresponding to the first label. If there is missing information, the missing label of the bidding document is marked. Based on the first type of character information and the second type of character information, it is determined whether there is associated information for the missing label. If so, the missing label is combined with the associated information to generate supplementary information.

[0026] Specifically, in this embodiment, preprocessing refers to the processing process of converting through file formats mentioned above, and the bidding document refers to the tender document to be processed. Label information refers to the character elements extracted by the standard model in the bidding document according to any first label. The standard model first performs information retrieval in the bidding document according to the first label extracted during training, and verifies whether the character elements corresponding to the first label exist in the bidding document. If there are no character elements corresponding to the first label, it means that there is missing information in the bidding document, and the first label is the missing label. For example, if the character elements corresponding to the first label "file attachment" are not retrieved in the bidding document, the first label "file attachment" is set as the missing label, and the character elements that have not been retrieved are marked blank, that is, missing information.

[0027] Related information refers to information in the first and second types of character information extracted from the standard sample set that is associated with entities in the bidding documents, where entities usually refer to any name of things that can be distinguished and identified in the real world. For example, if the missing label in the bidding document is "project record", and the project records between company name A and company name B are included in the first type of character information, and company name B belongs to the character element corresponding to the first label "bidder information" in the bidding document, it means that there are elements associated with the missing label "project record" in the first type of character information, that is, there is related information. Supplementary information refers to characters that supplement the search corresponding to the missing label through the first type of character information or the second type of character information. For example, the project records between company name A and company name B in the first type of character information are supplemented to the elements corresponding to the missing label "project record" to generate supplementary information.

[0028] S4: Based on the tag category, the supplementary information and tag information are summarized to generate extracted information, the bidding subject rules are set, the extracted information is analyzed based on the bidding subject rules, and the strategy recommendations corresponding to the bidding documents are output.

[0029] Specifically, in this embodiment, the label category refers to a list of classification categories of the first label, for example, the first label, the second label, and the third label. The extracted information refers to the character information formed by one-to-one correspondence between the supplementary information and the label information and the first label. The bidding subject rule refers to the business rule used to screen the bidding documents, which can enable the tenderer to better understand and evaluate the bidding documents. Therefore, by analyzing the extracted information through the bidding subject rule, an analysis report can be generated, which is a strategy recommendation.

[0030] As a preferred technical solution of the present invention, converting a file sample into a standard sample based on the file format includes the following steps: A text extraction method is set based on the file format, content information of the file sample is extracted based on the text extraction method, and the text position of the content information is marked, a standard format is set, and the content information and the text position are converted based on the standard format to generate a standard sample.

[0031] Specifically, in this embodiment, setting different text extraction methods according to different file formats can effectively summarize all file samples. For example, using the text extraction method of OCR (optical character recognition) technology to extract file samples with the file format of PDF. Content information refers to the text characters contained in the file sample, and the text position refers to the exact position in the content information file sample, such as page number, line number, chapter, etc. In the information extraction task, the text position can be used to verify whether the extracted entity appears in a logically reasonable page area. Standard format refers to a structured data format, such as XML, JSON and other data formats. Combining content information with text position and converting it into a standard sample corresponding to the standard format can more accurately understand and process the text content.

[0032] As a preferred technical solution of the present invention, S2 further includes the following steps: Set demand information based on the bidding document field, input the demand information and standard sample set into the large language model, and generate a training script.

[0033] The large language model recognizes the standard sample set based on the training script, retrieves the representation information type in any standard sample in the standard sample set based on the demand information, and extracts the predetermined keyword corresponding to the representation information type.

[0034] The semantic similarity between any two predetermined keywords is calculated. If the semantic similarity is greater than a first preset value, the two predetermined keywords are semantically expanded to generate an extended keyword. Otherwise, the two predetermined keywords are retained, and both the extended keyword and the retained predetermined keyword are set as the first label.

[0035] The large language model generates an extension instruction based on the first label, and extracts character information corresponding to the first label in any standard sample based on the extension instruction, and based on the label category corresponding to the first label, sets the character information belonging to the first category of labels as the first category of character information, sets the character information belonging to the second category of labels as the second category of character information, and sets the character information belonging to the third category of character labels as the third category of character information.

[0036] Specifically, in this embodiment, the requirement information refers to a character that provides a clear instruction or question to the large language model. The requirement information can help the model identify and extract specific information from the standard sample. Different requirement information can be set for different bidding document fields. For example, the requirement information is "labeling all standard samples in the standard sample set". The training script refers to a series of instructions corresponding to automatically extracting information from specific file data. The corresponding training script is generated from the requirement information, so that the large language model performs targeted training retrieval according to the training script.

[0037] Characteristic information type refers to features or signs that can represent or indicate a specific type of information, including but not limited to keywords, phrases, format structures, metadata, context information, etc. Predetermined keywords refer to characters used to identify and describe the characterization information type, for example, characters corresponding to predetermined keywords such as bidding number, project name, technical requirements, and commercial terms.

[0038] Semantic similarity refers to the degree of similarity between the semantic content of two predetermined keywords. If the semantic similarity is greater than a first preset value, it means that the semantics of the two predetermined keywords are the same. Therefore, the two predetermined keywords can be semantically fused and expanded through natural language processing technology to generate an extended keyword. For example, if the semantic similarity of the two predetermined keywords "project name" and "project topic" is greater than the first threshold, the extended keyword "project topic name" can be generated through natural language technology. If the semantic similarity is less than or equal to the first preset value, it means that the semantics of the two predetermined keywords are different, so the two predetermined keywords are retained. All the reserved predetermined keywords and extended keywords can be combined and set as the first label, so the number of first labels is multiple.

[0039] Extended instructions refer to instructions generated by the large language model for executing information extraction corresponding to the first label. For example, the extended instruction is "extract characters of the standard sample with the first label as the theme". The large language model itself has functions such as text analysis and information extraction. Therefore, the large language model will execute information extraction of each first label to generate character information according to the extended instructions. According to the label categories generated by the first label classification, all character information can be classified into first-category character information, second-category character information, and third-category character information in turn. Since the first label is generated in sequence by multiple standard samples, each standard sample does not necessarily contain the same first label. Therefore, it is necessary to classify the character information into the same category according to the importance value of the first label, which can subsequently improve the accuracy of the large language model evaluation.

[0040] As a preferred technical solution of the present invention, dividing all first tags into first-category tags, second-category tags, and third-category tags based on importance values ​​comprises the following steps: The number of standard samples in the standard sample set is set as a first value, the number of standard samples with a first label in the standard sample set is set as a second value, and the ratio of the second value to the first value is set as the importance value corresponding to the first label.

[0041] The division threshold interval is set based on the importance value, and the first label with an importance value greater than the threshold interval is classified as a first-category label, the first label with an importance value within the threshold interval is classified as a second-category label, and the first label with an importance value less than the threshold interval is classified as a third-category label.

[0042] Specifically, in this embodiment, the frequency of occurrence of any first label in the standard sample set, that is, the importance value, can be calculated by the first value and the second value. For example, the first value of all standard samples is 100, and the second value of the standard sample with the first label A1 is 80, then the importance value of the first label A1 is 80 / 100=0.8.

[0043] The threshold interval refers to the numerical interval used to divide the importance value. For example, if the threshold interval is set to [0.5, 0.7], the first label whose importance value is greater than 0.7 is the first category label, the first label whose importance value belongs to the threshold interval is the second category label, and the first label whose importance value is less than 0.5 is the third category label.

[0044] As a preferred technical solution of the present invention, the evaluation value is output based on the following steps: Preset proportions of first-category character information, second-category character information, and third-category character information are extracted respectively and set as a test information set, the correct label corresponding to the test information set is marked in the standard sample set, the first label corresponding to the test information set is extracted and set as the extracted label, the proportion distribution of the first-category label, the second-category label, and the third-category label in the extracted label is obtained, and the weight coefficient is set based on the proportion distribution.

[0045] A confusion matrix is ​​constructed based on the correct labels and extracted labels. The accuracy of the classification corresponding to any test information in the test information set is output based on the confusion matrix. The product of the accuracy and the weight coefficient is accumulated to generate an evaluation value.

[0046] Specifically, in the present embodiment, the preset ratio can be set according to the quantity of the first category character information, the second category character information and the third category character information. The test information set refers to indicators such as accuracy, recall rate and F1 score used to evaluate the large language model, wherein the test information set refers to a collection of character information of each category. The large language model performs reverse text topic aggregation based on the extracted test information set to generate a correct label, wherein the correct label refers to a topic label generated by the large language model after performing text aggregation on any test information in the test information set. Since all test information in the test information set belongs to each category of character information, the extracted label is the first label corresponding to each category of character information, and the number of extracted labels is multiple, wherein each category of character information includes the first category of character information, the second category of character information and the third category of character information. In order to improve the evaluation accuracy of the large language model, it is necessary to set the weight coefficient according to the proportional distribution of the extracted labels. For example, if the number of extracted labels belonging to the first category, the second category, and the third category are 10, 6, and 4 respectively, the proportional distribution is 5:3:2, and the weight coefficient can be set to 0.5, 0.3, and 0.2 respectively.

[0047] The confusion matrix is ​​a tool used to evaluate model performance, especially in the field of machine learning, which describes the relationship between the predicted first label of the classification model and the actual correct label through a table. Figure 2 As shown in the figure, a confusion matrix is ​​constructed. Precision is a measure of the consistency between the extracted test information and the extracted labels, as well as the accuracy of the extraction of the classification of each label, where the classification of each label includes the first category label, the second category label, and the third category label. The semantic similarity between the correct label and the extracted label is used to determine whether the extraction of each test information in the test information set is consistent. For example, if the semantic similarity between the correct label and the extracted label corresponding to the same test information is high, it means that the test information is accurately extracted by the large language model. Figure 2 It can be seen that the accuracy of the first category of labels is 8 / (8+1+1)=0.8, the accuracy of the second category of labels is 5 / (5+1)=0.83, and the accuracy of the third category of labels is 3 / (3+1)=0.75. If the weight coefficients corresponding to the first category of labels, the second category of labels and the third category of labels are 0.5, 0.3 and 0.2, then the evaluation value is (0.8×0.5+0.83×0.3+0.75×0.2)=0.8, where the evaluation value is between 0 and 1.

[0048] As a preferred technical solution of the present invention, generating a standard model includes: If the evaluation value is less than the second preset value, the difference between the evaluation value and the second preset value is calculated, and the learning rate corresponding to the parameter in the large language model is adjusted based on the difference. This step is repeated until the evaluation value is greater than or equal to the second preset value, and the adjusted large language model is set as the standard model. Otherwise, the trained large language model is set as the standard model.

[0049] Specifically, in this embodiment, the second preset value refers to a numerical value limited for evaluating the large language model. If the evaluation value is greater than or equal to the second preset value, it means that the large language model has a good training and learning effect on the standard sample set, and the large language model after training can be directly set as the standard model. Otherwise, it means that the training and learning effect of the large language model on the standard sample set is poor. Since there are many complex parameters in the large language model, the present invention uses the parameters corresponding to the adjustment learning rate to adjust the large language learning model. For example, if the second preset value is 0.75 and the evaluation value is 0.6, the difference is 0.75-0.6=0.15, and the ratio of the difference to the evaluation value 0.15 / 0.75=0.2 is set as the adjustment ratio of the learning rate. If the data quality of the standard sample set is high, the learning rate is increased and adjusted according to the adjustment ratio of 0.2. If the data quality of the standard sample set is low, the learning rate is reduced and adjusted according to the adjustment ratio of 0.2, wherein the data quality of the standard sample set can be calculated and measured according to the correlation of each standard sample in the standard sample set. Repeat this step to adjust the parameters of the large language model, and then set the adjusted large language model as the standard model.

[0050] As a preferred technical solution of the present invention, generating the extraction information includes the following steps: The standard model extracts file information from the bidding document based on the first label. If the file information corresponds one-to-one to the first label, it is determined that there is no missing information in the bidding document, and the file information is set as label information. Otherwise, the first label without file information is set as a missing label.

[0051] If the missing label belongs to the first category label or the second category label, multiple entity information is extracted from the first category character information and the second category character information. If there is entity information in the bidding document, the first category character information or the second category character information is semantically expanded based on the entity information to generate supplementary information. Otherwise, the missing label is removed.

[0052] The first tags are arranged in sequence based on importance values ​​to generate tag categories, and the supplementary information and tag information are combined in sequence based on the tag categories to generate extracted information.

[0053] Specifically, in this embodiment, the file information refers to the characters extracted by the first label after the preprocessed bidding document is input into the standard model. One-to-one correspondence means that any first label has corresponding file information. If the file information corresponds to the first label one-to-one, it means that the file information corresponding to the bidding document is completely extracted and there is no missing information. If there is a first label that does not extract the corresponding file information, the first label is set as a missing label.

[0054] The first and second category tags have higher importance values. Therefore, if the missing tag belongs to the first or second category tags, it is necessary to further check whether there is supplementary information corresponding to the missing tag. Otherwise, it means that the importance value of the missing tag is low and the missing tag can be removed. Entity information refers to words, phrases or names with specific meanings in the text, including but not limited to the name of a bidding company, a person, a facility, a place, etc. By extracting entity information from the first and second category character information, and then searching for the same entity information in the bidding documents, it can be determined whether there is an entity association between the bidding documents and multiple standard documents collected historically. If so, the first or second category character information can be semantically expanded according to the entity information to generate supplementary information.

[0055] Extracting information is to combine the supplementary information, label information and each first label in correspondence and arrange them in sequence according to the label category of the first label, and then summarize the generated characters, which can clearly and accurately describe the information contained in the bidding documents.

[0056] As a preferred technical solution of the present invention, outputting the strategy suggestions corresponding to the bidding documents includes the following steps: A business set is extracted in the first label based on the attribute type of the bidding document, a business structure is created based on the business set, and a bidding subject rule is set based on the business structure.

[0057] The business information contained in the business set is extracted from the extracted information, the business information is compared and verified with the bidding subject rules, and the verification results are output. All verification results are combined to generate an analysis report, and the analysis report is set as a strategy recommendation.

[0058] Specifically, in this embodiment, the attribute type refers to the industry business type, including but not limited to building attributes, equipment attributes, etc. The business set refers to a set of first tags containing key meanings, such as "bidder name", "bid price", "technical specifications", etc. The business structure refers to a hierarchical structure formed by organizing related key meanings together, for example, "equipment requirements" of the key meaning under "technical specifications" is a sub-meaning. The bidding subject rule can generate characters after setting a threshold according to the business structure, for example, "the threshold of equipment performance parameters in the technical specifications is 10."

[0059] Business information refers to the characters extracted from the extracted information based on the business set. Comparative verification refers to the numerical verification process. The verification results include two types: "meets the requirements" and "does not meet the requirements". The analysis report refers to the characters after all verification results are summarized through the bidding subject rules. For example, the business information of bidding document C is "the equipment performance parameter in the technical specifications is 6", which is compared and verified with the bidding subject rules. Among them, "the parameter is 6" is less than "the parameter threshold is 10", so the verification result is "does not meet the requirements", and the corresponding analysis report is "the equipment performance parameters in the technical specifications do not meet the requirements, and the bidding document is marked as a high-risk document", which is a strategy recommendation.

[0060] like Figure 3 As shown, the present invention also provides a bidding document information extraction system based on a large language model, which is used to implement the above-mentioned bidding document information extraction method based on a large language model. The system mainly includes: The sample collection module collects multiple file samples in the bidding document field, obtains the file format of the file samples, converts the file samples into standard samples based on the file format, and combines all standard samples to generate a standard sample set.

[0061] The model generation module constructs a large language model. The large language model extracts the first label contained in the standard sample based on the standard sample set, calculates the importance value corresponding to the first label based on the standard sample set, divides all the first labels into first-category labels, second-category labels and third-category labels based on the importance value, respectively extracts the first-category character information, second-category character information and third-category character information corresponding to the first-category labels, second-category labels and third-category labels in any standard sample, evaluates the first-category character information, the second-category character information and the third-category character information, and outputs the evaluation value. The parameters of the large language model are adjusted based on the evaluation value to generate a standard model.

[0062] The information extraction module pre-processes the bidding documents and inputs them into the standard model. The label information of the bidding documents is extracted in sequence based on the importance value corresponding to the first label. If there is missing information, the missing label of the bidding document is marked. Based on the first and second character information, it is determined whether there is related information for the missing label. If so, the missing label is combined with the related information to generate supplementary information.

[0063] The document analysis module aggregates supplementary information and label information based on label categories, generates extracted information, sets bidding theme rules, analyzes the extracted information based on the bidding theme rules, and outputs strategy recommendations corresponding to the bidding documents.

[0064] It should be understood that, although each step in the flow chart of each embodiment of the present invention is shown in sequence according to the indication of the arrow, these steps are not necessarily performed in sequence according to the order indicated by the arrow. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be performed in other orders. Moreover, at least a portion of the steps in each embodiment may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily performed at the same time, but can be performed at different times, and the execution order of these sub-steps or stages is not necessarily performed in sequence, but can be performed in turn or alternately with at least a portion of other steps or sub-steps or stages of other steps.

[0065] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the above-mentioned program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0066] The technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above-mentioned embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0067] The above embodiments only express several implementation methods of the present invention, and the description is relatively specific and detailed, but it cannot be understood as limiting the scope of the present invention. It should be pointed out that for ordinary technicians in this field, several modifications and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention.

[0068] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for extracting information from bidding documents based on a large language model, characterized in that: The method comprises the following steps: S1: Collect multiple file samples in the bidding document field, obtain the file formats of the file samples, convert the file samples into standard samples based on the file formats, and combine all the standard samples to generate a standard sample set; S2: constructing a large language model, wherein the large language model extracts a first label contained in the standard sample based on the standard sample set, calculates an importance value corresponding to the first label based on the standard sample set, divides all the first labels into first-category labels, second-category labels, and third-category labels based on the importance value, respectively extracts first-category character information, second-category character information, and third-category character information corresponding to the first-category labels, the second-category labels, and the third-category labels in any of the standard samples, evaluates the first-category character information, the second-category character information, and the third-category character information, and outputs an evaluation value, and adjusts parameters of the large language model based on the evaluation value to generate a standard model; S3: Preprocessing the bidding document and inputting it into the standard model, extracting the label information of the bidding document in sequence based on the importance value corresponding to the first label, marking the missing label of the bidding document if there is missing information, judging whether the missing label has associated information based on the first type of character information and the second type of character information, and if so, combining the missing label with the associated information to generate supplementary information; S4: Summarize the supplementary information and the label information based on the label category, generate extracted information, set bidding subject rules, analyze the extracted information based on the bidding subject rules, and output the strategy recommendations corresponding to the bidding document.

2. The method according to claim 1, characterized in that The converting the file sample into a standard sample based on the file format comprises the following steps: A text extraction method is set based on the file format, content information of the file sample is extracted based on the text extraction method, and the text position of the content information is marked, a standard format is set, and the content information and the text position are converted based on the standard format to generate the standard sample.

3. The method according to claim 1, characterized in that S2 also includes the following steps: Setting requirement information based on the bid document field, inputting the requirement information and the standard sample set into the large language model, and generating a training script; The large language model identifies the standard sample set based on the training script, retrieves the representation information type in any standard sample in the standard sample set based on the demand information, and extracts a predetermined keyword corresponding to the representation information type; Calculating the semantic similarity between any two predetermined keywords, if the semantic similarity is greater than a first preset value, semantically extending the two predetermined keywords to generate an extended keyword, otherwise, retaining the two predetermined keywords, and setting the extended keyword and the two predetermined keywords as the first label; The large language model generates an extension instruction based on the first label, and extracts character information corresponding to the first label in any of the standard samples based on the extension instruction, and based on the label category corresponding to the first label, sets the character information belonging to the first category of labels as the first category of character information, sets the character information belonging to the second category of labels as the second category of character information, and sets the character information belonging to the third category of character labels as the third category of character information.

4. The method according to claim 3, characterized in that The method of dividing all the first tags into first-category tags, second-category tags, and third-category tags based on the importance values ​​comprises the following steps: The number of the standard samples in the standard sample set is set as a first value, the number of the standard samples with the first label in the standard sample set is set as a second value, and the ratio of the second value to the first value is set as the importance value corresponding to the first label; A division threshold interval is set based on the importance value, and the first tag whose importance value is greater than the threshold interval is classified as the first category tag, the first tag whose importance value belongs to the threshold interval is classified as the second category tag, and the first tag whose importance value is less than the threshold interval is classified as the third category tag.

5. The method according to claim 1, characterized in that Output evaluation values ​​based on the following steps: Respectively extracting preset proportions of the first category of character information, the second category of character information, and the third category of character information and setting them as a test information set, marking a correct label corresponding to the test information set in the standard sample set, extracting the first label corresponding to the test information set and setting it as an extracted label, obtaining a proportion distribution of the first category of labels, the second category of labels, and the third category of labels in the extracted labels, and setting a weight coefficient based on the proportion distribution; A confusion matrix is ​​constructed based on the correct labels and the extracted labels, the accuracy of the classification corresponding to any test information in the test information set is output based on the confusion matrix, and the product of the accuracy and the weight coefficient is accumulated to generate the evaluation value.

6. The method according to claim 5, characterized in that The generation standard model includes: If the evaluation value is less than a second preset value, the difference between the evaluation value and the second preset value is calculated, and the learning rate corresponding to the parameter in the large language model is adjusted based on the difference. This step is repeated until the evaluation value is greater than or equal to the second preset value, and the adjusted large language model is set as the standard model. Otherwise, the trained large language model is set as the standard model.

7. The method according to claim 1, characterized in that The generating of the extraction information comprises the following steps: The standard model extracts file information from the bidding document based on the first label. If the file information corresponds to the first label one-to-one, it is determined that the bidding document does not contain the missing information, and the file information is set as the label information; otherwise, the first label that does not contain the file information is set as the missing label. If the missing tag belongs to the first category of tags or the second category of tags, multiple entity information is extracted from the first category of character information and the second category of character information. If the entity information exists in the bidding document, the first category of character information or the second category of character information is semantically expanded based on the entity information to generate the supplementary information. Otherwise, the missing tag is removed. The first tags are arranged in sequence based on the importance values ​​to generate the tag category, and the supplementary information and the tag information are combined in sequence based on the tag category to generate the extracted information.

8. The method according to claim 1, characterized in that Outputting the strategy suggestion corresponding to the bidding document comprises the following steps: Extracting a business set in the first tag based on the attribute type of the bidding document, creating a business structure based on the business set, and setting the bidding subject rule based on the business structure; The business information included in the business set is extracted from the extracted information, the business information is compared and verified with the bidding subject rules, and the verification result is output, all the verification results are combined to generate an analysis report, and the analysis report is set as the policy recommendation.

9. A bidding document information extraction system based on a large language model, used to implement the method according to any one of claims 1 to 8, characterized in that: The system includes the following modules: A sample collection module collects multiple file samples in the bidding document field, obtains the file format of the file samples, converts the file samples into standard samples based on the file format, and combines all the standard samples to generate a standard sample set; A model generation module is configured to construct a large language model, wherein the large language model extracts a first label contained in the standard sample based on the standard sample set, calculates an importance value corresponding to the first label based on the standard sample set, divides all the first labels into first-category labels, second-category labels, and third-category labels based on the importance value, respectively extracts first-category character information, second-category character information, and third-category character information corresponding to the first-category labels, the second-category labels, and the third-category labels in any of the standard samples, evaluates the first-category character information, the second-category character information, and the third-category character information, and outputs an evaluation value, and adjusts parameters of the large language model based on the evaluation value to generate a standard model; an information extraction module, which pre-processes the bidding document and then inputs it into the standard model, extracts the label information of the bidding document in sequence based on the importance value corresponding to the first label, marks the missing label of the bidding document if there is missing information, determines whether there is associated information for the missing label based on the first type of character information and the second type of character information, and if so, combines the missing label with the associated information to generate supplementary information; The document analysis module aggregates the supplementary information and the label information based on label categories, generates extracted information, sets bidding subject rules, analyzes the extracted information based on the bidding subject rules, and outputs strategy recommendations corresponding to the bidding document.

Images