Intelligent quotation analysis method and system based on large language model
By performing unified format preprocessing and effective information screening on the quotation, combined with the analysis of the large language generation model, the problems of slow analysis speed and poor effect in the existing technology are solved, and efficient and accurate quotation for multi-type and multi-format quotation is achieved. Analyze and evaluate the accuracy of the analysis results through evaluation indicators.
Patent Information
- Application Number
- CN202410918162.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-10
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2044-07-10
AI Technical Summary
In the building materials industry, it is difficult for the existing technology to effectively analyze multi-type and multi-format quotations, resulting in slow resolution speed and poor resolution effect, and it is difficult for large language generation models to evaluate the analysis results.
By performing unified format preprocessing of the quotation to be parsed, using the trained effective information filtering model to filter irrelevant information, constructing input text containing prompt words, inputting a large language generation model for analysis, and obtaining evaluation indicators for parsing the analysis results through comparison and correction .
It realizes efficient and accurate analysis of multi-type and multi-format quotations, significantly improves the analysis speed and accuracy, avoids the generation of invalid information, and evaluates the accuracy of the analysis results through evaluation indicators.
Smart Images

Figure CN118917298A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method and system for intelligently parsing quotations based on a large language model. Background Art
[0002] Currently, in the building materials industry, due to the diversity of types and formats of building materials product quotations, in order to facilitate their query, it is often necessary to unify the formats of quotations of multiple types and formats. Currently, there are two main methods: manual processing and AI processing.
[0003] Manually unifying the format and obtaining valid content is time-consuming and inaccurate. Common AI technologies (such as large language models, where only generative models based on large language models can be used because there is a certain amount of content conversion in the parsing results) often cause slow parsing speed and poor parsing results because the quotation contains certain irrelevant information; due to the parameter setting problems of the generative model itself, redundant product information occasionally appears. And due to the difficulty of evaluating the generative model, the parsing results of the large language generative model cannot be evaluated. Summary of the invention
[0004] The purpose of the present invention is to provide a method and system for intelligent parsing of quotations based on a large language model in order to solve at least one of the above technical problems.
[0005] In a first aspect, an embodiment of the present invention provides a method for intelligent parsing of quotations based on a large language model, comprising: performing format uniform preprocessing on the quotations to be parsed to obtain the preprocessed quotations to be parsed; based on a trained effective information screening model, performing effective information screening on the preprocessed quotations to be parsed to obtain screened information; based on the screened information, constructing an input parsing text containing prompt words; inputting the input parsing text containing the prompt words into the trained large language generation model to obtain a parsing result of the quotations to be parsed; comparing and correcting the parsing results based on the screened information to obtain an evaluation index of the parsing results.
[0006] Furthermore, the quotation to be parsed is preprocessed in a unified format, including: based on the file name suffix of the quotation to be parsed, determining the file type of the quotation to be parsed; the file type includes any one of the following: excel table, word document, picture file; if the file type is a word document, obtaining the table information in the document; if the file type is a picture file, obtaining the table information in the file based on an optical character recognition method; and storing the table information in the file type of an excel table.
[0007] Furthermore, it also includes: training a first preset large language model based on a manually annotated training data set to obtain an effective information screening model after the training; wherein the loss function of the training process includes an adaptive weighted loss function.
[0008] Furthermore, it also includes: training a second preset large language model based on a manually annotated training data set to obtain the trained large language generation model; wherein the manually annotated training data set includes parsed text data, valid data annotation information and parsing results.
[0009] Furthermore, based on the filtered information, an input parsed text containing prompt words is constructed, including: obtaining the number of product information of the filtered information; constructing prompt words based on the number of product information; and constructing the input parsed text containing prompt words based on the prompt words and the filtered information.
[0010] Furthermore, the evaluation index includes a product quantity consistency index and a product information content consistency index.
[0011] Furthermore, the analysis results are compared and corrected based on the filtered information to obtain evaluation indicators of the analysis results, including: segmenting the analysis results to separately determine the product information quantity and product content information in the analysis results; the product content information includes product name and market price; obtaining the product information quantity and product content information in the filtered information; determining the product quantity consistency indicator based on the product quantity information in the analysis results and the product information quantity in the filtered information; determining the product content consistency indicator based on the product content information in the analysis results and the product content information in the filtered information.
[0012] In a second aspect, an embodiment of the present invention further provides an intelligent parsing system for quotations based on a large language model, comprising: a preprocessing module, a screening module, a building module, a parsing module and an evaluation module; wherein the preprocessing module is used to perform format uniform preprocessing on the quotations to be parsed to obtain the preprocessed quotations to be parsed; the screening module is used to screen the effective information of the preprocessed quotations to be parsed based on the trained effective information screening model to obtain the screened information; the building module is used to construct an input parsed text containing prompt words based on the screened information; the parsing module is used to input the input parsed text containing prompt words into the trained large language generation model to obtain the parsing result of the quotation to be parsed; the evaluation module is used to compare and correct the parsing result based on the screened information to obtain the evaluation index of the parsing result.
[0013] Furthermore, it also includes a training module, wherein the training module is used to train the first preset large language model and the second preset large language model respectively based on the manually annotated training data set to obtain the effective information screening model after the training and the large language generation model after the training.
[0014] In a third aspect, an embodiment of the present invention further provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in the first aspect when executing the computer program.
[0015] The present invention provides a method and system for intelligent parsing of quotations based on a large language model. Through an early data screening model, irrelevant information in the quotation can be effectively screened out, providing effective parsing data for subsequent large language model parsing; and depending on the number of valid information items of the data screening model, prompt engineering is performed on the large language model to avoid the generation of invalid information; and indicator evaluation is performed based on the results of large language generation and the results of the screening model, alleviating the technical problems of slow parsing speed and poor parsing effect existing in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS
[0016] In order to more clearly illustrate the implementation methods of the present application or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0017] Figure 1 A flowchart of a method for intelligent parsing of quotations based on a large language model provided by an embodiment of the present invention;
[0018] Figure 2 A flowchart of another method for intelligent parsing of quotations based on a large language model provided by an embodiment of the present invention;
[0019] Figure 3 A schematic diagram of a quotation to be parsed provided by an embodiment of the present invention;
[0020] Figure 4 A schematic diagram of a quotation intelligent parsing system based on a large language model provided in an embodiment of the present invention. DETAILED DESCRIPTION
[0021] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0022] Embodiment 1
[0023] Figure 1 FIG. 1 is a flowchart of a method for intelligently parsing a quotation based on a large language model according to an embodiment of the present invention. Figure 1 As shown, the method specifically comprises the following steps:
[0024] Step S102, preprocessing the quotation to be parsed to unify its format, and obtaining the preprocessed quotation to be parsed.
[0025] Step S104, based on the trained effective information screening model, screen the effective information of the pre-processed quotation to be parsed to obtain the screened information.
[0026] Step S106: construct an input parsed text containing the prompt word based on the filtered information.
[0027] Step S108, inputting the input parsed text containing the prompt word into the trained large language generation model to obtain the parsed result of the quotation to be parsed.
[0028] Step S110, comparing and correcting the analysis results based on the filtered information to obtain evaluation indicators of the analysis results.
[0029] Figure 2 FIG. 1 is a flowchart of another method for intelligent parsing of quotations based on a large language model according to an embodiment of the present invention. Figure 2 As shown, specifically, step S102 also includes the following steps:
[0030] Step S1021, based on the file name suffix of the quotation to be parsed, determine the file type of the quotation to be parsed; the file type includes any of the following: Excel spreadsheet, Word document, picture file;
[0031] Step S1022, if the file type is a word document, obtain the table information in the document;
[0032] Step S1023, if the file type is a picture file, obtaining table information in the file based on an optical character recognition method;
[0033] Step S1024, storing the table information in the file type of an Excel table.
[0034] Specifically, the embodiment of the present invention performs a unified conversion for various quotation file types, converting them into Excel files (useful product information in the quotation is in the form of a table). There are three types of file types currently processed, namely, Excel files, whose file suffixes are: xlsx, xls, csv; docx files, whose file suffixes are docx; and picture files, whose suffixes are jpg and png. By distinguishing the file types based on the suffixes of the file names, Excel files do not need to be processed. For docx files, the existing python-docx third-party dependency is used to obtain the table information therein and save it as an Excel file; for picture files, the existing optical character recognition (OCR) technology is used to extract the table information therein and save it as an Excel file.
[0035] Specifically, Figure 2 As shown, it also includes: training a first preset large language model based on a manually annotated training data set to obtain an effective information screening model after training; wherein the loss function of the training process includes an adaptive weighted loss function.
[0036] In the embodiment of the present invention, the training data of the effective information screening model are all obtained by manual annotation, which mainly includes parsed text data and effective data annotation information.
[0037] Figure 3 FIG. 1 is a schematic diagram of a quotation to be parsed according to an embodiment of the present invention. Figure 3 Taking the file shown as an example, the parsed text data is Appendix 1 Excel file, and its valid data annotation information is: {"Appendix 1":"Invalid information, invalid information, index row, product invalid information, product invalid information, valid information, invalid information"}, indicating that the 1st, 2nd and 7th rows of the Appendix 1 Excel quotation are invalid information, the 3rd row is the index row of valid information, the 4th and 5th rows are invalid product information (although it is product information, it lacks price), and the 6th row is valid product information.
[0038] The training data format of the effective information screening model is:
[0039] [{"content":"Row content (different columns in the same row are separated by separators string concatenation)","label":"**"}]
[0040] by Figure 3 The training data constructed by the file shown is:
[0041] [{"content":"Purchase order","label":"Invalid information"},...
[0042] {"content":"Serial number name Model / Specification / Parameter Preliminary quantity unit price Notes","label":"Index Row"},...]
[0043] For the training data set consisting of all excel files, the data distribution is uneven. Here, an adaptive weighted loss function is used. By obtaining the number of categories in the training data, the loss weight of each category is obtained.
[0044] Count={c1,c2,...,c n} represents the number of times n categories appear in the training data, and c1 represents the number of times category 1 appears in the training data. The corresponding loss weight W for each category is:
[0045] Where N is the total number of training data.
[0046] During model training, the model loss function in represents the probability that training data j belongs to class i, Indicates that sample j belongs to category i, Indicates that sample j does not belong to category i. Then the loss of the category is multiplied by the weight of the corresponding category, and then superimposed twice to obtain the weighted loss of all samples in each category, which is used as the model loss of the current training to guide model optimization.
[0047] The input parsing text of the effective information screening model is a string concatenated from the information of each row in the Excel file, such as the content in the above content.
[0048] The output result of the valid information screening model is a row category (one of the four: invalid information, index row, invalid product information, and valid information).
[0049] The training data of the effective information screening model consists of the row contents of all Excel files and their corresponding label categories.
[0050] When applied, the input parsed text of the effective information screening model is the content of each row of the Excel file, and the output result is the row category.
[0051] Specifically, Figure 2 As shown, the method also includes: training a second preset large language model based on a manually annotated training data set to obtain a trained large language generation model; wherein the manually annotated training data set includes parsed text data, valid data annotation information and parsing results.
[0052] Specifically, step S106 also includes the following steps:
[0053] Step S1061, obtaining the number of product information after filtering;
[0054] Step S1062, constructing prompt words based on the quantity of product information;
[0055] Step S1063: construct an input parsed text containing the prompt word based on the prompt word and the filtered information.
[0056] In the embodiment of the present invention, the training data of the Excel parsing model (i.e., the large language generation model) are all obtained by manual annotation, which mainly includes parsed text data, valid data annotation information and parsing results.
[0057] Excel parsing model input content:
[0058] By parsing text data (such as Figure 3 Appendix 1 excel file shown in the figure), valid data annotation information, can get the valid information string of the excel quotation (only keep the excel row information labeled as index row and valid information, and splice the row information); to improve the reliability of the large language generation model, avoid generating additional product information. According to the valid annotation information, get the number of valid product information and add it to the prompt.
[0059] Prompt="Analyze the subsequent Excel string information to obtain the product information. The string contains ** (specific number of valid information, obtained from the statistics of valid data annotation information) product information. Please do not generate redundant product information. The Excel string to be parsed is as follows:\n"
[0060] The final input parsing statement format of the large language model is: Prompt+excel valid information string.
[0061] Excel analysis model output results:
[0062] The valid information in the parsing result (Appendix 4) (except the blue index row) is concatenated in the above manner (with columns separated by separators). Concatenate strings, with delimiters between lines Perform row string concatenation), empty cell information is marked with the character "-1" (some fields in some parsed files have values, while others are empty. They are of the same type and are all set to have values. -1 represents a meaningless placeholder value).
[0063] The training data of the Excel parsing model consists of the valid information string of Excel and its corresponding valid information string of the parsed file content.
[0064] When applied, after the Excel file passes through the valid information model, the Excel row string labeled as the index row and valid information is retained and spliced into a parsing string; and the Prompt is supplemented and improved according to the number of valid information categories; then the Prompt+parsing string is used as the input of the Excel parsing model, and the output result is a string based on the corresponding formatted parsing result.
[0065] Preferably, the evaluation index includes a product quantity consistency index and a product information content consistency index.
[0066] Specifically, step S110 also includes the following steps:
[0067] Step S1101, segmenting the analysis result, and determining the quantity of product information and product content information in the analysis result respectively; the product content information includes the product name and market price.
[0068] Step S1102, obtaining the quantity of product information and product content information in the filtered information.
[0069] Step S1103, determining a product quantity consistency index based on the product quantity information in the parsing result and the product information quantity in the filtered information.
[0070] Step S1104: determining a product content consistency index based on the product content information in the parsing result and the product content information in the filtered information.
[0071] Specifically, the Excel parsing model parses the quotation string containing only valid information and obtains the parsing result of the corresponding quotation. However, there is no evaluation of the parsing effect and no verification of the parsing result at this time.
[0072] The resulting string is By segmenting, the number of product information in the analysis results can be determined. This number should be consistent with the number of valid information categories in the valid information screening model, so as to obtain the product quantity consistency index (consistent or inconsistent);
[0073] by After segmentation, each piece of parsed product information is an independent string, and then “After string segmentation, the specific product name, market price, etc. of each product can be obtained according to the order. Since all this information comes from the parsed text, it is necessary to determine whether the product name and market price belong to the string composed of the original Excel file.
[0074] In this way, two types of indicators will be obtained: whether the product quantity is consistent, and whether the product name and market price of each line of product information in the analysis result are consistent. A preliminary evaluation of the analysis result is given, which also reduces a certain amount of inspection work.
[0075] From the above description, it can be seen that the embodiment of the present invention provides a method for intelligent parsing of quotations based on a large language model, which can efficiently and accurately parse quotation files of various types and formats. Compared with the existing AI processing method, the method provided by the embodiment of the present invention reduces the length of the parsed text of the large language model through the Excel effective information classification model with millisecond response, and the parsing speed is about 10 times the original parsing speed; through prompt construction, the number of product information items in the model parsing results and the actual number of product information items in the parsed text have been greatly improved, which effectively improves the accuracy of the parsing results.
[0076] Embodiment 2
[0077] Figure 4 FIG. 1 is a schematic diagram of a quotation intelligent parsing system based on a large language model according to an embodiment of the present invention. Figure 4 As shown, the system includes: a preprocessing module 10, a screening module 20, a construction module 30, a parsing module 40 and an evaluation module 50.
[0078] Specifically, the preprocessing module 10 is used to preprocess the quotation to be parsed in a unified format to obtain the preprocessed quotation to be parsed.
[0079] The screening module 20 is used to screen the valid information of the pre-processed quotation to be parsed based on the trained valid information screening model to obtain the screened information.
[0080] The construction module 30 is used to construct the input parsed text containing the prompt word based on the filtered information.
[0081] The parsing module 40 is used to input the input parsing text containing the prompt word into the trained large language generation model to obtain the parsing result of the quotation to be parsed.
[0082] The evaluation module 50 is used to compare and correct the analysis results based on the filtered information to obtain evaluation indicators of the analysis results.
[0083] Specifically, the preprocessing module 10 is also used for:
[0084] Based on the file name suffix of the quotation to be parsed, determine the file type of the quotation to be parsed; the file type includes any of the following: Excel spreadsheet, Word document, image file;
[0085] If the file type is a word document, obtain the table information in the document;
[0086] If the file type is a picture file, the table information in the file is obtained based on the optical character recognition method;
[0087] Store the table information in the file type of Excel table.
[0088] Specifically, Figure 4 As shown, the system further includes a training module 60 for:
[0089] Based on the manually annotated training data set, the first preset large language model and the second preset large language model are trained respectively to obtain a trained effective information screening model and a trained large language generation model.
[0090] The specific building block 30 is also used for:
[0091] Get the number of product information after filtering;
[0092] Construct prompt words based on the amount of product information;
[0093] Based on the prompt word and the filtered information, construct an input parsed text containing the prompt word.
[0094] Preferably, the evaluation index includes a product quantity consistency index and a product information content consistency index.
[0095] Specifically, the evaluation module 50 is further used for:
[0096] Segment the parsing results to determine the quantity of product information and product content information in the parsing results; the product content information includes product name and market price;
[0097] Obtain the number of product information and product content information in the filtered information;
[0098] Determine the product quantity consistency index based on the product quantity information in the parsing results and the product information quantity in the filtered information;
[0099] Based on the product content information in the parsing result and the product content information in the filtered information, a product content consistency index is determined.
[0100] An embodiment of the present invention further provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method in the above-mentioned embodiment 1 when executing the computer program.
[0101] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.
[0102] In addition, it should be understood that although the present specification is described according to implementation modes, not every implementation mode contains only one independent technical solution. This description of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment may also be appropriately combined to form other implementation modes that can be understood by those skilled in the art.
Claims
1. A method for intelligent parsing of quotations based on a large language model, characterized in that: include: Perform unified format preprocessing on the quotations to be parsed, and obtain the quotations to be parsed after preprocessing; Based on the trained effective information screening model, the effective information of the quotation to be parsed after the preprocessing is screened to obtain the screened information; Based on the filtered information, construct an input parsed text containing the prompt word; Inputting the input parsed text containing the prompt word into the trained large language generation model to obtain the parsed result of the quotation to be parsed; The analysis result is compared and corrected based on the screened information to obtain an evaluation index of the analysis result.
2. The method according to claim 1, characterized in that: Perform unified format preprocessing on the quotations to be parsed, including: Based on the file name suffix of the quotation to be parsed, determine the file type of the quotation to be parsed; the file type includes any one of the following: Excel spreadsheet, Word document, picture file; If the file type is a word document, obtaining table information in the document; If the file type is a picture file, obtaining table information in the file based on an optical character recognition method; The table information is stored in the file type of Excel table.
3. The method according to claim 1, characterized in that: Also includes: Based on a manually annotated training data set, a first preset large language model is trained to obtain an effective information screening model after the training; wherein the loss function of the training process includes an adaptive weighted loss function.
4. The method according to claim 1, characterized in that: Also includes: Based on the manually annotated training data set, the second preset large language model is trained to obtain the trained large language generation model; wherein the manually annotated training data set includes parsed text data, valid data annotation information and parsing results.
5. The method according to claim 1, characterized in that: Based on the filtered information, construct an input parsed text containing prompt words, including: Obtain the quantity of product information of the information after the screening; Constructing a prompt word based on the quantity of product information; Based on the prompt word and the filtered information, the input parsed text containing the prompt word is constructed.
6. The method according to claim 1, characterized in that: The evaluation index includes a product quantity consistency index and a product information content consistency index.
7. The method according to claim 6, characterized in that: The analysis result is compared and corrected based on the screened information to obtain evaluation indicators of the analysis result, including: Segmenting the analysis results, and determining the quantity of product information and product content information in the analysis results respectively; the product content information includes product name and market price; Obtaining the quantity of product information and product content information in the filtered information; Determining the product quantity consistency index based on the product quantity information in the parsing result and the product information quantity in the filtered information; The product content consistency index is determined based on the product content information in the parsing result and the product content information in the filtered information.
8. A quotation intelligent parsing system based on a large language model, characterized in that: include: Preprocessing module, screening module, construction module, parsing module and evaluation module; among them, The preprocessing module is used to preprocess the quotation to be parsed in a unified format to obtain the preprocessed quotation to be parsed; The screening module is used to screen the effective information of the pre-processed quotation to be parsed based on the trained effective information screening model to obtain the screened information; The construction module is used to construct an input parsed text containing prompt words based on the filtered information; The parsing module is used to input the input parsing text containing the prompt word into the trained large language generation model to obtain the parsing result of the quotation to be parsed; The evaluation module is used to compare and correct the analysis result based on the screened information to obtain an evaluation index of the analysis result.
9. The system according to claim 8, characterized in that: Also included is a training module, where The training module is used to train the first preset large language model and the second preset large language model respectively based on the manually annotated training data set to obtain the trained effective information screening model and the trained large language generation model.
10. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method according to any one of claims 1 to 7 when executing the computer program.
Citation Information
Patent Citations
Intelligent dialogue method and device, computer equipment and storage medium
CN117591649A
Instruction analysis method, device and equipment based on large language model
CN117910454A
Workpiece three-dimensional model design generation method and system based on large language model
CN118012416A
Summary information generating system and method for review of product and service
KR1020130083092A
Method and system for electronic document handling, such as for requests for quotations under an electronic auction
US20040215467A1
Cited By
Method and device for generating quotation sheet based on large language model
CN119311698A
