Intelligent bidding document qualification information extraction method based on knowledge graph and mixed retrieval

Abstract

The invention relates to the technical field of data information extraction, in particular to a bidding document qualification information intelligent extraction method based on a knowledge graph and hybrid retrieval, which comprises the following steps: a document preprocessing and partitioning step: preprocessing and partitioning a bidding document to obtain a plurality of document blocks; a vector database construction step: vectorizing the document blocks, storing the vectorized document blocks into a vector database, and constructing an index; a knowledge graph construction step: extracting triads from the document blocks, writing the extracted triads into a graph database to construct a knowledge graph, and constructing an index; a mixed retrieval step: carrying out mixed retrieval in combination with the knowledge graph and the vector database; a qualification information extraction step: taking the plurality of document blocks returned by mixed retrieval and the corresponding knowledge graph sub-graphs as contexts, and enabling an Agent agent to perform extraction based on the contexts and an extraction template; and finally, structured qualification item information is output. According to the method, high-quality structured extraction of the qualification information in the bidding document can be realized.

Description

Technical Field

[0001] The present invention relates to the technical field of data information extraction, and particularly relates to an intelligent extraction method for tender qualification information based on a knowledge graph and hybrid retrieval. Background Art

[0002] Tenders usually contain a large amount of complex information, covering key contents such as project background, technical specifications, commercial terms, evaluation criteria, and qualification items. This information is presented in various formats, such as text paragraphs, tables, charts, etc. For example, the technical specifications section will elaborate on the performance parameters of the product in detail, often listed in a table form, and accompanied by relevant text to explain its scope of application and quality requirements. Among them, the qualification item is particularly important, which is the key basis for screening the participating bidders and directly determines whether the bidder has the basic conditions to participate in the project. Accurately extracting the qualification item information can ensure the fairness and compliance of the tendering process, prevent bidders without corresponding capabilities or qualifications from participating, and thus ensure the smooth progress of the project. At the same time, for bidders, they also need to match the qualification items to ensure that they meet the tender conditions to enter the bidding process. If the qualification item information is extracted inaccurately or incompletely, it may lead to inappropriate bidders entering the tendering process, which will not only waste the resources of all parties but also bring many risks to the project implementation.

[0003] Traditional information extraction methods, such as rule-based methods or simple text matching methods, are difficult to effectively handle the complex structure and diverse information formats in tenders. With the increase in the volume of tender business and the increasing complexity of tender content, manual extraction of tender information becomes inefficient and error-prone. Although the development of artificial intelligence technology has provided new possibilities for information extraction, existing methods usually can only extract basic items, and there is still a lack of effective solutions for the extraction of qualification items.

[0004] In recent years, the RAG (Retrieval-Augmented Generation) technology combines the advantages of retrieval and generation and can extract key information from tenders. However, there are a large number of professional terms in the tender field, resulting in a lack of domain knowledge in the large language model in this scenario and insufficient ability to represent and model entities. If only relying on RAG for extraction, it will lead to low accuracy and unstable results. Summary of the Invention

[0005] The purpose of the present invention is to provide an intelligent extraction method for tender qualification information based on a knowledge graph and hybrid retrieval, to achieve high-quality structured extraction of qualification information in tenders and solve the deficiencies of traditional methods and existing technologies in practical applications.

[0006] To achieve the above purpose, an intelligent extraction method for tender qualification information based on a knowledge graph and hybrid retrieval is provided, including the following steps:

[0007] Document preprocessing and chunking steps: Preprocess and chunk the bidding documents to obtain multiple document chunks;

[0008] Vector database construction steps: Vectorize the document chunks, store the corresponding document chunks and vectorized data in the vector database, and build an index for each document chunk;

[0009] Knowledge graph construction steps: Design and extract extraction Prompts for triples based on bidding qualification items, perform few-shot learning (FSL), and add example data for improving extraction effects; then call the large language model to extract triples from the document chunks according to the Prompts, write the extracted triples into the graph database to construct the knowledge graph, and build an index for the document chunks;

[0010] Hybrid retrieval steps: Combine the knowledge graph and the vector database for hybrid retrieval. The hybrid retrieval includes vector similarity retrieval based on the vector database and triple subgraph recall based on the document index of the knowledge graph; Obtain the top k document chunks through vector similarity retrieval, and recall the corresponding triple subgraphs according to the document index; Finally, return the relevant document chunks and their corresponding knowledge graph subgraphs; In the triple subgraph recall process based on the document index, recall the document chunks where the qualification items are located by constructing a qualification item recall template;

[0011] Qualification information extraction steps: Use the multiple document chunks and their corresponding knowledge graph subgraphs returned by the hybrid retrieval as the context, and enable the Agent intelligent body to extract the tender qualification information based on the context and the extraction template; At the same time, enable the Agent intelligent body to review the extraction results of the tender qualification information according to the chain of thought and call the hybrid retrieval multiple times. When calling the hybrid retrieval, enable the Agent intelligent body to extract keywords from the recall template and expand similar words, and then vectorize them for hybrid retrieval; Finally, output the structured qualification item information.

[0012] Furthermore, the preprocessing in the document preprocessing and chunking steps is as follows:

[0013] Convert the bidding documents into Markdown format, retain the document structure information, and form a structured Markdown document;

[0014] The chunking process in the document preprocessing and chunking steps is as follows:

[0015] Chunk the document according to the title hierarchy of the Markdown document, retain the structure information, and obtain multiple document chunks.

[0016] Furthermore, in the vector database construction steps, the document chunks are vectorized through the RAG vector model.

[0017] Further, the triple includes an eligibility item, corresponding conditions, and the score that can be obtained by meeting the conditions.

[0018] Further, the content of the recall template includes: eligibility review, bid evaluation method, compliance review, front schedule, evaluation factors, scoring factors, and scoring criteria.

[0019] Further, the extraction template includes: role item, task item, attention item, and skill item. The role item is the role that the Agent intelligent agent needs to play. The task item is the triple extraction task that the Agent intelligent agent needs to complete. The attention item is the matters that need attention during the process of the Agent intelligent agent extracting triples. The skill item includes context understanding and knowledge graph understanding.

[0020] Advantages of the technical solution:

[0021] 1. Customized extraction of tender eligibility items: Based on the tender qualification items, a customized extraction template is constructed and few-shot learning (FSL) is carried out. The large language model is called to extract triples to construct a knowledge graph, which can accurately extract information closely related to tender qualifications and meet specific business requirements.

[0022] 2. Improve extraction accuracy: Combine the semantic understanding of the RAG vector model and the knowledge extraction ability of the large language model, and at the same time use few-shot learning to optimize the extraction template, which can reduce the deviation that may exist in a single method, thereby improving the accuracy of retrieval results. By integrating the advantages of vector retrieval and knowledge graph, the understanding and extraction accuracy of complex qualification information in tender documents are further improved.

[0023] 3. Improve extraction stability: Utilize the deterministic triple knowledge structure provided by the knowledge graph to assist the large language model in overcoming the instability problem of its output, ensuring the consistency and reliability of the generated content. This hybrid method not only improves the stability of data processing, but also enhances the performance consistency of the system in the face of variable inputs.

[0024] 4. Support dynamic decision-making and intelligent matching: The structured qualification information extraction mechanism driven by the Agent intelligent agent enhances the matching similarity through keyword rewriting and similar word matching technologies, realizes dynamic decision-making based on context, and can perform multiple hybrid retrievals to enhance the recall effect. This method effectively improves the accuracy and flexibility of information matching, adapts to diverse tender formats and contents. Achieve high-quality structured extraction of qualification information in tenders, and solve the deficiencies of traditional methods and existing technologies in practical applications. Description of the Drawings

[0025] Figure 1This is the flowchart of the intelligent extraction method for tender qualification information based on knowledge graph and hybrid retrieval in the embodiments of the present invention;

[0026] Figure 2 This is the schematic diagram of the intelligent extraction method for tender qualification information based on knowledge graph and hybrid retrieval in the embodiments of the present invention. Specific embodiments

[0027] The following is a more detailed description through specific embodiments:

[0028] Embodiment

[0029] An intelligent extraction method for tender qualification information based on knowledge graph and hybrid retrieval is basically as Figure 1 、 Figure 2 shown, and includes the following steps:

[0030] Document preprocessing and chunking step: Preprocess and chunk the tender documents to obtain multiple document chunks; the preprocessing in the document preprocessing and chunking step is:

[0031] First, identify the input file type. Tender documents are generally in pdf format. Then, use the MinerU tool to convert the tender documents into Markdown format, retaining the document structure information to form a structured Markdown document;

[0032] The chunking process in the document preprocessing and chunking step is:

[0033] Chunk the document according to the title hierarchy of the Markdown document, retaining the structure information to obtain multiple document chunks.

[0034] Vector database construction step: Vectorize the document chunks through the RAG vector model, store the corresponding document chunks and vectorized data in the vector database, and build an index for each document chunk;

[0035] Knowledge graph construction step: Design an extraction Prompt for extracting triples based on tender qualification items, perform few-shot learning (FSL), and add example data for improving the extraction effect; then call the large language model to extract triples from the document chunks according to the Prompt, write the extracted triples into the graph database to construct the knowledge graph, and build an index for the document chunks; the triples include qualification items, corresponding conditions, and scores that can be obtained by meeting the conditions.

[0036] The content of the extraction Prompt is as follows:

[0037] "The following provides some texts. The text format is markdown and the table format is html. Extract the qualification certification materials required for bidding, such as certificates, professional title qualifications, performance, and cases, from the text."

[0038] "Extract triples (subject: supporting materials, solutions, services, proofs, cases, certificates, etc., corresponding conditions, scores obtainable for meeting such conditions) for constructing a knowledge graph."

[0039] "The text may span multiple lines and be truncated. You need to pay attention."

[0040] "The text may consist of multiple paragraphs. Each paragraph needs to be extracted to ensure no omission."

[0041] "You need to perform a certain degree of understanding and summarization. The specific names extracted should be as concise and accurate as possible, and you need to summarize them yourself."

[0042] "Note:"

[0043] "1. The format of the first item of the triple: Connect with an auxiliary explanation using '_' and connect with an alternative name using '[]'. Parentheses cannot appear. If they do, use '[]' instead."

[0044] "2. The format of the second item of the triple: If ',' appears, use '、' instead. Parentheses cannot appear. If they do, use '[]' instead."

[0045] "2. When extracting supporting materials, pay attention to what kind of personnel need to possess them, which should be reflected in the first item of the triple."

[0046] "3. Please extract based on the given text. Do not speculate or fabricate information by yourself. You can understand and summarize based on the given text, but cannot fabricate."

[0047] "4. Output in the format of triples (subject: supporting materials, solutions, services, proofs, cases, certificates, etc., corresponding conditions, scores obtainable for meeting such conditions)\nSeparate triples with \n. Do not use serial numbers. Do not fabricate non-existent information. If there is no extractable information, return None."

[0048] "--------------\n"

[0049] Example data is as follows:

[0050] "Example input 1:"

[0051] "The bidder understands the technical standards of this tender project and provides a complete and detailed technical solution, which should at least include the project background, construction ideas, overall architecture, system functions, etc. The technical solution is complete, detailed, specific, reasonable, feasible, and operable and meets the requirements of this project to get 4 points; for each item missing or not meeting the actual situation of this project, 1 point will be deducted until the deduction is completed."

[0052] "Example output 1 Triplets:"

[0053] "(Bidder_Complete and detailed technical solution, the technical solution is complete, detailed, specific, reasonable, feasible, highly operable and in line with this project, 4 points)\n"

[0054] "---------------------\n"

[0055] "Example Input 2:

[0056] "1. The cloud product service provider provided by the bidder has the Trusted Cloud Certification Certificate issued by XXX Information and Communication Research Institute: Cloud Service Security Capability Inspection Certificate - Elastic Cloud Host (Enhanced Level), Cloud Service Security Capability Inspection Certificate - Block Storage. If all are provided, 4 points will be awarded, and 2 points will be deducted for each missing one until the deduction is complete.

[0057] "Example Output 2 Triplets:

[0058] "(The cloud product service provider provided by the bidder_Cloud Service Security Capability Inspection Certificate - Elastic Cloud Host [Enhanced Level], issued by XXX Information and Communication Research Institute, 2 points)\n"

[0059] "(The cloud product service provider provided by the bidder_Cloud Service Security Capability Inspection Certificate - Block Storage, issued by XXX Communication Research Institute, 2 points)\n"

[0060] "---"

[0061] "Example Input 3:

[0062] "3. The bidder needs to send 1 security person in charge, who has the following certificates. 1 point will be awarded for each certificate, with a maximum of 2 points. No points will be awarded if not provided:

[0063] "1) Have the one issued by XXX department (or professional title reform institution): Senior Engineer (Professional Name: Network Information Professional)

[0064] "2) Have the one issued by XX and XX, XX and the Ministry of Information Technology: System Planning and Management Division Certificate

[0065] "Example Output 3 Triplets:

[0066] "(Security person in charge_Senior Engineer_Network Information Professional, issued by XX department [or professional title reform institution], 1 point)\n"

[0067] "(Security person in charge_System Planning and Management Division, issued by XXX Ministry, 1 point)\n"

[0068] Hybrid Retrieval Step: Conduct hybrid retrieval by combining a knowledge graph and a vector database. The hybrid retrieval includes vector similarity retrieval based on the vector database and triple subgraph recall based on the document index of the knowledge graph. Retrieve the top k document chunks through vector similarity retrieval, and recall the corresponding triple subgraphs according to the document index. Finally, return the relevant document chunks and their corresponding knowledge graph subgraphs. In the process of triple subgraph recall based on the document index, recall the document chunks where the qualification items are located by constructing a qualification item recall template. The content of the recall template includes: qualification review, bid evaluation method, compliance review, front schedule, evaluation factors, scoring factors, and scoring criteria.

[0069] Qualification Information Extraction Step: Use the multiple document chunks and their corresponding knowledge graph subgraphs returned by the hybrid retrieval as the context, and enable the Agent intelligent agent to extract the tender qualification information based on the context and the extraction template (i.e., generate the Prompt used by the Agent intelligent agent). At the same time, enable the Agent intelligent agent to review the extraction results of the tender qualification information according to the chain of thought and call the hybrid retrieval multiple times. When calling the hybrid retrieval, enable the Agent intelligent agent to extract keywords from the recall template and expand similar words, and then vectorize them for hybrid retrieval. Finally, output the structured qualification item information.

[0070] The extraction template includes: role item, task item, attention item, and skill item. The role item is the role that the Agent intelligent agent needs to play. The task item is the triple extraction task that the Agent intelligent agent needs to complete. The attention item is the matters that need attention during the process of the Agent intelligent agent extracting triples. The skill item includes context understanding and knowledge graph understanding.

[0071] The content of the extraction template is as follows:

[0072] "## Role"

[0073] "You are very good at combining the [context] information and [knowledge graph] information provided by the prompt template, and can extract qualification certification materials required for bidding, such as certificates, professional titles, performance, cases, etc., and ensure that no information unrelated to the context and knowledge graph will be output."

[0074] "## Task"

[0075] "Extract triples (subject: certification materials, solutions, services, proofs, cases, certificates, etc., corresponding conditions, scores that can be obtained by meeting the conditions)"

[0076] "The text may be across multiple lines and truncated. You need to pay attention."

[0077] "The text may have multiple paragraphs, and each paragraph needs to be extracted to ensure no omission."

[0078] "The provided knowledge graph content may be incomplete, and you need to make reasonable supplements and adjustments based on the context."

[0079] "Note:"

[0080] "1. Format of the first item in the triple: Append supplementary explanations with '_' and aliases with '[]'. Parentheses cannot appear. If they do, replace them with '[]'."

[0081] "2. Format of the second item in the triple: If ',' appears, replace it with '、'. Parentheses cannot appear. If they do, replace them with '[]'."

[0082] "2. When extracting supporting materials, note who needs to possess them, which is reflected in the first item of the triple."

[0083] "3. Please extract based on the given text. Do not speculate or fabricate information on your own. You can understand and summarize based on the given text, but do not fabricate."

[0084] "4. Output in the format of triple (Subject: supporting materials, solutions, services, proofs, cases, certificates, etc., corresponding conditions, scores that can be obtained by meeting these conditions)\nSeparate triples with \n. Do not use serial numbers. Do not fabricate non-existent information. If there is no extractable information, return None."

[0085] "## Skills"

[0086] " Skill 1: Context Understanding"

[0087] "- Accurately understand the information provided by [context], and the context information may be split into multiple chapters."

[0088] "- The text format of the context content is markdown, and the table format is html."

[0089] " Skill 2: Knowledge Graph Understanding"

[0090] "- The triple structure is (Subject: supporting materials, solutions, services, proofs, cases, certificates, etc., corresponding conditions, scores that can be obtained by meeting these conditions)"

[0091] "- Correctly associate the entity names / IDs in the relationship information with the entity information to restore the graph structure."

[0092] "- Use the information expressed by the graph structure as context to assist in generating better answers."

[0093] "---------------------\n"

[0094] "[Context]:"

[0095] "{context}"

[0096] "[Knowledge Graph]:"

[0097] "{graph}"

[0098] The advantages or beneficial effects in the above technical solutions at least include:

[0099] 1. Customized extraction of tender qualification items: Based on tender qualification items, a customized extraction template is constructed and few-shot learning is carried out. The large language model is called to extract triples to construct a knowledge graph, which can accurately extract information closely related to tender qualifications and meet specific business requirements.

[0100] 2. Improve extraction accuracy: Combine the semantic understanding of the RAG vector model and the knowledge extraction ability of the large language model, and at the same time use few-shot learning to optimize the extraction template, which can reduce the possible biases of a single method, thereby improving the accuracy of retrieval results. By integrating the advantages of vector retrieval and knowledge graph, the understanding and extraction accuracy of complex qualification information in tender documents are further improved.

[0101] 3. Improve extraction stability: Use the deterministic triple knowledge structure provided by the knowledge graph to assist the large language model to overcome the instability problem of its output, and ensure the consistency and reliability of the generated content. This hybrid method not only improves the stability of data processing, but also enhances the performance consistency of the system in the face of variable inputs.

[0102] 4. Support dynamic decision-making and intelligent matching:

[0103] The structured qualification information extraction mechanism driven by the Agent intelligent agent enhances the matching similarity through keyword rewriting and similar word matching technologies, realizes dynamic decision-making based on context, and can perform multiple hybrid retrievals to enhance the recall effect. This method effectively improves the accuracy and flexibility of information matching and adapts to diverse tender formats and contents.

[0104] The above are only embodiments of the present invention. Specific structures and characteristics and other common knowledge in the art are not described in detail herein. Those of ordinary skill in the art know all the common general technical knowledge in the technical field to which the invention pertains before the filing date or the priority date, are able to obtain all the prior art in this field, and have the ability to apply the conventional experimental means before this date. Those of ordinary skill in the art can, under the inspiration given in this application, combine their own abilities to improve and implement this solution. Some typical well-known structures or well-known methods should not become an obstacle for those of ordinary skill in the art to implement this application. It should be noted that for those skilled in the art, without departing from the structure of the present invention, several deformations and improvements can also be made, and these should also be regarded as the protection scope of the present invention, and these will not affect the implementation effect of the present invention and the practicality of the patent. The protection scope claimed in this application should be based on the content of its claims, and the specific implementation manners and other records in the specification can be used to interpret the content of the claims.

Claims

1. An intelligent extraction method for tender qualification information based on a knowledge graph and hybrid retrieval, characterized in that It includes the following steps: Document preprocessing and chunking step: Preprocess and chunk the tender documents to obtain multiple document chunks; Vector database construction step: Vectorize the document chunks, store the corresponding document chunks and vectorized data in the vector database, and build an index for each document chunk; Knowledge graph construction step: Design and extract the extraction Prompt for triples based on tender qualification items, perform few-shot learning (FSL), and add example data for improving the extraction effect; then call the large language model to extract triples from the document chunks according to the Prompt, write the extracted triples into the graph database to construct the knowledge graph, and build an index for the document chunks; Hybrid retrieval step: Combine the knowledge graph and the vector database for hybrid retrieval. The hybrid retrieval includes vector similarity retrieval based on the vector database and triple subgraph recall based on the document index of the knowledge graph; Obtain the top k document chunks through vector similarity retrieval, and recall the corresponding triple subgraph according to the document index; Finally, return the relevant document chunks and their corresponding knowledge graph subgraphs; In the triple subgraph recall process based on the document index, recall the document chunks where the qualification items are located by constructing a qualification item recall template; Qualification information extraction step: Use the multiple document chunks and their corresponding knowledge graph subgraphs returned by the hybrid retrieval as the context, and enable the Agent intelligent agent to extract the tender qualification information based on the context and the extraction template; At the same time, enable the Agent intelligent agent to review the extraction results of the tender qualification information according to the chain of thought and call the hybrid retrieval multiple times. When calling the hybrid retrieval, enable the Agent intelligent agent to extract keywords from the recall template and expand similar words, and then vectorize them for hybrid retrieval; Finally, output the structured qualification item information.

2. The intelligent extraction method of tender qualification information based on knowledge graph and hybrid retrieval according to claim 1, wherein: The preprocessing in the document preprocessing and chunking step is: Convert the tender documents into Markdown format, retain the document structure information, and form a structured Markdown document; The chunking process in the document preprocessing and chunking step is: Chunk the document according to the title hierarchy of the Markdown document, retain the structure information, and obtain multiple document chunks.

3. The intelligent extraction method of tender qualification information based on knowledge graph and hybrid retrieval according to claim 2, characterized in that: In the vector database construction step, the document chunks are vectorized through the RAG vector model.

4. The intelligent extraction method of tender qualification information based on knowledge graph and hybrid retrieval according to claim 1, characterized in that: The triples include qualification items, corresponding conditions, and scores that can be obtained by meeting the conditions.

5. The intelligent extraction method of tender qualification information based on knowledge graph and hybrid retrieval according to claim 4, characterized in that: The content of the recall template includes: qualification review, bid evaluation method, compliance review, front schedule, evaluation factors, scoring factors, and scoring criteria.

6. The intelligent extraction method for tender qualification information based on knowledge graph and hybrid retrieval according to claim 5, wherein: The extraction template includes: role item, task item, attention item, and skill item. The role item is the role that the Agent intelligent agent needs to play, the task item is the triple extraction task that the Agent intelligent agent needs to complete, the attention item is the matters that the Agent intelligent agent needs to pay attention to during the triple extraction process, and the skill item includes context understanding and knowledge graph understanding.

Images