Bidding knowledge graph construction method and system based on OCR and NLP

Abstract

The invention discloses a bidding and tendering knowledge graph construction method and system based on OCR and NLP. The method comprises the following steps: collecting a multi-source bidding document, and converting an unstructured document into text data by utilizing OCR (Optical Character Recognition); key entities, relations and attributes are extracted through the NLP technology; constructing a knowledge graph with entities as nodes and relationships as edges, and generating entity node vectors; analyzing the bidding and tendering file, extracting text demands such as qualification requirements and scoring standards, and converting the text demands into demand vectors; performing semantic matching on the demand vector and the entity node vector, and calculating the similarity; and extracting high-similarity entities and attribute relationships thereof from the knowledge graph based on a matching result, and automatically filling and generating bidding document core chapters according to a preset template. The system can realize structured storage and intelligent reuse of bidding and tendering knowledge, and improves the bidding document compiling efficiency and quality.

Description

[0001] Method Domain This application relates to the field of knowledge graph construction technology, and in particular to a bidding knowledge graph construction method and system based on OCR and NLP.

[0002] Background Method In bidding processes, bidders need to prepare a large number of qualification documents, such as past company performance records, personnel qualification certificates, and equipment information. Currently, companies typically manage this information and prepare their bids using the following methods: 1. Unstructured Storage and Management: A large number of historical performance records and personnel qualification certificates are scattered and stored as scanned copies (PDFs, images) or unstructured Word / Excel documents. When bidding is required, staff have to manually sift through massive amounts of documents to find information relevant to the bidding requirements, which is extremely inefficient and prone to missing crucial information.

[0003] 2. Manual matching and bid document preparation: After reading the bidding documents, professionals manually judge the bidding requirements and select, copy, and paste relevant content from historical data to prepare the bid documents. This process has the following significant problems: (1) Inaccurate matching: Manual matching is highly subjective and it is difficult to ensure that all bidding keywords and requirements are accurately responded to, which may lead to the rejection of the bid. (2) Inefficient: Each bid requires repeated data search and content editing, which is time-consuming and laborious. (3) Poor content consistency: Manual copying and pasting is prone to errors, and the quality of bid documents prepared by different people varies.

[0004] Therefore, the core technical problem of existing technologies is: how to automatically transform scattered and unstructured enterprise historical data into structured knowledge that can be accurately and efficiently retrieved and utilized by computers, and how to achieve intelligent and highly accurate semantic matching with bidding requirements, thereby automatically generating high-quality bid content to solve the problems of low efficiency, large errors, and unstable quality under manual methods. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this application aims to provide a method and system for constructing a knowledge graph for bidding and tendering based on OCR and NLP. This system can automatically transform scattered and unstructured historical enterprise data into structured knowledge that can be accurately and efficiently retrieved and utilized by computers. It also enables intelligent and highly accurate semantic matching with bidding and tendering requirements, thereby automatically generating high-quality tender documents. This solves the problems of low efficiency, large errors, and unstable quality associated with manual methods.

[0006] To achieve the above objectives, this application adopts the following method: This application provides a method for constructing a bidding knowledge graph based on OCR and NLP, the method comprising: S1 collects bidding-related documents from multiple data sources inside and outside the enterprise. These documents include: historical winning bid notices, contract documents, scanned copies of personnel qualification certificates, scanned copies of company qualifications, and project acceptance reports. The OCR method is used to convert the images and scanned PDF documents into processable text data. S2, based on NLP methods, extracts information from the preprocessed text, including: identifying and extracting key entities, which include: project name, personnel name, qualification certificate number, project amount, owner unit, and project duration; identifying the relationships between key entities; and extracting attributes for each key entity. S3 stores the key entities, attributes, and relationships extracted in S2 into a graph database in the form of a graph structure to construct a bidding knowledge graph. Key entities serve as nodes in the knowledge graph, relationships serve as edges, and attributes serve as attributes of nodes, thus obtaining entity node vectors. S4. Upload the bidding documents and use NLP methods to parse them, including: extracting the textual requirements of qualification requirements, method parameters and scoring criteria from the bidding documents; and converting the extracted textual requirements into requirement vectors through a pre-trained semantic model. The requirement vectors are used to represent the semantic information of the textual requirements. S5. Perform semantic matching and similarity calculation on the attributes of all nodes in the entity node vector for each demand vector to filter out the items with the highest similarity and obtain the matching results. S6. Based on the matching results of S5, extract the key entities with high similarity, as well as the attributes and relationships of the key entities, from the knowledge graph, and fill them in according to the preset tender template to generate the core chapters of the tender.

[0007] As a preferred technical solution, in S1, the collection of bidding-related documents from multiple internal and external data sources includes: Deploy a lightweight agent program to periodically scan specified shared directories on the enterprise file server, set up a scheduled task to automatically traverse these directories and identify newly added or modified files; By providing corresponding API adapters, it can interface with existing enterprise content management systems, OA systems, or project management systems to retrieve structured data such as archived contracts and project acceptance reports; Based on the structured data, the relevant tables are queried through JDBC / ODBC connections to obtain metadata such as project number, name, and amount, and then an association is established with the corresponding unstructured documents. Configure a web crawler module to periodically access the designated public resource trading center website, and capture the public information of bidding projects by simulating login or parsing public pages, and automatically download the PDF attachments of the public information to the bidding document library; It integrates with the enterprise's email system, monitors specific email accounts, parses email subjects and content, downloads email attachments, and categorizes and stores the attachments in a bidding document library.

[0008] As a preferred technical solution, in S1, the step of using the OCR method to convert images and scanned PDF documents into processable text data includes: The scanned PDF or image files are converted into a unified image format for processing; image tilt is detected and corrected using algorithms such as Hough transform; noise generated during scanning is removed using filtering algorithms; histogram equalization is used to improve the contrast between text and background; and color or grayscale images are converted into black and white binary images to highlight text outlines. The preprocessed image is input into the OCR engine, preserving the coordinate position of each text segment and each character recognized by the OCR in the original image; basic cleaning is performed on the recognized text, and garbled characters caused by unclear images are removed by rule filtering method; each original document is converted into a standardized structured text file containing hierarchical structure and text coordinate information, and the structured text file is used as the input for step S2.

[0009] As a preferred technical solution, in S2, the information extraction from the preprocessed text based on the NLP method includes: A sequence labeling model based on a pre-trained language is constructed, and a classification layer is connected to the output layer of the sequence labeling model. The classification layer is used to predict a label for each word in the structured text file. The sequence labeling model is initially trained using a general domain dataset, and then fine-tuned using a small-scale dataset manually annotated from bidding documents. A domain-specific dictionary is constructed, and word features from the domain-specific dictionary are integrated into the sequence labeling model. The text blocks of the structured text file output in step S1 are input sequentially into the trained sequence labeling model. The sequence labeling model outputs a labeled sequence, merges consecutive characters into complete key entities based on the labels, and records the position of the key entities in the original text. The core set of relationships in the bidding and tendering field is predefined; the text of two key entities is input, the text is encoded, and the tag information is embedded in the position of the key entities. The classification layer determines which predefined relationship the two key entities belong to, or whether there is no relationship; then, the coordinate information of the key entities and syntactic dependency analysis are used to determine whether the two key entities are in the same sentence or adjacent text blocks, and pattern matching rules are written to verify the relationship between the two key entities. For semi-fixed format documents, define attribute extraction templates; use a pre-trained machine reading comprehension model to extract attributes through questioning; and normalize the extracted attributes.

[0010] As a preferred technical solution, in S3, nodes and their attributes include: Project, with attributes such as project name, contract amount, commencement date, completion date, and owner; Personnel, with attributes such as name, position, and years of service; Company, with attributes such as company name, registered capital, and qualification level; Qualification certificate, with attributes such as certificate number, issuing authority, and validity period; Edges include: the role of the connecting personnel and the project, the contracting of the connecting company and the project, the qualifications of the connecting personnel and the qualification certificate, and the affiliation of the connecting personnel and the company. The construction of the bidding knowledge graph includes: converting the list of key entities and attributes output by S2 into node insertion operations in the graph database; and connecting the inserted nodes according to the relationship list output by S2. The process of obtaining entity node vectors includes: encoding relevant text of key entities using a pre-trained semantic model, inputting attributes and relationships, and obtaining entity node vectors.

[0011] As a preferred technical solution, S4 also includes: converting the bidding documents into processable text data using an OCR method; The method of parsing bidding documents using NLP includes: using a rule-based dependency model or machine learning model to structurally partition the bidding documents; using regular expressions to match fixed-format clauses in the bidding documents; using a sequence labeling model to identify the textual requirements of qualification requirements, method parameters, and scoring criteria in the clauses; merging the extracted fragmented textual requirements into complete semantic units; constructing a semantic model based on the complete semantic units; fine-tuning the semantic model using a bidding document corpus to better understand technical terms; and outputting a requirement vector from the semantic model.

[0012] As a preferred technical solution, in step S5, the semantic matching and similarity calculation of the attributes of all nodes in the entity node vector with each demand vector to filter out the projects with the highest similarity and obtain the matching results includes: obtaining the demand vector list generated in step S4, where each demand vector is associated with a demand vector type to limit the range of key entity types for retrieval; obtaining similar key entities of the demand vectors using an approximate nearest neighbor search method based on the bidding knowledge graph, and calculating the similarity of the attributes of the nodes of the demand vectors and similar key entities using a cosine similarity algorithm; verifying whether similar key entities meet the constraints through a graph query language and obtaining the graph relationship matching degree; calculating a comprehensive score using a weighted algorithm based on the similarity and graph relationship matching degree, retaining key entities with a comprehensive score greater than the comprehensive score threshold, and sorting the key entities in descending order according to the comprehensive score; outputting the matched key entities and the entity node vectors corresponding to the key entities; and outputting the matched projects if the percentage of the entity node vectors corresponding to the matched key entities in the total entity node vectors of the projects exceeds a percentage threshold.

[0013] As a preferred technical solution, in step S6, the extraction of highly similar key entities, their attributes, and relationships from the knowledge graph, and the filling of these entities according to a preset bid template to generate the core chapters of the bid, includes: designing the layout and static content of the bid in Microsoft Word, and then inserting special markers at the locations where dynamic filling is required; performing a detailed graph query based on the key entity IDs returned by the matching results in step S5 to obtain complete attribute chains and relationship chains, and obtaining structured matching data based on the key entities, attribute chains, and relationship chains; synthesizing the structured matching data into fluent natural language, and inserting it into the bid document according to the special markers to generate the core chapters of the bid.

[0014] This application also provides a bidding knowledge graph construction system based on OCR and NLP, the system comprising: The data acquisition and OCR processing unit is used to collect bidding-related documents from multiple data sources and use OCR methods to convert unstructured documents into processable text data. The information extraction unit is used to identify and extract key entities, relationships between entities, and entity attributes from text data based on NLP technology. The knowledge graph construction unit is used to store the extracted entities, attributes and relationships in the form of a graph database, construct the bidding knowledge graph, and generate entity node vectors; The tender document parsing unit is used to parse the uploaded tender documents, extract the text requirements, and transform them into a requirement vector. The semantic matching unit is used to calculate the similarity between the demand vector and the entity node vector to obtain the matching result. The tender document generation unit is used to extract relevant information from the knowledge graph based on the matching results and automatically generate the core chapters of the tender document according to the tender document template.

[0015] Compared with the prior art, the beneficial effects of this application are as follows: This application systematically extracts key information from scattered, unstructured scanned documents and files. This information is stored in a graph structure, constructing a bidding knowledge graph. This transforms a company's historical performance, personnel qualifications, equipment information, etc., from isolated, difficult-to-retrieve documents into an interconnected, structured knowledge network that can be understood and processed by computers. When querying a specific qualification or project experience, there's no need to manually sift through massive amounts of documents; efficient and accurate related queries and tracing can be performed through the knowledge graph, fundamentally changing the data storage and management model.

[0016] Traditional manual matching methods are highly subjective and easily overlook keywords and requirements in the bidding documents. This method utilizes NLP technology to transform the textual requirements of the bidding documents into requirement vectors that represent their semantic information. The semantic similarity of this requirement vector is then calculated between it and the vectorized attributes of all entity nodes in the knowledge graph. This deep learning-based semantic matching can understand the true intent behind the requirements, rather than just keyword matching, thus greatly improving the accuracy and comprehensiveness of the matching and effectively avoiding the risk of bid rejection due to improper matching.

[0017] After achieving precise matching, this application can automatically extract key entities such as projects, personnel, and qualifications that highly match the bidding requirements, along with their detailed attributes and relationships, from the knowledge graph based on a pre-set bid template, and populate the corresponding sections of the bid document. This frees staff from tedious and repetitive tasks of searching, copying, and pasting data, reducing work that would otherwise take days or even weeks to hours, greatly improving bid response speed and work efficiency.

[0018] Because information extraction is based on a unified NLP model, it ensures that the information extracted from the raw data is standardized and compliant. Furthermore, the bid document generation is based on a pre-set, high-quality template for automated filling, avoiding problems such as inconsistent formatting, data errors, and contradictory statements that are common with manual writing. This results in accurate, standardized, and consistently high-quality core bid documents, without significant differences due to different writers, thus enhancing the professionalism and overall quality of the company's bid documents. Detailed Implementation

[0019] To enable those skilled in the art to better understand the present application, the method scheme in the specific embodiments of the present application will be clearly and completely described below.

[0020] This application provides a method for constructing a bidding knowledge graph based on OCR and NLP, the method comprising: S1 collects bidding-related documents from multiple internal and external data sources, including: historical winning bid notices, contract documents, scanned copies of personnel qualification certificates, scanned copies of company qualifications, and project acceptance reports. It uses OCR methods to convert images and scanned PDF documents into processable text data.

[0021] S2, based on NLP methods, extracts information from preprocessed text, including: identifying and extracting key entities, such as project name, personnel names, qualification certificate numbers, project amount, owner, and project duration; identifying the relationships between key entities; and extracting attributes for each key entity.

[0022] S3 stores the key entities, attributes, and relationships extracted in S2 into a graph database in the form of a graph structure to construct a bidding knowledge graph. In this graph, key entities serve as nodes, relationships serve as edges, and attributes serve as attributes of nodes, thus obtaining entity node vectors.

[0023] S4. Upload the bidding documents and parse them using NLP methods, including extracting the textual requirements for qualification, method parameters, and scoring criteria. The extracted textual requirements are then transformed into requirement vectors using a pre-trained semantic model. These requirement vectors represent the semantic information of the textual requirements.

[0024] S5 performs semantic matching and similarity calculation on the attributes of all nodes in the entity node vector for each demand vector to filter out the items with the highest similarity and obtain the matching results.

[0025] S6. Based on the matching results of S5, extract the key entities with high similarity, as well as the attributes and relationships of the key entities, from the knowledge graph, and fill them in according to the preset tender template to generate the core chapters of the tender.

[0026] This application systematically extracts key information from scattered, unstructured scanned documents and files. This information is stored in a graph structure, constructing a bidding knowledge graph. This transforms a company's historical performance, personnel qualifications, equipment information, etc., from isolated, difficult-to-retrieve documents into an interconnected, structured knowledge network that can be understood and processed by computers. When querying a specific qualification or project experience, there's no need to manually sift through massive amounts of documents; efficient and accurate related queries and tracing can be performed through the knowledge graph, fundamentally changing the data storage and management model.

[0027] Traditional manual matching methods are highly subjective and easily overlook keywords and requirements in the bidding documents. This method utilizes NLP technology to transform the textual requirements of the bidding documents into requirement vectors that represent their semantic information. The semantic similarity of this requirement vector is then calculated between it and the vectorized attributes of all entity nodes in the knowledge graph. This deep learning-based semantic matching can understand the true intent behind the requirements, rather than just keyword matching, thus greatly improving the accuracy and comprehensiveness of the matching and effectively avoiding the risk of bid rejection due to improper matching.

[0028] After achieving precise matching, this application can automatically extract key entities such as projects, personnel, and qualifications that highly match the bidding requirements, along with their detailed attributes and relationships, from the knowledge graph based on a pre-set bid template, and populate the corresponding sections of the bid document. This frees staff from tedious and repetitive tasks of searching, copying, and pasting data, reducing work that would otherwise take days or even weeks to hours, greatly improving bid response speed and work efficiency.

[0029] Because information extraction is based on a unified NLP model, it ensures that the information extracted from the raw data is standardized and compliant. Furthermore, the bid document generation is based on a pre-set, high-quality template for automated filling, avoiding problems such as inconsistent formatting, data errors, and contradictory statements that are common with manual writing. This results in accurate, standardized, and consistently high-quality core bid documents, without significant differences due to different writers, thus enhancing the professionalism and overall quality of the company's bid documents.

[0030] Furthermore, in S1, the bidding-related documents collected from multiple internal and external data sources include: Deploy a lightweight agent program to periodically scan specified shared directories on the enterprise file server. Set up a scheduled task to automatically traverse these directories and identify newly added or modified files. In this application, the shared directory is \\NAS\bid documents\. The agent program automatically traverses these directories at 2 AM daily to identify newly added or modified files. Supported file formats include, but are not limited to: .pdf, .jpg, .png, .tiff, .docx, .xlsx.

[0031] By providing a corresponding API adapter, it can interface with existing enterprise content management systems, OA systems, or project management systems to retrieve structured data such as archived contracts and project acceptance reports. In this application, the API adapter is a RESTful API call.

[0032] Based on the structured data, relevant tables are queried via JDBC / ODBC connections to obtain metadata such as project number, name, and amount, and then associated with the corresponding unstructured documents. In this application, the structured data is stored in MySQL or Oracle, and the unstructured documents include scanned copies of contracts.

[0033] Configure a web crawler module to periodically access the designated public resource trading center website, and capture the public information of bidding projects by simulating login or parsing public pages, and automatically download the PDF attachments of the public information to the bidding document library.

[0034] It integrates with the enterprise's email system, monitors specific email accounts, parses email subjects and content, downloads email attachments, and categorizes and stores the attachments in a bidding document library.

[0035] Furthermore, before OCR recognition, the acquired documents are preprocessed to improve the accuracy of subsequent OCR and text processing. In S1, the conversion of images and scanned PDF documents into processable text data using OCR methods includes: converting scanned PDF or image files into a unified image format for processing; detecting and correcting image tilt using algorithms such as Hough transform; removing noise generated during scanning using filtering algorithms; improving the contrast between text and background using histogram equalization; and converting color or grayscale images into black-and-white binary images to highlight text outlines.

[0036] In step S1, the preprocessed image is input into the OCR engine, preserving the coordinate positions of each text segment and each character recognized by the OCR within the original image. In this application, the OCR engine used is the open-source engine Tesseract, or a commercial / cloud service engine such as Google Cloud Vision API, Baidu OCR, or Alibaba Cloud OCR. The coordinate positions of each recognized text segment and each character within the original image provide crucial spatial context for relation extraction in subsequent NLP steps. Basic cleaning is performed on the recognized text, removing garbled characters caused by image blurriness using rule-based filtering methods. Each original document is converted into a standardized structured text file (JSON format) containing hierarchical structure and text coordinate information, and this structured text file is used as input for step S2.

[0037] Furthermore, in S2, the information extraction from the preprocessed text based on the NLP method includes: A sequence labeling model based on a pre-trained language is constructed. A classification layer is connected to the output layer of the sequence labeling model. The classification layer is used to predict a label for each word in the structured text file. In this application, the sequence labeling model based on the pre-trained language uses BERT, RoBERTa, etc., the classification layer is such as a CRF layer or a Softmax layer, and the labels adopt the BIO (Begin, Inside, Outside) or BIOES (Begin, Inside, Outside, End, Single) labeling system. For example, the labeling result of the sentence "Project Manager Zhang San is responsible for the G15 Expressway Project." is: Project / O Item / O Manager / O Zhang / B-PER Name / I-PER Responsible / OG / B-PROJ 15 / I-PROJ Expressway / I-PROJ Highway / I-PROJ Project / I-PROJ Item / I-PROJ.

[0038] The sequence labeling model was initially trained using a general domain dataset, and then fine-tuned using a small-scale dataset manually annotated from bidding documents. A domain-specific dictionary was constructed, and word features from this dictionary were integrated into the sequence labeling model to further improve the accuracy of identifying professional entities. In this application, the domain-specific dictionary includes dictionaries such as common job titles, qualification certificate prefixes such as 'Construction and Installation B Certificate', and common project types.

[0039] The text blocks of the structured text file output in step S1 are input sequentially into the trained sequence labeling model. The sequence labeling model outputs a labeled sequence, merges consecutive characters into complete key entities based on the labels, and records the position of the key entities in the original text.

[0040] This application predefines the core set of relationships within the bidding and tendering field. In this application, the set of relationships includes: role (connecting personnel and projects); contracting (connecting companies and projects); affiliation (connecting personnel and companies); and qualifications (connecting personnel and qualification certificates).

[0041] The process involves inputting text containing two key entities, encoding the text, and embedding markers at the locations of the key entities. A classification layer determines whether the two key entities belong to a predefined relationship or have no relationship. Then, using the coordinate information of the key entities and syntactic dependency analysis, it is determined whether the two key entities are in the same sentence or adjacent text blocks. Pattern matching rules are then written, and the relationship between the two key entities is validated based on these rules. This application employs a preliminary screening process to obtain candidate relationship pairs; subsequently, high-confidence results are further validated using rules to ensure the accuracy of the core relationships.

[0042] For semi-fixed format documents, define an attribute extraction template. Exemplarily, for project entities, define the pattern: "Contract amount: " {value}, "Start date: " {value}, "Completion date: " {value}. The system searches for these patterns in the surrounding text of the identified project entity and extracts subsequent strings such as numerical values and dates as attribute values.

[0043] Use a pre-trained machine reading comprehension model to extract attributes by asking questions. Normalize the extracted attributes. Exemplarily, for the text "G15 Highway Project, contract amount is 580 million yuan, started in January 2020...", for the project entity, ask the question: "What is the contract amount of this project?". The model will locate the answer "580 million yuan" from the text. Convert "Five hundred and eighty million", "580 million", "580,000,000 yuan" to the standard numerical format 580000000 and unify the currency unit to "yuan". Convert "January 2020", "2020 / 01", "Jan 2020" to the ISO standard format 2020-01 or timestamp.

[0044] Furthermore, in S3, the nodes and their attributes include: Project, and the attributes of the project include: project name, contract amount, start date, completion date, owner unit. Personnel, and the attributes of personnel include name, position, years of work experience. Company, and the attributes of the company include: company name, registered capital, qualification level. Qualification certificate, and the attributes of the qualification certificate include: certificate number, issuing agency, validity period. The edges include: the role connecting personnel and project, the construction connecting company and project, the qualification owning connecting personnel and qualification certificate, and the subordination connecting personnel and company.

[0045] The construction of the bidding knowledge graph includes: converting the list of key entities and attributes output by S2 into node insertion operations in the graph database. Connect the inserted nodes according to the relationship list output by S2. In this application, the graph database such as Neo4j (mature, supports Cypher query language), Nebula Graph (distributed, high performance), ArangoDB (multi-model support).

[0046] The process of obtaining entity node vectors includes: encoding the relevant text of key entities using a pre-trained semantic model, inputting attributes and relationships, and obtaining entity node vectors. In this application, for project nodes: the project name + owner unit + project type + contract amount is concatenated. For personnel nodes: the name + position + company + qualification certificate is concatenated. For example, an entity node vector might be: G15 Expressway Project, XX Provincial Transportation Construction Group, Expressway Construction General Contracting Contract Amount: 580 million yuan; Zhang San, Project Manager, ABC Construction First-Class Registered Construction Engineer (No. JZ123456). In this application, entity IDs are stored in a graph database, and the mapping from entity IDs to entity node vectors is stored in a vector database. Queries are performed by linking the two databases through the ID.

[0047] Furthermore, S4 also includes: converting the tender documents into processable text data using the same OCR method as in step S1.

[0048] The method of parsing bidding documents using NLP includes: using a rule-based dependency model or a machine learning model to structurally partition the bidding documents. In this application, the machine learning model is such as LayoutLM. The bidding documents are divided into "qualification requirements", "technical parameters", and "scoring criteria".

[0049] Regular expressions are used to match fixed-format clauses in bidding documents. Sequence labeling models are used to identify textual requirements for eligibility, method parameters, and scoring criteria within the clauses. Extracted fragmented textual requirements are then merged into complete semantic units.

[0050] A semantic model is constructed based on complete semantic units, and fine-tuned using a corpus of bidding documents to better understand technical terms (such as "qualifications" and "performance"). The semantic model outputs a requirement vector. In this application, the semantic model uses the Cohere Embed API or OpenAItext-embedding-3-large.

[0051] Furthermore, in S5, the semantic matching and similarity calculation of the attributes of each demand vector with all nodes in the entity node vector are performed to filter out the items with the highest similarity and obtain the matching results, including: Obtain the list of demand vectors generated in step S4. Each demand vector is associated with a type of demand vector (such as qualification / technology / personnel), which is used to limit the range of key entity types for retrieval.

[0052] Based on the bidding knowledge graph, a near nearest neighbor search method is used to obtain similar key entities to the demand vector. A cosine similarity algorithm is then used to calculate the similarity of attributes between the nodes of the demand vector and the similar key entities. A graph query language is used to verify whether the similar key entities meet the constraints (such as time range and role relationships), and the graph relationship matching degree is obtained.

[0053] Based on similarity and graph relationship matching degree, a weighted algorithm is used to calculate a comprehensive score. Key entities with a comprehensive score greater than a threshold are retained, and the key entities are sorted in descending order of their comprehensive scores. In this application, the similarity weight is set to 0.7, the graph relationship matching degree weight is set to 0.3, and the comprehensive score threshold is set to 0.75. Finally, deduplication is performed: results pointing to the same actual entity but different nodes are merged (such as the roles of the same person in different projects).

[0054] Output the matching key entities and their corresponding entity node vectors. If the percentage of the entity node vector corresponding to the matching key entity exceeds a certain threshold, then output the matching items.

[0055] Furthermore, in S6, the key entities with high similarity, their attributes, and relationships are extracted from the knowledge graph and filled in according to a preset tender template to generate the core chapters of the tender document, including: Design the layout and static content of the tender document in Microsoft Word, and then insert special markers in the locations where dynamic filling is required. Based on the key entity IDs returned by the matching results in step S5, perform a detailed graph query to obtain the complete attribute chain and relationship chain. Obtain structured matching data based on the key entities, attribute chains, and relationship chains. Synthesize the structured matching data into fluent natural language and insert it into the tender document according to the special markers to generate the core chapters of the tender document.

[0056] This application also provides a bidding knowledge graph construction system based on OCR and NLP, the system including: a data acquisition and OCR processing unit, an information extraction unit, a knowledge graph construction unit, a bidding document parsing unit, a semantic matching unit, and a bid document generation unit.

[0057] The data acquisition and OCR processing unit is used to collect bidding-related documents from multiple data sources and use OCR methods to convert unstructured documents into processable text data.

[0058] The information extraction unit is used to identify and extract key entities, relationships between entities, and entity attributes from text data based on NLP technology.

[0059] The knowledge graph construction unit is used to store the extracted entities, attributes, and relationships in a graph database in the form of a graph structure, construct a bidding and tendering knowledge graph, and generate entity node vectors.

[0060] The bidding and tendering document parsing unit is used to parse the uploaded bidding and tendering documents, extract text requirements, and convert them into requirement vectors.

[0061] The semantic matching unit is used to calculate the similarity between the requirement vector and the entity node vector to obtain a matching result.

[0062] The tender document generation unit is used to extract relevant information from the knowledge graph according to the matching result and automatically generate the core chapters of the tender document according to the tender document template.

[0063] The following provides an application scenario of the above system: A construction company "ABC Construction" needs to bid for a project of "Highway No. 3 Section from XX City to YY City".

[0064] ABC Construction uploads scanned copies of all project contracts, completion materials, and first-class constructor certificates of employees in the past 5 years to the system.

[0065] The system converts hundreds of PDF scanned copies into text through the OCR module.

[0066] The NLP module runs: From the text of a "G15 Highway Project Contract", the entities are extracted: Project name: G15 Highway, Project manager: Zhang San, Contract amount: 580 million yuan.

[0067] The relationships are extracted: <Zhang San, serves as the project manager of, G15 Highway>, <ABC Construction, undertakes the construction of, G15 Highway>.

[0068] Attributes are added to the G15 Highway node: {Type: Highway, Mileage: 50 km, Completion date date: 2020-12}.

[0069] All data is stored in a graph database such as Neo4j to form an enterprise-specific bidding and tendering knowledge graph.

[0070] The project manager uploads the "Highway No. 3 Section Bidding Document from XX to YY.pdf" to the system.

[0071] The system analyzes the bidding document and discovers the key requirements: "The bidder must have completed at least one highway project with a mileage of not less than 30 km in the past 5 years... The project manager must have more than 10 years of experience in large-scale road and bridge projects...".

[0072] The system performs semantic matching: "Highway Project" is matched with the project type attribute in the knowledge graph, finding multiple projects such as G15 Expressway (50km>30km, matching). "Project Manager...10+ years of experience" is matched with the attribute of the Zhang San node, and the system finds that Zhang San's years of experience are 15 years, and successfully associated with the G15 Expressway project, resulting in a successful match.

[0073] The system automatically generates the "Company Performance Table" section of the tender document, the content of which is as follows: Performance table of similar projects in the past five years Project Name: G15 Expressway Project Contract amount: 580 million yuan Project Overview: General contracting for the construction of a 50-kilometer-long expressway. Completion date: December 2020 Role in this project: Contractor Resume of the proposed project manager Name: Zhang San Relevant experience: Served as project manager for the G15 expressway, which was completed in 2020...

[0074] The generated content is automatically filled into the designated location in the tender document template, and staff only need to review and make minor adjustments.

[0075] It should be noted that the terms "first," "second," and similar terms used in this application specification and claims do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Similarly, "a" or "one," and similar terms do not indicate a quantity limitation, but rather indicate the presence of at least one. "A plurality" or "several" indicates at least two. Unless otherwise stated, terms such as "front," "back," "left," "right," "lower," and / or "upper" are for illustrative purposes only and are not limited to a location or spatial orientation. Terms such as "comprising" or "including" indicate that the elements or objects preceding "comprising" encompass the elements or objects listed following "comprising" or "including" and their equivalents, and do not exclude other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect.

[0076] The singular forms “a,” “the,” and “the” used in this application specification and appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0077] It should be understood that those skilled in the art can make improvements or modifications based on the above description, and all such improvements and modifications should fall within the scope of protection of the appended claims.

Claims

1. A method for constructing a knowledge graph for bidding and tendering based on OCR and NLP, characterized in that, The method includes: S1 collects bidding-related documents from multiple data sources inside and outside the enterprise. These documents include: historical winning bid notices, contract documents, scanned copies of personnel qualification certificates, scanned copies of company qualifications, and project acceptance reports. The OCR method is used to convert the images and scanned PDF documents into processable text data. S2, based on NLP methods, extracts information from the preprocessed text, including: identifying and extracting key entities, which include: project name, personnel name, qualification certificate number, project amount, owner unit, and project duration; identifying the relationships between key entities; and extracting attributes for each key entity. S3 stores the key entities, attributes, and relationships extracted in S2 into a graph database in the form of a graph structure to construct a bidding knowledge graph. Key entities serve as nodes in the knowledge graph, relationships serve as edges, and attributes serve as attributes of nodes, thus obtaining entity node vectors. S4. Upload the bidding documents and use NLP methods to parse them, including: extracting the textual requirements of qualification requirements, method parameters and scoring criteria from the bidding documents; and converting the extracted textual requirements into requirement vectors through a pre-trained semantic model. The requirement vectors are used to represent the semantic information of the textual requirements. S5. Perform semantic matching and similarity calculation on the attributes of all nodes in the entity node vector for each demand vector to filter out the items with the highest similarity and obtain the matching results. S6. Based on the matching results of S5, extract the key entities with high similarity, as well as the attributes and relationships of the key entities, from the knowledge graph, and fill them in according to the preset tender template to generate the core chapters of the tender.

2. The method for constructing a bidding knowledge graph based on OCR and NLP according to claim 1, characterized in that, In S1, the bidding-related documents collected from multiple internal and external data sources include: Deploy a lightweight agent program to periodically scan specified shared directories on the enterprise file server, set up a scheduled task to automatically traverse these directories and identify newly added or modified files; By providing corresponding API adapters, it can interface with existing enterprise content management systems, OA systems, or project management systems to retrieve structured data such as archived contracts and project acceptance reports; Based on the structured data, the relevant tables are queried through JDBC / ODBC connections to obtain metadata such as project number, name, and amount, and then an association is established with the corresponding unstructured documents. Configure a web crawler module to periodically access the designated public resource trading center website, and capture the public information of bidding projects by simulating login or parsing public pages, and automatically download the PDF attachments of the public information to the bidding document library; It integrates with the enterprise's email system, monitors specific email accounts, parses email subjects and content, downloads email attachments, and categorizes and stores the attachments in a bidding document library.

3. The method for constructing a bidding knowledge graph based on OCR and NLP according to claim 1 or 2, characterized in that, In S1, the process of converting images and scanned PDF documents into processable text data using the OCR method includes: The scanned PDF or image files are converted into a unified image format for processing; image tilt is detected and corrected using algorithms such as Hough transform; noise generated during scanning is removed using filtering algorithms; histogram equalization is used to improve the contrast between text and background; and color or grayscale images are converted into black and white binary images to highlight text outlines. The preprocessed image is input into the OCR engine, preserving the coordinate position of each text segment and each character recognized by the OCR in the original image; basic cleaning is performed on the recognized text, and garbled characters caused by unclear images are removed by rule filtering method; each original document is converted into a standardized structured text file containing hierarchical structure and text coordinate information, and the structured text file is used as the input for step S2.

4. The method for constructing a bidding knowledge graph based on OCR and NLP according to claim 2, characterized in that, In S2, the information extraction from the preprocessed text based on the NLP method includes: A sequence labeling model based on a pre-trained language is constructed, and a classification layer is connected to the output layer of the sequence labeling model. The classification layer is used to predict a label for each word in the structured text file. The sequence labeling model is initially trained using a general domain dataset, and then fine-tuned using a small-scale dataset manually annotated from bidding documents. A domain-specific dictionary is constructed, and word features from the domain-specific dictionary are integrated into the sequence labeling model. The text blocks of the structured text file output in step S1 are input sequentially into the trained sequence labeling model. The sequence labeling model outputs a labeled sequence, merges consecutive characters into complete key entities based on the labels, and records the position of the key entities in the original text. The core set of relationships in the bidding and tendering field is predefined; the text of two key entities is input, the text is encoded, and the tag information is embedded in the position of the key entities. The classification layer determines which predefined relationship the two key entities belong to, or whether there is no relationship; then, the coordinate information of the key entities and syntactic dependency analysis are used to determine whether the two key entities are in the same sentence or adjacent text blocks, and pattern matching rules are written to verify the relationship between the two key entities. For semi-fixed format documents, define attribute extraction templates; use a pre-trained machine reading comprehension model to extract attributes through questioning; and normalize the extracted attributes.

5. The method for constructing a bidding knowledge graph based on OCR and NLP according to claim 1, characterized in that, In S3, nodes and their attributes include: Projects, with attributes such as project name, contract amount, start date, completion date, and owner; Personnel, with attributes such as name, position, and years of service; Companies, with attributes such as company name, registered capital, and qualification level; Qualification certificates, with attributes such as certificate number, issuing authority, and validity period; Edges include: the role of the personnel connecting to the project, the company connecting to the project's construction, the qualifications of the personnel and the qualification certificate, and the affiliation of the personnel and the company. The construction of the bidding knowledge graph includes: converting the list of key entities and attributes output by S2 into node insertion operations in the graph database; and connecting the inserted nodes according to the relationship list output by S2. The process of obtaining entity node vectors includes: encoding relevant text of key entities using a pre-trained semantic model, inputting attributes and relationships, and obtaining entity node vectors.

6. The method for constructing a bidding knowledge graph based on OCR and NLP according to claim 4, characterized in that, S4 also includes: using OCR methods to convert bidding documents into processable text data; The method of parsing bidding documents using NLP includes: using a rule-based dependency model or machine learning model to structurally partition the bidding documents; using regular expressions to match fixed-format clauses in the bidding documents; using a sequence labeling model to identify the textual requirements of qualification requirements, method parameters, and scoring criteria in the clauses; merging the extracted fragmented textual requirements into complete semantic units; constructing a semantic model based on the complete semantic units; fine-tuning the semantic model using a bidding document corpus to better understand technical terms; and outputting a requirement vector from the semantic model.

7. The method for constructing a bidding knowledge graph based on OCR and NLP according to claim 1, characterized in that, In step S5, the semantic matching and similarity calculation of the attributes of all nodes in the entity node vector with each demand vector to filter out the projects with the highest similarity and obtain the matching results includes: obtaining the demand vector list generated in step S4, where each demand vector is associated with a demand vector type to limit the range of key entity types to be retrieved; obtaining similar key entities of the demand vectors using an approximate nearest neighbor search method based on the bidding knowledge graph, and calculating the similarity of the attributes of the nodes of the demand vectors and similar key entities using a cosine similarity algorithm; verifying whether similar key entities meet the constraints through a graph query language and obtaining the graph relationship matching degree; calculating a comprehensive score using a weighted algorithm based on the similarity and graph relationship matching degree, retaining key entities with a comprehensive score greater than the comprehensive score threshold, and sorting the key entities in descending order according to the comprehensive score; outputting the matched key entities and the entity node vectors corresponding to the key entities; and outputting the matched projects if the percentage of the entity node vectors corresponding to the matched key entities to the total entity node vectors of the projects exceeds a percentage threshold.

8. The method for constructing a bidding knowledge graph based on OCR and NLP according to claim 1, characterized in that, In step S6, the extraction of highly similar key entities, their attributes, and relationships from the knowledge graph, and the filling of these entities according to a preset bid template to generate the core chapters of the bid, includes: designing the layout and static content of the bid in Microsoft Word, and then inserting special markers at the locations where dynamic filling is required; performing a detailed graph query based on the key entity IDs returned by the matching results in step S5 to obtain complete attribute chains and relationship chains, and obtaining structured matching data based on the key entities, attribute chains, and relationship chains; synthesizing the structured matching data into fluent natural language, and inserting it into the bid document according to the special markers to generate the core chapters of the bid.

9. A knowledge graph construction system for bidding and tendering based on OCR and NLP, characterized in that, The system includes: The data acquisition and OCR processing unit is used to collect bidding-related documents from multiple data sources and use OCR methods to convert unstructured documents into processable text data. The information extraction unit is used to identify and extract key entities, relationships between entities, and entity attributes from text data based on NLP technology. The knowledge graph construction unit is used to store the extracted entities, attributes and relationships in the form of a graph database, construct the bidding knowledge graph, and generate entity node vectors; The tender document parsing unit is used to parse the uploaded tender documents, extract the text requirements, and transform them into a requirement vector. The semantic matching unit is used to calculate the similarity between the demand vector and the entity node vector to obtain the matching result. The tender document generation unit is used to extract relevant information from the knowledge graph based on the matching results and automatically generate the core chapters of the tender document according to the tender document template.