Key information extraction method and system for construction engineering documents

Abstract

The application relates to the technical field of building engineering information processing and application, and particularly provides a key information extraction method and system for building engineering files, which comprises the following steps: processing multiple source heterogeneous files into building engineering files, obtaining a first text through text analysis, and obtaining a second text through picture analysis; a model used for picture analysis is obtained by training an OCR model based on a domain fine-tuning strategy and a color balance strategy; inputting the first text and the second text into a screening model to obtain target text, wherein the screening model is obtained by training an initial model based on a mixed fine-tuning of a domain dataset and a general dataset, the domain dataset is obtained by processing a preselected dataset based on multiple semantic generalization strategies, and the initial model adopts a lightweight information extraction framework based on K-class global pointers; and inputting the target text into an extraction model to obtain key information. The method can solve the problem that, in the related art, when key information is extracted from building engineering files, it is difficult to balance the scalability, high precision and high efficiency.

Description

Technical Field

[0001] This invention relates to the field of building engineering information processing and application technology, and in particular to a method and system for extracting key information from building engineering documents. Background Technology

[0002] In the field of construction engineering, projects often involve bidding documents, pricing documents, architectural design specifications, and various CAD drawings (DWG format), as well as heterogeneous files from multiple sources such as PDF, DOC, DOCX, and XML. These files lack a unified data structure and organization method.

[0003] Most related technologies rely on manual processing or local parsing methods for single-type files, lacking a unified organization and preprocessing mechanism at the project level, and thus failing to meet the actual engineering needs for automated integration of multi-source information. They also lack scalability for larger-scale engineering projects and struggle to guarantee the accuracy and efficiency of key information extraction. Especially in CAD drawing scenarios, character information is commonly distributed across layers, overlapping, or obscuring; methods based on single-layer clustering or parsing often miss key information or produce recognition errors, leading to reduced efficiency in subsequent data utilization.

[0004] There is currently no effective solution to the problem that it is difficult to balance scalability, high accuracy, and high efficiency when extracting key information from building engineering documents in related technologies. Summary of the Invention

[0005] The present invention provides a method and system for extracting key information from building engineering documents, which at least solves the problem in related technologies that it is difficult to balance scalability, high accuracy and high efficiency when extracting key information from building engineering documents.

[0006] This invention provides a method for extracting key information from construction project documents, comprising: standardizing and annotating multi-source heterogeneous documents of a target construction project to obtain construction project documents; recognizing text files in the construction project documents using a text parsing tool to obtain first text; inputting image files in the construction project documents into an image parsing model for recognition to obtain second text, wherein the image parsing model is obtained by training an optical character recognition model based on a domain fine-tuning strategy and a color balance strategy; filtering the first and second texts using a filtering model to obtain target text, wherein the filtering model is obtained by fine-tuning an initial model based on a hybrid domain dataset and a general dataset, the domain dataset is obtained by processing a pre-selected dataset using multiple semantic generalization strategies, and the initial model adopts a lightweight information extraction framework based on K-class global pointers; and extracting information from the target text using an extraction model to obtain key information, wherein the key information is the output result corresponding to the prompt words input to the extraction model.

[0007] Preferably, before inputting the image files from the architectural engineering documents into the image parsing model for recognition, the above method further includes: obtaining multiple image-text pairs based on the engineering drawings to construct an image-text dataset, wherein the engineering drawings include drawings with multiple layers and drawings where characters are covered by lines, the number of image-text pairs is less than 10k, and the number of samples with different background colors and font colors in the image-text dataset is a first preset ratio for color equalization; training the optical character recognition model based on the DeepSpeed ​​optimization framework and the image-text dataset to obtain the image parsing model.

[0008] Preferably, multiple image-text pairs are obtained based on engineering drawings to construct a graphic-text dataset, including: obtaining logical clusters from the engineering drawings based on a text clustering algorithm, wherein the query radius of the text clustering algorithm is set to 2-3 times the average size of the text blocks in the logical cluster, and the center point of the text block is calculated based on the minimum and maximum coordinates of the bounding box; filling the text blocks in the logical clusters with a fill-in-the-blank text algorithm to obtain integrated text, wherein the fill-in-the-blank text algorithm is related to the overlap length, overlap ratio, and horizontal position relationship; and determining multiple image-text pairs based on the integrated text and a first preset ratio to construct a graphic-text dataset.

[0009] Preferably, before filtering the first and second texts based on the filtering model, the above method further includes: obtaining original text data from multiple types of engineering files; selecting statements containing target fields from the original text data based on the type of the target construction project, and labeling the entities in the target fields to obtain a pre-selected dataset; generalizing the pre-selected dataset based on multiple semantic generalization strategies to obtain a domain dataset; and training the initial model of the filtering model based on the domain dataset and the general dataset to obtain the filtering model.

[0010] Preferably, the pre-selected dataset is generalized based on multiple semantic generalization strategies to obtain a domain dataset, including: generalizing the pre-selected dataset based on multiple semantic generalization strategies to obtain multiple generalized datasets with equal sample sizes; and filtering the generalized datasets to obtain a domain dataset.

[0011] Preferably, the pre-selected dataset is generalized using multiple semantic generalization strategies to obtain multiple generalized datasets with equal sample sizes, including: a first semantic generalization strategy based on keeping the reference text unchanged, changing the entity name, and retaining the field values, to obtain a first generalized dataset; a second semantic generalization strategy based on changing the reference text, keeping the entity name unchanged, and retaining the field values, to obtain a second generalized dataset; a third semantic generalization strategy based on changing the reference text, changing the entity name, and retaining the field values, to obtain a third generalized dataset; a fourth semantic generalization strategy based on changing the reference text, changing the entity name, and omitting or missing field values, to obtain a fourth generalized dataset; and a fifth semantic generalization strategy based on changing the reference text, keeping the entity name unchanged, and omitting or missing field values, to obtain a fifth generalized dataset; wherein the sample sizes of the first, second, third, fourth, and fifth generalized datasets are equal.

[0012] Preferably, before training the initial model of the screening model based on the domain dataset and the general dataset, the above method further includes: determining the ratio between the number of samples in the general dataset and the number of samples in the domain dataset based on the type of the target construction project, wherein the ratio is greater than 20:1; determining the number of samples in the general dataset based on the number of samples in the domain dataset and the ratio; and extracting publicly available Chinese information based on the number of samples in the general dataset to obtain the general dataset.

[0013] Preferably, the first text and the second text are filtered based on the filtering model to obtain the target text, including: calculating the relevance scores of the first text and the second text to the target field based on the filtering model; and selecting the first text and the second text with relevance scores greater than a preset threshold as the target text.

[0014] Preferably, the multi-source heterogeneous files of the target construction project are standardized and annotated to obtain construction project files, including: classifying the multi-source heterogeneous files into the following categories: pricing documents, bidding documents, general architectural design specifications, business format documents, and civil defense documents; classifying the files in each category according to their formats to obtain files to be annotated; annotating the fields in the files to be annotated with actual values ​​based on a preset field table to obtain annotated field information; and determining the construction project files based on the files to be annotated and the annotated field information.

[0015] This invention provides a key information extraction system for construction project documents, comprising: a document parsing subsystem for standardizing multi-source heterogeneous documents of a target construction project to obtain construction project documents; a text parsing tool for recognizing text files within the construction project documents to obtain first text; an image recognition subsystem for inputting image files from the construction project documents into an image parsing model for recognition to obtain second text, wherein the image parsing model is obtained by training an optical character recognition model based on a domain fine-tuning strategy and a color balance strategy; a filtering subsystem for filtering the first and second texts based on a filtering model to obtain target text, wherein the filtering model is obtained by fine-tuning a hybrid domain dataset and a general dataset, the domain dataset is obtained by processing a pre-selected dataset using multiple semantic generalization strategies, and the initial model adopts a lightweight information extraction framework based on K-class global pointers; and an information extraction subsystem for extracting information from the target text based on an extraction model to obtain key information, wherein the key information is the output result corresponding to the prompt words input to the extraction model.

[0016] The present invention provides a method and system for extracting key information from construction project documents. It standardizes and annotates multi-source heterogeneous documents of a target construction project to obtain construction project documents. By standardizing various documents in the project, it achieves efficient data extraction and information processing.

[0017] The first text is obtained by recognizing the text files in the architectural engineering documents using a text parsing tool; the second text is obtained by inputting the image files in the architectural engineering documents into an image parsing model for recognition. The image parsing model is obtained by training an optical character recognition model based on a domain fine-tuning strategy and a color balance strategy, and still has strong generalization ability in diverse engineering drawing environments.

[0018] The first and second texts are filtered using a filtering model to obtain the target text. The filtering model is obtained by fine-tuning an initial model using a hybrid domain dataset and a general dataset. The domain dataset is obtained by processing a pre-selected dataset using various semantic generalization strategies. The initial model employs a lightweight information extraction framework based on K-class global pointers. Information is extracted from the target text using an extraction model to obtain key information, which is the output corresponding to the prompt words input to the extraction model. This unified lightweight framework enables efficient collaboration, balancing speed, performance, and practical application requirements even when processing complex architectural engineering documents, achieving joint modeling and output of entity recognition and relation extraction.

[0019] This addresses the challenge of simultaneously achieving scalability, high accuracy, and high efficiency when extracting key information from architectural engineering documents in related technologies. Attached Figure Description

[0020] To more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely some embodiments of the present invention, and those skilled in the art can obtain other embodiments based on these drawings without creative effort.

[0021] Figure 1 This is a flowchart illustrating the steps of a method for extracting key information from building engineering documents in an embodiment of the present invention.

[0022] Figure 2 This is a schematic diagram illustrating the division of multi-source heterogeneous files in an embodiment of the present invention, taking the tennis hall project of the sports center as an example.

[0023] Figure 3 This is a schematic diagram of a key information extraction system for building engineering documents in an embodiment of the present invention.

[0024] Figure 4 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention.

[0025] The above figures include the following reference numerals: 31. File parsing subsystem; 32. Image recognition subsystem; 33. Filtering subsystem; 34. Information extraction subsystem. Detailed Implementation

[0026] Embodiments of the present invention will now be described in more detail with reference to the accompanying drawings. While some embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the present invention. It should be understood that the drawings and embodiments of the present invention are for illustrative purposes only and are not intended to limit the scope of protection of the present invention.

[0027] In the field of construction engineering, projects often involve bidding documents, pricing documents, architectural design specifications, and various CAD drawings (DWG format), as well as heterogeneous files from multiple sources such as PDF, DOC, DOCX, and XML. These files lack a unified data structure and organization method.

[0028] Most related technologies rely on manual processing or local parsing methods for single-type files, lacking a unified organization and preprocessing mechanism at the project level, and thus failing to meet the actual engineering needs for automated integration of multi-source information. They also lack scalability for larger-scale engineering projects and struggle to guarantee the accuracy and efficiency of key information extraction. Especially in CAD drawing scenarios, character information is commonly distributed across layers, overlapping, or obscuring; methods based on single-layer clustering or parsing often miss key information or produce recognition errors, leading to reduced efficiency in subsequent data utilization.

[0029] Furthermore, using large-scale pre-trained language models in Chinese and multilingual environments results in a massive number of parameters and high computational resource consumption, making them unsuitable for localized deployments, low-computing-power scenarios, or specific vertical applications in engineering. Meanwhile, the three sub-tasks of information extraction—Named Entity Recognition (NER), Relation Extraction (RE), and Event Extraction (EE)—are not isolated but rather interdependent and mutually reinforcing. However, related methods often employ single models or separate strategies to complete different sub-tasks, lacking a unified, lightweight framework for efficient collaboration. This leads to insufficient accuracy when processing complex engineering documents, making it difficult to balance speed, performance, and practical application requirements.

[0030] Therefore, please refer to Figure 1 As shown, the present invention provides a method for extracting key information from building engineering documents, including steps S101 to S105.

[0031] Step S101: Standardize and annotate the multi-source heterogeneous files of the target building project to obtain the building project files.

[0032] Step S102: Based on the text parsing tool, the text file in the construction project file is identified to obtain the first text.

[0033] Step S103: Input the image file in the construction project file into the image parsing model for recognition to obtain the second text. The image parsing model is obtained by training an optical character recognition model based on a domain fine-tuning strategy and a color balance strategy.

[0034] Step S104: Filter the first text and the second text based on the filtering model to obtain the target text. The filtering model is obtained by fine-tuning the initial model based on a mixture of domain dataset and general dataset. The domain dataset is obtained by processing the pre-selected dataset based on multiple semantic generalization strategies. The initial model adopts a lightweight information extraction framework based on K-class global pointers.

[0035] Step S105: Extract information from the target text based on the extraction model to obtain key information, wherein the key information is the output result corresponding to the prompt words input to the extraction model.

[0036] The target building project can be, but is not limited to, medical, cultural, and educational building projects. This embodiment will use a sports center tennis hall project as an example for further explanation. A standardized annotation and processing method is proposed for multi-source heterogeneous files in the target building project. By standardizing various types of files in the project, efficient data extraction and information processing are achieved. Modular data preprocessing and field annotation techniques are employed to ensure accurate extraction of file content and system scalability, as detailed later.

[0037] Text parsing tools include, but are not limited to, PDF parsers, DOCX parsers, and XML parsers.

[0038] Image files in architectural engineering documents include CAD files in DWG format.

[0039] Domain fine-tuning refers to training an optical character recognition model using a graphic dataset obtained from complex engineering drawings. These complex engineering drawings include, but are not limited to, drawings with multiple layers and drawings where characters are covered by lines.

[0040] Color balance refers to controlling and balancing the number of samples with different background colors and different font colors in the aforementioned image and text dataset. Different background colors and different font colors include three cases: different background colors but the same font color, the same background color but different font color, and both background colors and font colors are different. Performing color balance control can prevent the model from overfitting to a single color mode, thereby enhancing its generalization ability in diverse engineering drawing environments.

[0041] The optical character recognition (OCR) model is an open-source model, which can be, but is not limited to, the GOT-OCR model based on the Qianwen architecture. The GOT-OCR model based on the Qianwen architecture is an end-to-end multimodal OCR model based on general OCR theory.

[0042] The pre-selected dataset (seed test set) is created by extracting raw text data from various types of engineering documents (including CAD drawing files, bidding documents, construction instructions and engineering contract texts, etc.) and extracting fields of relevant statements and entities, which can provide an accurate training benchmark for the model.

[0043] The domain dataset is obtained by processing the pre-selected dataset using various semantic generalization strategies, which can form a wider range of training samples. This embodiment will subsequently provide preferred methods for implementing various semantic generalization strategies.

[0044] The general dataset may be, but is not limited to, the following general domain test sets: DUEE1.0, DuIE2.0, CMeEE-V2, CLUENER2020, CMeIE, MSRA, and People's Daily.

[0045] Specifically, DUEE1.0 is a typical Chinese event extraction dataset, used to improve the model's ability in event recognition and argument extraction. DuIE2.0 is a large-scale Chinese relation extraction dataset, covering multiple relation categories. CMeEE-V2 is a medical named entity recognition dataset, enhancing the model's ability to recognize specialized entities. CLUENER2020 is a general fine-grained named entity recognition dataset. CMeIE is a Chinese medical information extraction dataset. MSRA is an early open standard dataset for Chinese entity recognition. People's Daily is a publicly annotated corpus from the People's Daily, covering common entity categories such as personal names, place names, and organization names.

[0046] By learning from massive amounts of general data (e.g., hundreds of thousands of records), the screening model can acquire general language understanding and logical reasoning abilities, preventing it from being limited to specific expressions in the domain dataset. When encountering new architectural expressions or similar domain tasks not seen in the training set, the screening model can better transfer and infer based on the knowledge learned from the general data, thus enhancing its generalization ability. Even during the fine-tuning phase for specific tasks, the presence of general data helps the screening model maintain its judgment on the fluency and grammatical correctness of natural language, making the extracted architectural field information more fluent and accurate in context, thus helping to optimize the language model's foundation. Domain datasets are relatively small (e.g., 20,000-30,000 records). If only the domain dataset is used for training, the screening model is prone to overfitting, only able to process training samples and unable to handle new data. Mixing the domain dataset and the general dataset helps balance the data distribution; the specific ratio of general data to domain data will be explained later. A larger proportion of general data can produce an effective regularization effect, diluting the specific noise of the domain data, allowing the screening model to learn more robust feature representations.

[0047] Furthermore, with the development of large-scale pre-trained language models, information extraction (IE) has made significant progress in Chinese and multilingual environments. However, these models have a large number of parameters and consume high computational resources, making them unsuitable for localization, low-computing-power, or specific vertical deployment scenarios in engineering fields. Meanwhile, the three sub-tasks of information extraction—named entity recognition (NER), relation extraction (RE), and event extraction (EE)—are not isolated but rather interdependent and mutually reinforcing. Compared to related technologies that often employ a single model or separate strategies to complete different sub-tasks, the method provided in this invention can achieve efficient collaboration through a unified, lightweight framework, balancing speed (high efficiency), performance (high accuracy), and practical application requirements (scalability) when processing complex architectural engineering documents.

[0048] Specifically, the initial model for the filtering process employs a lightweight information extraction (IE) framework based on the K-Global Pointer algorithm. This framework is fine-tuned using pre-trained small models such as Qwen2.5-0.5B and BERT-zh-large, achieving high-precision information extraction and filtering even with limited computing power. This lightweight information extraction framework utilizes a multi-head global pointer mechanism. By setting multiple pointer heads based on the text vector representation, it models the beginning and end positions of entities and relationships of different categories, enabling joint extraction of entities and relationships.

[0049] Input text sequence Input to pre-trained language model This yields the context vector representation of the text. Where n represents the text length and d represents the vector dimension. Represents the real number field.

[0050] The vector representation matrix H is input into the screening model, and a pointer head is configured for each type of entity k to predict the probability distribution of its starting position. Probability distribution of the ending position : ; ; In the formula, This represents the activation function. This represents the weight matrix at the starting position. This represents the weight matrix at the end position.

[0051] When the probability value of a certain interval (i,j) satisfies , The probability threshold is used to determine if the text interval (i,j) belongs to entity of category k, thus obtaining the entity set. : ; In the formula, Indicates the starting position of the entity. Indicates the end position of the entity. Indicates the entity category.

[0052] Given the entity set E, enumerate all entity pairs. Input the data into the filtering model. Configure a pointer header for each type of relation r and calculate the relation score for entity pairs. : ; In the formula, This represents the Sigmoid activation function. Representing entities The corresponding vector representation matrix, This represents the weight matrix corresponding to relation r. Representing entities The corresponding vector representation matrix.

[0053] when When the entity pair exists, it is determined that a relation r exists, thus obtaining the relation set. : ; In other words, the information extraction framework based on K-class global pointers can achieve joint modeling and output of entity recognition and relation extraction under lightweight model conditions, thereby improving the efficiency and accuracy of information extraction.

[0054] For example, the code example for entity extraction is shown below. Here, id represents the unique identifier of a data record, named in the format of "dataset name-dataset type-number", such as GCCAD-test-001.

[0055] {

[0056] "id": "GCCAD-test-00001",

[0057] "instruction": "Find all entities of type [Project Name] in the article? Article: [I. Project Overview\nProject Name: Central District N1 Area School Construction Project\nProject Location: Intersection of XX Road and XX Road, Central District, XX District Approval (Filing) Certificate Number:____________\nProject Scale and Characteristics: The total land area of ​​this project is 14249.43 square meters, built according to the standard of a primary school with 36 classes / 1620 student places]",

[0058] "start": [ 25

[0060] ],

[0061] "end": [ 89

[0063] ],

[0064] "target": "Central Area N1 Zone School Construction Project"

[0065] }

[0066] For example, a code example for relation extraction is shown below.

[0067] {

[0068] "id": "GCCAD-test-00002",

[0069] "instruction": "Find the 'durability' of the 'building' in the article? Article: 'Overview of the Civil Defense Project\n 1. This project is located at the intersection of XX Road and XX Road in the central area of ​​XX City, XX Province. It is an attached civil defense project combining peacetime and wartime functions. In peacetime, it serves as an underground parking garage; in wartime, it serves as two Class 6 nuclear and Class 6 conventional shelters.\n The civil defense basement protective unit is located in the basement parking garage area, with a sports field, faculty and staff dormitories, teaching buildings, and science buildings above. All are multi-story buildings with a seismic fortification intensity of 6 degrees and a building durability of 50 years. The main structural type is a frame structure.\n 2. For detailed construction drawings of the dormitories, science buildings, and teaching buildings, please refer to the architectural drawings above. There are no factories or warehouses producing or storing flammable or explosive materials within 50m of this air-raid shelter; and no storage tanks for harmful liquids or highly toxic gases within 100m.\n 3.'" The total building area of ​​the civil defense basement is 1424 square meters, with a wartime personnel shelter area of ​​1165 square meters, accommodating a total of 1165 people during wartime. The unit functions and shelter areas are detailed in the table below.

[0070] "start": [ 45

[0072] ],

[0073] "end": [ 48

[0075] ],

[0076] "target": "50 years"

[0077] }

[0078] In this format, `instruction` represents the query command for the input model, with entity types enclosed in square brackets `[]`, such as `[School Name]`, `[Project Address]`, etc.; `target` represents the actual entity result, separated by the separator "|" if multiple results exist; `start` is the starting index (counting from 0) of the target entity in the `instruction` string; and `end` is the ending index of the target entity in the `instruction` string. By truncating the `instruction` according to the range `[start, end)`, the entity value corresponding to `target` can be directly obtained, thus ensuring machine readability and verifiability of the data.

[0079] In the method provided in this embodiment, the first and second texts are filtered by a filtering model to obtain the target text. This quickly filters out document content unrelated to the target field, significantly reducing the number of documents that the subsequent extraction model needs to process. This mechanism of pre-filtering before extracting key information can significantly improve processing efficiency and reduce computational costs, while providing more accurate input to the extraction model, thereby improving the accuracy of the final extraction results.

[0080] In addition, the image parsing model, filtering model and extraction model used in the above method provided in this embodiment can be integrated into a whole intelligent recognition model, which will be explained in detail later through the content recognition module.

[0081] Preferably, step S101 involves standardizing and annotating the multi-source heterogeneous files of the target construction project to obtain construction project files, including: classifying the multi-source heterogeneous files into the following categories: pricing documents, bidding documents, general architectural design specifications, business format documents, and civil defense documents. The files in each category are then categorized according to their file formats to obtain files to be annotated. Fields in the files to be annotated are annotated with actual values ​​based on a preset field table to obtain annotated field information. Finally, the construction project files are determined based on the files to be annotated and the annotated field information.

[0082] Within the "Business Type Documents" category, specific business type names are created and further subdivided into three target categories: "Architecture," "Structure," and "Facade." This is done to facilitate subsequent management of document data and automated retrieval. Taking the Sports Center Tennis Hall project as an example, the multi-source heterogeneous documents for the Sports Center Tennis Hall project are categorized as follows: Figure 2 As shown.

[0083] Collection of multi-source heterogeneous files Expressed as a formula: ; In the formula, Indicates pricing documents, This refers to the tender documents. This refers to the general architectural design specification document. Documents indicating business format This refers to civil defense documents.

[0084] After normalization, fields are defined and labeled according to the requirements given in the field table. Field information contained in each file category, such as project name, building area, number of floors, etc., needs to be labeled with actual values ​​to ensure the accuracy of subsequent automated information extraction. This labeled data will be used to train and validate the model, ensuring that the field information extracted from the input files is correct. The formula is as follows: ; In the formula, This indicates the fields to be extracted. This indicates the correct answer.

[0085] After completing the field annotation, the normalized files to be annotated are combined with the annotated field information to generate a complete project dataset (construction engineering documents). This dataset contains all file types (pricing documents, tender documents, etc.) and the annotated field information and corresponding answers for each file, providing reliable data support for the subsequent automated information extraction system. The formula is as follows: ; In the formula, This represents the complete project dataset corresponding to the building engineering documents. This represents a collection of all field information and their actual answers.

[0086] Taking the sports center tennis hall project as an example, the fields and actual values ​​are labeled as shown in Table 1.

[0087] Table 1. Fields and Actual Value Labels for the Sports Center Tennis Hall Project

[0088]

[0089] Preferably, in step S103, before inputting the image files from the architectural engineering documents into the image parsing model for recognition, the method further includes: acquiring multiple image-text pairs based on the engineering drawings to construct an image-text dataset. The engineering drawings include drawings with multiple layers and drawings where characters are covered by lines. The number of image-text pairs is less than 10k. The number of samples with different background colors and font colors in the image-text dataset is a first preset ratio for color equalization. The optical character recognition model is trained based on the DeepSpeed ​​optimization framework and the image-text dataset to obtain the image parsing model.

[0090] For example, based on engineering drawings, 1.5k CAD drawing image-text paired samples were obtained to construct a 2:1 ratio of training set to test set samples. Specifically, 1000 samples were used as the training set for model feature extraction and parameter optimization, and 500 samples were used as the testing set for independent evaluation of the corrected model's performance. The proportion of samples with different background colors and font colors was equal in the image-text dataset. The GOT-OCR model was then fine-tuned using this dataset to better handle complex issues such as character cross-layer distribution, overlapping occlusion, and color differences in CAD drawing scenarios, thereby improving the overall text extraction accuracy and stability.

[0091] The method described in this embodiment uses the DeepSpeed ​​optimization framework to fine-tune the OCR model, which optimizes training efficiency and reduces memory usage compared to the PyTorch training framework. The core source code and configuration templates of the DeepSpeed ​​optimization framework are open source, and will not be elaborated upon here. Subsequent experimental results show that the method provided in this embodiment significantly improves the performance of the image parsing model in CAD drawing text recognition and content extraction tasks, with key indicators improving by more than 5 percentage points compared to the base model. It not only improves the adaptability and extraction accuracy of the image parsing model to multi-layered nested drawings, but also balances training efficiency and inference performance, possessing strong engineering application value and promotion potential.

[0092] To improve the adaptability and robustness of the image parsing model in CAD drawing scenarios, especially for complex issues such as character overlap, occlusion, and cross-layer display in multi-layer drawings, the aforementioned image-text dataset was constructed to train the GOT-OCR model. In the data preparation phase, typical CAD sub-images exhibiting the aforementioned complex issues were selected and cropped based on the application characteristics of actual engineering drawings. This ensured that the selected samples could fully cover multi-layer mixed display and character overlap, enabling the image parsing model to deeply learn the complex background features in engineering drawings. By segmenting and dividing these sub-images, features of difficult samples (such as overlapping and occluded text) were preserved while maintaining the proportion of ordinary samples. This resulted in a more reasonable sample distribution in the overall dataset, preventing the model from favoring a single type of sample and enhancing the generalization ability of the image parsing model in diverse engineering drawing environments.

[0093] Furthermore, considering that CAD drawings may have various background and font colors in practical applications, these factors directly affect the recognition performance of the visual model. Therefore, the number of samples with different background and font colors was strictly controlled and balanced during the data construction process to enhance the model's generalization ability under diverse input conditions. In the annotation stage, this embodiment adopts a semi-automated annotation process, using the PPOCR-Label tool to annotate the selected CAD images with text boxes and corresponding text content, ultimately completing the construction of the aforementioned high-quality image-text dataset of 1.5k image-text pairs. This image-text dataset was used to fine-tune the image parsing model, enabling it to better handle complex issues such as character cross-layer distribution, overlapping occlusion, and color differences in CAD drawing scenarios, thereby significantly improving the overall text extraction accuracy and stability.

[0094] For example, the process of obtaining image-text pairs includes automated initial recognition, logical reorganization of text blocks, precise calibration of text content, persistent storage of raw data, image segmentation based on minimum bounding rectangle, and automated generation of image-label pairs.

[0095] Automated initial recognition refers to the application of a pre-defined PaddleOCR automated detection and annotation framework to perform a full scan of the CAD drawing. Deep learning algorithms are used to perform text detection and character recognition, generating initial text segment descriptions for the entire image area with a single click. This step aims to automatically and initially locate potential text regions in the image, establishing an initial mapping between pixel coordinates and text content.

[0096] Text block logical reorganization refers to the auxiliary merging process used in the initial recognition process to address the problem of complete text in the same paragraph or line being incorrectly divided into multiple scattered image blocks due to complex layout and inconsistent character spacing. Based on the geometric topological relationships and semantic continuity of the text, physically broken but logically belonging image regions are merged to ensure the integrity of the OCR recognition object.

[0097] Precise text content calibration refers to the process of verifying each automatically generated text pair string against the original image after region reconstruction. It focuses on correcting easily confused engineering terms, special symbols, and punctuation marks in CAD drawings, correcting character substitution errors, omissions, or over-recognition in the recognition results, thereby obtaining high-confidence ground truth data.

[0098] Persistent storage of raw data refers to serializing and saving the manually corrected text content, corresponding coordinates, and image metadata in standard JSON format. This JSON file contains the original geometric parameters of each text entity, providing a unified index reference for subsequent image segmentation.

[0099] Image segmentation based on minimum bounding rectangle refers to calculating the minimum bounding box that can contain the target text content based on the region coordinates in the JSON data. To prevent damage to the pixel features of edge characters during automated segmentation, an edge redundancy hyperparameter space=5 pixels is preset, which means that 5 pixels are extended outward from each side of the minimum bounding box as a buffer boundary to ensure that the segmented sub-image completely covers all text information and necessary background edges.

[0100] Automated generation of image-label pairs: An automated parsing program is executed to crop the original large image based on the calculated bounding boxes, extract the corresponding local image patches, and associate them with their corrected labels. This ultimately generates one-to-one image-label data pairs, which are saved in a preset format, providing high-quality structured samples for subsequent model training.

[0101] In addition, the image parsing model is based on the open-source model GOT-OCR and has been fine-tuned in a supervised manner. The dataset is organized in the form of image-text pairs, and a human-computer dialogue-style annotation structure is uniformly adopted. The sample elements are triples. ,in, Indicates the first A CAD sub-drawing, This indicates the OCR instruction prompt text. This represents the labeled answer to the target text. Sample code is shown below.

[0102] {

[0103] "conversations": [

[0104] {"from": "human", "value": "\nOCR:"},

[0105] {"from": "gpt", "value": "Building footprint area 8615.38"}

[0106] ],

[0107] image: 1.png

[0108] }

[0109] During training, the cross-entropy loss function is used as the optimization objective. In each training step, the image parsing model maximizes the likelihood between the predicted sequence and the reference answer, thereby improving the matching degree between the output text and the ground truth annotation. Cross-entropy loss function The mathematical expression is as follows: ; In the formula, In the target distribution, the th The true label of each word This represents the probability value corresponding to the distribution predicted by the image parsing model.

[0110] The image parsing model was trained using two NVIDIA RTX 3090 GPUs, employing the Hugging FaceAccelerate framework for efficient distributed training. AdamW was used as the optimizer, with a learning rate of 2.0e-5, a total of 20 training epochs, a batch size of 2, a maximum sequence length of 8192, and a warm-up ratio of 0.001. Bf16 mixed-precision training and gradient checkpointing were also enabled to effectively reduce GPU memory usage and improve training efficiency.

[0111] Furthermore, multiple image-text pairs are obtained based on engineering drawings to construct a graphic-text dataset. This includes: obtaining logical clusters from the engineering drawings using a text clustering algorithm, where the query radius of the text clustering algorithm is set to 2-3 times the average size of the text blocks in the logical cluster, and the center point of the text block is calculated based on the minimum and maximum coordinates of the bounding box; filling the text blocks in the logical clusters using a fill-in-the-blank text filling algorithm to obtain integrated text, where the filling conditions of the fill-in-the-blank text filling algorithm are related to the overlap length, overlap ratio, and horizontal position relationship; and determining multiple image-text pairs based on the integrated text and a first preset ratio to construct the graphic-text dataset.

[0112] Extracting logical clusters from engineering drawings using text clustering algorithms is a crucial step in structuring CAD drawing information, solving the problems of scattered text information and unclear relationships in CAD drawings. By organizing spatially similar text blocks into logical clusters, this algorithm provides a structured data foundation for subsequent text completion, field recognition, and semantic understanding.

[0113] Specifically, the text clustering algorithm first calculates the coordinates of the center point of each text block, and then calculates the center point using the minimum and maximum coordinates of the bounding box: ,in, Indicates the first The center point of each text block Indicates the first The left boundary X coordinate of a text block, Indicates the first The X-coordinate of the right boundary of a text block. Indicates the first The Y-coordinate of the lower boundary of a text block. Indicates the first The Y-coordinate of the upper boundary of a text block.

[0114] Then, a KDTree spatial index structure is constructed, organizing the center points of all text blocks into an efficient multidimensional spatial index: ,in, This represents the KDTree spatial index structure.

[0115] Next, perform a neighbor lookup for each text block to find other text blocks whose center point is within a set radius. The lookup radius is set to 2-3 times the average size of the text blocks.

[0116] After obtaining candidate text blocks, the minimum distance between bounding boxes is used as the criterion to evaluate spatial relationships by calculating the overlap between the bounding boxes of two text blocks. ; In the formula, Indicates the first The first text block and the first The minimum distance between text blocks Indicates the first The left boundary X coordinate of a text block, Indicates the first The X-coordinate of the right boundary of a text block. Indicates the first The left boundary X coordinate of a text block, Indicates the first The right boundary X-coordinate of a text block. When the minimum distance... When the threshold is less than or equal to the preset threshold, it is considered that the first... The first text block and the first The text blocks belong to the same cluster.

[0117] Finally, text blocks that meet the clustering criteria are organized into multiple logical clusters, each containing text blocks that are spatially close to each other: ; In the formula, Indicates the first Clusters, Indicates the threshold radius.

[0118] The entire clustering result can be represented as: ; In the formula, Represents the set of all clusters. Representing the 1st cluster, the 2nd cluster, ..., the 3rd cluster respectively. Clusters.

[0119] In addition, a text fill-in algorithm is used to identify and fill in blanks in CAD drawings. This algorithm automatically completes the filling of blanks by analyzing the spatial relationships and content features between text blocks, thus improving the completeness of text information.

[0120] Specifically, the text completion algorithm first identifies text blocks containing fill-in-the-blank markers, using regular expressions to identify various fill-in-the-blank marker patterns such as "___", "...", "□", etc. For each text block, the system determines whether it contains a fill-in-the-blank marker: ; In the formula, Indicates the first A text block, This function determines whether a text block contains fill tags.

[0121] Next, calculate the vertical overlap length between text blocks, considering the spatial relationship between the text blocks in the vertical direction: ; In the formula, Indicates the overlap length. Indicates the first The Y-coordinate of the upper boundary of a text block. Indicates the first The Y-coordinate of the lower boundary of a text block. Indicates the first The Y-coordinate of the upper boundary of a text block. Indicates the first The Y-coordinate of the lower boundary of a text block.

[0122] Then, the filling conditions are judged, and the degree of matching of the text blocks is evaluated through multiple conditions, including overlap length, overlap ratio, and horizontal positional relationship: ; In the formula, This represents the height difference between two text blocks. Indicates the overlap ratio threshold. Indicates the first The left boundary X coordinate of a text block, Indicates the first The X-coordinate of the right boundary of a text block. Indicates the first The left boundary X coordinate of a text block, Indicates the first The X-coordinate of the right boundary of a text block.

[0123] After identifying the text blocks that meet the filling criteria, the candidate text blocks are sorted according to their right boundary position to ensure that the filling order conforms to the reading habit of left to right: ; In the formula, This represents the sorted list of candidate text blocks. Represents a set of candidate text blocks. Indicates the first The X-coordinate of the right boundary of a text block.

[0124] After sorting, the sorted text content is filled into the blank text blocks to complete text integration. Finally, the processed result is constructed, and the processed text blocks are filtered out: ; In the formula, This indicates the final processing result. This represents the set of indices of the text blocks to be deleted.

[0125] Preferably, in step S104, before filtering the first and second texts based on the filtering model, the method further includes: obtaining raw text data from multiple types of engineering files; selecting statements containing target fields from the raw text data based on the type of the target construction project, and labeling the entities in the target fields to obtain a pre-selected dataset; generalizing the pre-selected dataset based on multiple semantic generalization strategies to obtain a domain dataset; and training the initial model of the filtering model based on the domain dataset and the general dataset to obtain the filtering model.

[0126] Furthermore, the pre-selected dataset is generalized using multiple semantic generalization strategies to obtain the domain dataset, including: generalizing the pre-selected dataset using multiple semantic generalization strategies to obtain multiple generalized datasets with equal sample sizes. The generalized datasets are then filtered to obtain the domain dataset.

[0127] Filtering generalized datasets refers to deduplicating generalized datasets and removing data that is semantically incomplete, lacks entities, or does not conform to the format specifications.

[0128] Furthermore, the pre-selected dataset is generalized using multiple semantic generalization strategies to obtain several generalized datasets with equal sample sizes. These include: a first semantic generalization strategy based on unchanged reference text, changed entity names, and retained field values, which is used to generalize the pre-selected dataset to obtain the first generalized dataset. Specifically, changing entity names refers to replacing entity names with semantically equivalent synonyms, such as changing "project address" to "construction location".

[0129] Based on a second semantic generalization strategy that changes the reference text, keeps the entity name unchanged, and retains the field values, the pre-selected dataset is generalized to obtain a second generalized dataset. Specifically, changing the reference text means rewriting or reorganizing the reference text, such as rewriting the context but keeping the "project address" intact.

[0130] Based on a third semantic generalization strategy that modifies the reference text, entity names, and preserves field values, the pre-selected dataset is generalized to obtain a third generalized dataset. For example, "project address" is changed to "location", and the context is rewritten.

[0131] Based on the fourth semantic generalization strategy, which involves changes to the reference text, changes to the entity name, and omissions or missing field values, the pre-selected dataset is generalized to obtain the fourth generalized dataset. Specifically, omissions or missing field values ​​mean that values ​​corresponding to the entity name are not required. For example, in a scenario simulating a missing entity, the field value does not appear.

[0132] Based on the fifth semantic generalization strategy—which involves changing the reference text, keeping the entity name unchanged, and omitting or missing field values—the pre-selected dataset is generalized to obtain the fifth generalized dataset. For example, a scenario with missing field values ​​is simulated, but the "project address" entity remains fixed.

[0133] The first, second, third, fourth, and fifth generalized datasets all have the same number of samples.

[0134] The five semantic generalization strategies mentioned above can be implemented using open-source models such as the GPT-4o model, but are not limited to these.

[0135] By using the above rules, each seed sample in the pre-selected dataset can be expanded into several generalized samples, thereby significantly increasing the size of the domain dataset.

[0136] For example, the pre-selected dataset includes the following 32 fields: Project Name, Construction Nature, Project Location, Construction Location Level, Primary Project Category, Total Building Area, Above-ground Building Area, Underground Building Area, Land Area, Floor Area Ratio, Total Number of Residential Units, Number of School Places, Number of Beds, Investment Type, Cost Stage, Execution List, Building Footprint Area, Landscape Area, Green Area, Greening Rate, Elevators, Charging Pile Parking Spaces, Total Parking Spaces, Start Date, Completion Date, Construction Period, Foundation Type, Construction Unit, Design Unit, Civil Defense Building Area, Seismic Resistance Level, and Assembly Rate. The values ​​corresponding to the above fields are the key information that the model needs to extract in practical applications. Therefore, the above fields need to be selected for model training.

[0137] During model training, the above 32 fields are labeled. Five items are randomly selected for each field to search. Only one data point needs to be labeled for each item, resulting in a total of 32 × 5 = 160 labeled data points.

[0138] Specifically, randomly selecting 5 items for each field helps avoid bias caused by excessively high sample concentration or uneven data distribution, thus ensuring that the test set (pre-selected dataset) has broad representativeness and the objectivity and fairness of the evaluation results. During annotation, it is necessary to ensure that the selected text blocks contain the field name and its corresponding value. For text files, directly extract a complete sentence from the field name to the field value. For DWG drawings, a drawing viewing tool is used. Since text elements in the drawing may belong to different layers, a combination of screenshot and visual large-scale model recognition is employed. First, the text block area containing the field name and field value is extracted (the text in this block area does not need to be a complete sentence to adapt to subsequent clustering algorithms; that is, in real-world scenarios, the text content obtained by clustering algorithms may not be a complete sentence). Then, the text is refined using a visual large-scale model (which can be extracted manually, but introducing a visual large-scale model can speed up the extraction process), ensuring that the original text does not undergo semantic, formatting, or layout changes.

[0139] The 160 labeled data points were generalized using the five semantic generalization strategies described above. Each semantic generalization strategy generalized each labeled data point into 32 data points, resulting in a total of 160 × 5 × 32 = 25,600 generalized data points. These 25,600 generalized data points were then filtered to obtain a domain dataset containing 23,268 domain data points.

[0140] Preferably, before training the initial model of the screening model based on the domain dataset and the general dataset, the method further includes: determining the ratio between the sample size of the general dataset and the sample size of the domain dataset based on the type of the target construction project, wherein the ratio is greater than 20:1; determining the sample size of the general dataset based on the sample size and ratio of the domain dataset; and extracting publicly available Chinese information based on the sample size of the general dataset to obtain the general dataset.

[0141] For example, the general dataset has 574,205 samples, while the domain dataset has 23,268 samples. The ratio between the two is approximately 24:1, which is greater than 20:1.

[0142] While training on a dataset labeled only in a single specific vertical domain can improve the model's accuracy in extracting key entity information from construction project documents, it can lead to overfitting and incorrect extraction when faced with content unseen in real-world scenarios. Therefore, a greater proportion of general-purpose data enhances the model's generalization ability and robustness. Simultaneously, the specific ratio between general-purpose and domain-specific data is determined based on the type of the target construction project to achieve an optimal balance between domain-specific accuracy and general generalization. This specific ratio can be determined by those skilled in the art, based on a ratio greater than 20:1, through a limited number of experiments.

[0143] Preferably, the samples in the general dataset and the domain dataset are preprocessed in a unified manner, including format conversion, field normalization and label consistency correction, so as to ensure that all data can be adapted to the lightweight information extraction framework based on K-class global pointers for training.

[0144] Preferably, step S104, filtering the first text and the second text based on the filtering model to obtain the target text, includes: calculating the relevance scores of the first text and the second text to the target field based on the filtering model. The first text and the second text with relevance scores greater than a preset threshold are selected as the target text.

[0145] In engineering document processing, the filtering model can identify target text (target document fragments) containing keywords of the target field, laying the foundation for subsequent accurate extraction based on the extraction model. The algorithm implementation process of the filtering model is shown below.

[0146] For the retrieved Filter the texts to be filtered (first text and second text): ; In the formula, This represents the set of target texts obtained through filtering. Indicates the first One text to be filtered, Indicates the text relevance score. Indicates the target field. Indicates the threshold. Indicates the text to be filtered For fields Relevance score of the value.

[0147] The model used in the above technical solution provided in this embodiment adopts the following comprehensive evaluation indicators.

[0148] The macro-average F1 score and the micro-average F1 score are used as the core evaluation indicators of the overall model performance.

[0149] The macro average F1 score is calculated as follows: Each category is treated equally; first, the F1 score for each category is calculated, and then the arithmetic mean is taken. The formula is expressed as: ; ; ; ; In the formula, Indicates the first F1 scores for each category This represents the macro average F1 score. Indicates the total number of categories. Indicates the first True examples of each category, Indicates the first Each category of false positives Indicates the first False counterexamples for each category.

[0150] The micro-average F1 score is calculated by considering the prediction results for all categories, summing them, and then calculating the overall F1 score. The formula is expressed as: ; ; ; In the formula, This represents the micro-average F1 value.

[0151] To evaluate the performance of the image parsing model, BLEU was used to measure the closeness of the model's output recognized text to the real text. METEOR, which focuses more on semantic matching, was also employed, based on word-level alignment and considering factors such as synonyms and stemming. F1 score, precision, and recall were also used for comprehensive evaluation.

[0152] The formulas for calculating BLEU and METEOR are as follows: ; ; In the formula, This represents an adjustable length penalty value. If the generated text length is greater than or equal to the reference text length, the value is 1; otherwise, it is calculated using exp(1 – reference text length / generated text length). Indicates weight, It represents the precision of an n-gram (a series of n consecutive characters or words in text). Indicates accuracy. Recall is represented by precision. METEOR combines precision and recall and calculates a weighted harmonic average.

[0153] The automated extraction framework of the intelligent recognition model (integrating image parsing, filtering, and extraction models) used in the method described in this embodiment was compared and evaluated with the standard prompt engineering solution of the Qwen2.5 open-source series of large models. Since large models cannot directly extract certain fields, the evaluation specifically selected "representative fields that can be directly used by large models" for comparison. Accuracy was defined as "number of correctly extracted items / total number of test items." The standard prompt engineering solution refers to a benchmark method based on prompt engineering, which provides detailed prompt design for each field to ensure the fairness of the comparison. The test coverage included 10 manually labeled project engineering datasets, covering 7 major project types, including residential buildings, educational buildings, office buildings, medical buildings, cultural buildings, hotel buildings, and residential service buildings, ensuring the representativeness and reliability of the evaluation results.

[0154] Experimental tests were conducted on the technical solutions provided in this embodiment.

[0155] All fine-tuning experiments were conducted in a unified environment with the following configuration: training was performed on eight NVIDIA RTX 3090 GPUs using Hugging Face Accelerate for distributed training. Key hyperparameters included: a learning rate of 2.0e-5, an AdamW optimizer, three training epochs, a batch size of 8, a maximum sequence length of 512, and 200 warm-up steps. Bf16 mixed-precision training was also enabled to improve efficiency. The model fine-tuning process was tracked using WandB.

[0156] Specifically, the performance test results of the image parsing model are shown in Table 2.

[0157] Table 2 Performance test results of the image parsing model

[0158]

[0159] As shown in Table 2, the image parsing model GOT-OCR-ft in the above method provided in this embodiment improves the key indicators such as BLEU, METEOR, F-measure (F1), Precision, and Recall by more than 5 percentage points compared with the optical character recognition model GOT-OCR. This indicates that the above image parsing model performs well in the task of CAD drawing text recognition and content extraction.

[0160] Specifically, the selection model was evaluated using both the general dataset (DUEE1.0, DuIE2.0, CMeEE-V2, CLUENER2020, CMeIE, MSRA, People's Daily) and the domain dataset GCCAD. The experimental results are shown in Table 3. In this embodiment, Chinese-MacBERT-Large was selected as the initial model (baseline model) for the selection process. The initial weights of this initial model came from the official release version, without introducing any additional pre-trained parameters.

[0161] Table 3 Performance test results of the screening model

[0162]

[0163] Analysis of Table 3 shows that the baseline model (model-general) using only the general dataset and without domain fine-tuning performs well on the general dataset, with macro_f1 and micro_f1 scores of 0.8889 and 0.8847 respectively, indicating that the model has strong generalization ability. In this embodiment, macro_f1 and micro_f1... The meanings are the same, micro_f1 and The meaning is the same. However, it performs poorly on the domain dataset GCCAD, scoring only 0.6509, indicating its limited adaptability to vertical domain data.

[0164] The baseline model (model-gccad), which was fine-tuned using only the domain dataset GCCAD without using the general dataset, significantly improved its score on the GCCAD dataset to 0.9864, achieving high-precision extraction of domain entities and relations. However, the model's performance on the general dataset decreased, with scores dropping by approximately 0.02-0.03 on some general datasets, indicating a risk of overfitting.

[0165] The baseline model (model-mixed) fine-tuned using a hybrid approach of the general dataset and the domain dataset GCCAD is the screening model provided in this embodiment. The model achieves a score of 0.9764 on the domain dataset GCCAD, while its macro_f1 and micro_f1 scores on the general dataset are 0.9004 and 0.8788, respectively, which basically maintains the general performance level.

[0166] Therefore, the method provided in this embodiment can effectively balance the generalization ability and vertical domain adaptability of the screening model, and achieve stable performance in multiple scenarios.

[0167] Specifically, the automated extraction framework of the intelligent recognition model (which integrates image parsing, filtering, and extraction models) provided in this embodiment is compared and evaluated with the standard prompting engineering solution of the Qwen2.5 open-source series of large models. The performance test results are shown in Table 4.

[0168] Table 4 Performance test results of the extraction model

[0169]

[0170] In Table 4, GC(ours) represents the intelligent recognition model provided in this embodiment, and Qwen2.5-7B, Qwen2.5-14B, Qwen2.5-32B, and Qwen2.5-72B represent four Qwen2.5 open-source large models. By comparing the accuracy on different fields and the average accuracy of all projects, it can be found that the above-mentioned intelligent recognition model can achieve 100% accuracy on sub-fields such as plot ratio, start date, completion date, construction period, and design unit, which far exceeds the performance of the Qwen2.5 open-source large model. In addition, the average extraction effect of the above-mentioned intelligent recognition model can reach 93.7%, which is more than 40% better than the standard prompt engineering solution.

[0171] In other words, the method provided in this embodiment, by combining image parsing, filtering, and extraction models, can provide intelligent field value extraction, significantly improving the accuracy and efficiency of information extraction. Compared with related methods, the accuracy is improved by more than 40%, reducing the need for manual intervention.

[0172] In addition, standardized interfaces and verification mechanisms can be set up, which helps to further improve scalability, maintainability, and high reliability of data processing.

[0173] Please refer to Figure 3 As shown, the present invention also provides a key information extraction system for construction engineering documents, including a document parsing subsystem 31, an image recognition subsystem 32, a filtering subsystem 33, and an information extraction subsystem 34.

[0174] The document parsing subsystem 31 performs normalization processing on the multi-source heterogeneous documents of the target construction project to obtain construction project documents; and identifies the text files in the construction project documents based on the text parsing tool to obtain the first text.

[0175] Image recognition subsystem 32 inputs image files from architectural engineering documents into an image parsing model for recognition to obtain the second text. The image parsing model is obtained by training an optical character recognition model based on a domain fine-tuning strategy and a color balance strategy.

[0176] The filtering subsystem 33 filters the first and second texts based on the filtering model to obtain the target text. The filtering model is obtained by fine-tuning the initial model based on a hybrid domain dataset and a general dataset. The domain dataset is obtained by processing the pre-selected dataset based on multiple semantic generalization strategies. The initial model adopts a lightweight information extraction framework based on K-class global pointers.

[0177] The information extraction subsystem 34 extracts information from the target text based on the extraction model to obtain key information, where the key information is the output result corresponding to the prompt words input to the extraction model.

[0178] The image recognition subsystem 32, filtering subsystem 33, and information extraction subsystem 34 can be integrated into a content recognition module (intelligent recognition model). The file parsing module (file parsing subsystem 31) is responsible for processing engineering files in various formats such as DWG, PDF, DOCX, and XML, extracting text, geometric, and tabular information. Based on the parsing results, the content recognition module achieves intelligent field extraction and content recognition through multi-model collaboration. The file parsing module and the content recognition module work together to achieve a complete conversion process from multi-format engineering files to structured field information. The entire system framework adopts a modular design, and the modules communicate with each other through standardized data interfaces, ensuring the system's scalability and maintainability.

[0179] The file parsing module (file parsing subsystem 31) is a fundamental component of the system, responsible for processing project files in various formats. This module employs an adapter pattern, providing a unified parsing interface for different file types. The DWG parser, implemented using the VJMap engine, is the core parsing component, specifically designed for handling CAD drawing files. This component plays a fundamental role in CAD drawing preprocessing and structured information extraction, providing standardized data input for subsequent intelligent information extraction.

[0180] The content recognition module integrates image parsing, filtering, and extraction models to achieve intelligent content filtering and information extraction. The content recognizer (the core software logic component of the content recognition module) architecture comprises a field registry, recognition engine, context management component, value extractor, classifier, and validator connected in sequence. These components interact closely to form a highly efficient and collaborative intelligent recognition system.

[0181] As the starting point of the entire content recognizer architecture, the field registry provides the recognition engine with standardized field definitions and validation rules, which directly guide the recognition engine's recognition strategy and accuracy requirements.

[0182] After receiving the field definition, the recognition engine uses the image parsing model to process the image information, providing multimodal data support for subsequent processing. It also performs content pre-screening through the filtering model and passes relevant document fragments to the context management component.

[0183] The context management component receives the initial recognition results from the recognition engine, refines and calibrates the recognition results by combining project, document and business context information, eliminates ambiguity and improves relevance, and then passes the optimized results to the value extractor.

[0184] The value extractor extracts specific values, including basic values, calculated values, and composite values, from identified field instances based on refined information provided by context management. These extraction results are then passed to the classifier for further semantic classification and label prediction.

[0185] The classifier performs multi-classification processing on the extracted data according to business needs, adds rich semantic information to the data, and finally passes the classification results to the validator for quality inspection.

[0186] As the final checkpoint in the entire process, the validator performs format validation, range checks, and consistency checks on all processed data to ensure the accuracy and reliability of the output data.

[0187] Through this close interaction, the entire content recognizer achieves fully automated processing from field definition to high-quality structured data. The information transmission and feedback mechanism between components ensures the continuity and accuracy of the recognition process.

[0188] The field registry is the core foundation of the content recognizer. By establishing a standardized field pool management system, it provides structured field definitions and validation rules for the entire recognition process. This technology enables intelligent field registration, type management, and rule configuration. Through a field pool mechanism, it uniformly manages all recognizable field types, including basic fields (such as text, numeric, and date), composite fields (such as address and contact information), and calculated fields (such as area and volume). Logically, the system first establishes a field type definition library, defining standard data structures, validation rules, and business constraints for each field type. Then, through a dynamic registration mechanism, it allows users to add new field types according to specific business needs, and the system automatically generates corresponding validation rules and recognition templates. Finally, it establishes a mapping of relationships between fields, supporting complex relationships such as field dependencies, inheritance, and combination, providing a complete field knowledge base to support subsequent intelligent recognition.

[0189] The recognition engine is the core processing component of the content recognizer. It adopts a multi-model collaborative architecture design, which organically combines the image parsing model, the filtering model and the extraction model to form a complete recognition processing chain.

[0190] In terms of image parsing model integration, the system integrates the GOT-OCR model, which is specifically designed to handle image recognition tasks in engineering documents. After training and fine-tuning, it can recognize handwritten text, special symbols, and complex graphics, providing image-dimensional data support for information extraction.

[0191] In terms of screening model integration, the system integrates an information extraction model based on the Transformer architecture (which serves as the basic network architecture providing contextual semantic encoding capabilities in a lightweight information extraction framework based on K-class global pointers). This model is specifically responsible for pre-screening and relevance evaluation of document content, and can quickly identify text fragments related to the target field, significantly improving the efficiency of subsequent processing.

[0192] The entire recognition engine ensures that the various models can work together through intelligent task scheduling and result fusion mechanisms, achieving fully automated processing from document parsing to field recognition.

[0193] Specifically, the image analysis model is designed to identify specific data within drawings, particularly adjacent text information that cannot be accurately obtained through clustering and tabular extraction. This provides a complete data foundation for subsequent information extraction. This integration expands the system's application scope, enabling it to handle more types of engineering documents.

[0194] Specifically, the text recognition process is as follows.

[0195] The system first identifies text block regions containing the target field, then expands these regions to generate text screenshots containing complete information. ; In the formula, This indicates the expanded text block area. This represents the original text block region. Indicates the expansion factor. This represents an extended function.

[0196] The recognition engine employs a multi-stage information extraction strategy, ensuring the accuracy and completeness of the extracted results through progressive filtering and precise processing. The system first retrieves document fragments containing the target field through keyword matching: ; In the formula, This indicates that the retrieved data contains the target field. A collection of documents, Indicates the first One document, This represents a keyword matching function.

[0197] Use a filtering model to filter the search results and remove documents with low relevance: ; In the formula, This represents the filtered collection of documents (the target text collection). This indicates that the retrieved data contains the target field. A collection of documents, Indicates the document relevance score. This represents the threshold.

[0198] Next, the extraction model is used to extract precise field values ​​from the filtered documents: ; In the formula, Represents the set of values ​​to be extracted. This represents the Qwen2.5 model. Represents the target field Extraction prompts, Indicates the first One document.

[0199] Finally, the extraction results are deduplicated and validated to ensure the accuracy and consistency of the final results: ; In the formula, Represents the set of values ​​after deduplication. Represents the set of values ​​to be extracted. This represents the deduplication function.

[0200] Furthermore, addressing the bottlenecks of high cost and slow speed when large language models process long documents, the recognition engine employs a three-stage collaborative strategy: keyword retrieval, pre-screening by a filtering model, and precise extraction using LLM. The core improvement lies in introducing a lightweight filtering model as a pre-filter. This model scores the semantic relevance of initially retrieved document fragments and removes low-confidence noise data based on thresholds. After this filtering, only highly relevant fragments are retained for final extraction by the large language model, achieving precise allocation of computational resources. Common methods typically input all retrieved document fragments directly into the large language model, relying on the model itself to find answers amidst massive amounts of noise, or simply perform keyword filtering. This approach requires the large model to consume significant computational power to process irrelevant information, resulting in slow inference speed, high cost, and susceptibility to illusions or extraction errors due to excessive contextual noise. In contrast, this solution utilizes an extraction model for pre-semantic refinement, significantly reducing the number of tokens and computational overhead entering the extraction model. Simultaneously, the significantly improved signal-to-noise ratio of the input data dramatically enhances the accuracy and stability of the final field extraction.

[0201] The technical solution provided by the embodiments of this invention standardizes various formats of construction engineering documents, enabling efficient reading and parsing of different file types and eliminating processing difficulties caused by format differences. By accurately annotating key fields in each file and providing accurate answers, it ensures high accuracy in automated information extraction and effectively improves data quality. A clear data classification structure is adopted to rationally organize and efficiently store file information, providing reliable support for subsequent automated extraction and verification. The above technical solution has flexible scalability, can adapt to the processing needs of more construction project documents, and promotes the digitalization and intelligentization of the construction engineering field.

[0202] The K-Global Pointer information extraction framework enables simultaneous named entity recognition and relation extraction tasks under lightweight model conditions, offering the following advantages: Lightweight and Low-Computational-Requirement Adaptability: Utilizing a pre-trained model with a small number of parameters combined with K-Global Pointer fine-tuning significantly reduces computational resource consumption and inference costs compared to large models, making it suitable for localized deployments and computationally constrained environments. Unified Modeling and Task Collaboration: Through a multi-head global pointer mechanism, entities and relations are jointly extracted within the same framework, avoiding error accumulation caused by independent modeling of entity recognition and relation extraction in related methods, thus improving overall extraction accuracy. Modularity and Scalability: The framework consists of an encoding layer, an entity recognition module, and a relation extraction module, with clear interfaces between modules, allowing for easy expansion to other downstream information extraction tasks or multilingual scenarios. High Accuracy and Robustness: Joint training and multi-head pointer modeling effectively utilize semantic association information between entities, improving extraction stability and accuracy in complex text environments.

[0203] In summary, the key information extraction method and system for building engineering documents provided by the embodiments of the present invention can improve the recognition accuracy of characters in complex CAD drawings by introducing a lightweight information extraction model and OCR technology on the basis of unified data preprocessing, and realize the integration of multi-source information at the project level. This overcomes the shortcomings of the prior art and ensures higher accuracy and stability when processing complex engineering project documents. It has the advantages of unified data organization, high extraction accuracy, low computational overhead and strong scalability.

[0204] The present invention also provides a non-transitory machine-readable medium storing a computer program, wherein the computer program, when executed by a computer's processor, is used to cause the computer to perform a method according to an embodiment of the present invention.

[0205] The present invention also provides a computer program product, including a computer program, wherein the computer program, when executed by a computer's processor, is used to cause the computer to perform the method of the embodiments of the present invention.

[0206] This invention also provides an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor. The memory stores a computer program executable by the at least one processor, which, when executed by the at least one processor, causes the electronic device to perform the method of this invention.

[0207] refer to Figure 4 This is a structural block diagram of an electronic device, either a server or a client, according to an embodiment of the present invention. It is an example of a hardware device that can be applied to various aspects of the present invention. The electronic device is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0208] like Figure 4 As shown, the electronic device includes a computing unit 401, which can perform various appropriate actions and processes based on a computer program stored in a read-only memory (ROM) 402 or a computer program loaded from a storage unit 408 into a random access memory (RAM) 403. The RAM 403 may also store various programs and data required for the operation of the electronic device. The computing unit 401, the ROM 402, and the RAM 403 are interconnected via a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.

[0209] Multiple components in the electronic device are connected to I / O interface 405, including: input unit 406, output unit 407, storage unit 408, and communication unit 409. Input unit 406 can be any type of device capable of inputting information into the electronic device. Input unit 406 can receive input digital or character information and generate key signal inputs related to user settings and / or function control of the electronic device. Output unit 407 can be any type of device capable of presenting information and may include, but is not limited to, a display, speaker, video / audio output terminal, vibrator, and / or printer. Storage unit 408 may include, but is not limited to, disks and optical discs. Communication unit 409 allows the electronic device to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks, and may include, but is not limited to, modems, network cards, infrared communication devices, and / or wireless communication transceivers, such as Bluetooth devices, WiFi devices, WiMax devices, cellular communication devices, and / or the like.

[0210] The computing unit 401 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 401 include, but are not limited to, CPUs, graphics processing units (GPUs), various special-purpose artificial intelligence (AI) computing units, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any suitable processor, controller, microcontroller, etc. The computing unit 401 performs the various methods and processes described above. For example, in some embodiments, the method embodiments of the present invention can be implemented as computer programs tangibly contained in a machine-readable medium, such as storage unit 408. In some embodiments, part or all of the computer program can be loaded and / or installed on an electronic device via ROM 402 and / or communication unit 409. In some embodiments, the computing unit 401 can be configured to perform the methods described above by any other suitable means (e.g., by means of firmware).

[0211] Computer programs for implementing the methods of embodiments of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0212] In the context of embodiments of this invention, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable signal medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, or infrared systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0213] It should be noted that the term "comprising" and its variations used in the embodiments of this invention are open-ended, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The modifications of "one" and "a plurality" mentioned in the embodiments of this invention are illustrative and not restrictive, and those skilled in the art should understand that unless explicitly indicated otherwise in the context, they should be understood as "one or more". The descriptions of terms such as "first", "second", etc., are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of indicated technical features.

[0214] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the embodiments of this invention are all information and data authorized by the user or fully authorized by all parties.

[0215] The steps described in the method embodiments provided by the present invention can be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of protection of the present invention is not limited in this respect.

[0216] The term "embodiment" in this specification refers to a specific feature, structure, or characteristic described in connection with an embodiment that may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily imply the same embodiment, nor does it imply that it is independent of or alternative to other embodiments. The various embodiments in this specification are described in a related manner, and similar or identical parts between embodiments can be referred to mutually.

[0217] The above-described embodiments are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of protection. It should be noted that those skilled in the art can make various modifications and improvements without departing from the inventive concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the appended claims.

Claims

1. A method for extracting key information from construction project documents, characterized in that, include: The multi-source heterogeneous files of the target construction project are standardized and annotated to obtain the construction project files; The first text is obtained by recognizing the text files in the construction project documents using a text parsing tool; The image files in the architectural engineering documents are input into the image parsing model for recognition to obtain the second text. The image parsing model is obtained by training an optical character recognition model based on a domain fine-tuning strategy and a color balance strategy. The first text and the second text are filtered based on the filtering model to obtain the target text. The filtering model is obtained by fine-tuning the initial model based on a hybrid domain dataset and a general dataset. The domain dataset is obtained by processing the pre-selected dataset based on multiple semantic generalization strategies. The initial model adopts a lightweight information extraction framework based on K-class global pointers. Information is extracted from the target text based on the extraction model to obtain key information, wherein the key information is the output result corresponding to the prompt words input into the extraction model.

2. The method according to claim 1, characterized in that, Before inputting the image files from the architectural engineering documents into the image parsing model for recognition, the method further includes: Based on engineering drawings, multiple image-text pairs are obtained to construct an image-text dataset. The engineering drawings include drawings with multiple layers and drawings where characters are covered by lines. The number of image-text pairs is less than 10k. The number of samples with different background colors and font colors in the image-text dataset is a first preset ratio for color equalization. The optical character recognition model is trained using the DeepSpeed ​​optimization framework and the image dataset to obtain the image parsing model.

3. The method according to claim 2, characterized in that, Based on engineering drawings, multiple image-text pairs are obtained to construct an image-text dataset, including: Logical clusters are obtained from the engineering drawings based on a text clustering algorithm, wherein the query radius of the text clustering algorithm is set to 2-3 times the average size of the text blocks in the logical cluster, and the center point of the text block is calculated based on the minimum and maximum coordinates of the bounding box. The text blocks in the logical cluster are filled using a fill-in-the-blank text filling algorithm to obtain integrated text. The filling conditions of the fill-in-the-blank text filling algorithm are related to the overlap length, overlap ratio, and horizontal position relationship. Based on the integrated text and the first preset ratio, multiple image-text pairs are determined to construct the image-text dataset.

4. The method according to claim 1, characterized in that, Before filtering the first text and the second text based on the filtering model, the method further includes: Retrieves raw text data from various types of project files; Based on the type of the target construction project, sentences containing the target field are selected from the original text data, and the entities in the target field are labeled to obtain the pre-selected dataset; The preselected dataset is generalized using multiple semantic generalization strategies to obtain the domain dataset; The initial model of the screening model is trained based on the domain dataset and the general dataset to obtain the screening model.

5. The method according to claim 4, characterized in that, The preselected dataset is generalized using multiple semantic generalization strategies to obtain the domain dataset, including: The preselected dataset is generalized using multiple semantic generalization strategies to obtain multiple generalized datasets with equal sample sizes. The generalized dataset is filtered to obtain the domain dataset.

6. The method according to claim 5, characterized in that, The pre-selected dataset is generalized using multiple semantic generalization strategies to obtain various generalized datasets with equal sample sizes, including: Based on the first semantic generalization strategy of keeping the reference text unchanged, changing the entity name, and retaining the field value, the preselected dataset is generalized to obtain the first generalized dataset. Based on the second semantic generalization strategy of changing the reference text, keeping the entity name unchanged, and retaining the field values, the preselected dataset is generalized to obtain the second generalized dataset; Based on the third semantic generalization strategy of changing reference text, changing entity name, and retaining field value, the preselected dataset is generalized to obtain the third generalized dataset. Based on the fourth semantic generalization strategy of changing reference text, changing entity name, omitting or missing field values, the preselected dataset is generalized to obtain the fourth generalized dataset. Based on the fifth semantic generalization strategy of changing reference text, keeping entity names unchanged, and omitting or missing field values, the preselected dataset is generalized to obtain the fifth generalized dataset. The number of samples in the first generalized dataset, the second generalized dataset, the third generalized dataset, the fourth generalized dataset, and the fifth generalized dataset are equal.

7. The method according to claim 4, characterized in that, Before training the initial model of the screening model based on the domain dataset and the general dataset, the method further includes: The ratio between the number of samples in the general dataset and the number of samples in the domain dataset is determined based on the type of the target construction project, and the ratio is greater than 20:

1. The number of samples in the general dataset is determined based on the number of samples in the domain dataset and the proportion. The general dataset is obtained by extracting publicly available Chinese information based on the sample size of the general dataset.

8. The method according to claim 1, characterized in that, Based on the filtering model, the first text and the second text are filtered to obtain the target text, including: The relevance scores of the first text and the second text to the target field are calculated based on the filtering model. The first and second texts with relevance scores greater than a preset threshold are selected as the target texts.

9. The method according to claim 1, characterized in that, The multi-source heterogeneous files of the target construction project are standardized and annotated to obtain construction project documents, including: The aforementioned multi-source heterogeneous documents are classified into the following categories: pricing documents, tender documents, general architectural design specifications, business format documents, and civil defense documents; Based on file format, files in each category are categorized by format to obtain the files to be labeled; Based on a preset field table, the fields in the file to be annotated are annotated with actual values ​​to obtain annotated field information; The architectural engineering file is determined based on the file to be annotated and the annotation field information.

10. A system for extracting key information from construction project documents, characterized in that, include: The document parsing subsystem performs normalization processing on the multi-source heterogeneous documents of the target construction project to obtain the construction project documents; The first text is obtained by recognizing the text files in the construction project documents using a text parsing tool; The image recognition subsystem inputs the image files in the architectural engineering documents into the image parsing model for recognition to obtain the second text. The image parsing model is obtained by training an optical character recognition model based on a domain fine-tuning strategy and a color balance strategy. A filtering subsystem filters the first text and the second text based on a filtering model to obtain the target text. The filtering model is obtained by fine-tuning an initial model based on a hybrid domain dataset and a general dataset. The domain dataset is obtained by processing a pre-selected dataset based on multiple semantic generalization strategies. The initial model adopts a lightweight information extraction framework based on K-class global pointers. The information extraction subsystem extracts information from the target text based on the extraction model to obtain key information, wherein the key information is the output result corresponding to the prompt words input into the extraction model.

Images