Method for automatically extracting and classifying semi-structured engineering drawing information of engineering cost based on OCR (Optical Character Recognition) and large model classification

Abstract

The invention discloses an automatic extraction and classification method for engineering cost semi-structured engineering drawing information based on OCR (optical character recognition) and large model classification. The method comprises the following steps: S1, converting an engineering drawing PDF (portable document format) document into an RGB (red, green and blue) image; s2, acquiring configuration information of each field region in the RGB image; s3, intelligently cutting out a sub-region image of each piece of engineering drawing; s4, identifying the number or character information of the sub-region image and outputting structured table data; s5, inputting each record and the classification knowledge base into an Embedding model for similarity matching; and S6, collaboratively inputting each record in the structured table data and two corresponding classification data with the highest similarity into the large model for precise classification. By applying the method, the degree of manual intervention can be greatly reduced, the accuracy of data extraction and classification is improved, and the defects that engineering drawing data processing consumes a large amount of manpower, data is prone to being poor and leaked, and data classification is tedious are overcome.

Description

Technical Field

[0001] This invention belongs to the field of large-scale engineering application technology, and in particular relates to a method for automatically extracting and classifying semi-structured engineering drawing information based on OCR recognition and large-scale model classification. Background Technology

[0002] With the deepening of digital transformation, enterprises and institutions have accumulated a large number of semi-structured documents (such as PDF forms and drawings), which contain a large amount of key business data (such as numerical values ​​and specifications). Currently, data is mainly entered manually for each drawing in the PDF, which is labor-intensive, prone to errors and omissions, and cumbersome to verify. Specifically, existing technologies have the following defects and shortcomings:

[0003] ① Currently, manual data entry and classification are inefficient and easily affected by subjective factors, making it difficult to handle batch document scenarios; lack of precise positioning of target areas makes it difficult to extract key data from complex backgrounds (such as documents with mixed text and images), resulting in redundant or missing extraction results.

[0004] ② Traditional OCR technology analyzes the layout of engineering drawings to extract all information, including tables and text, from the drawings. However, manual extraction and summarization of key data are still required. Furthermore, the accuracy of data obtained by OCR decreases in engineering drawings with a large amount of information, and it is difficult to integrate structured table data from all images, making it difficult to meet the needs of business scenarios.

[0005] ③ Currently, classification based on keyword matching or fixed templates cannot understand semantic context and has weak ability to handle vague expressions, synonyms, or cross-domain terms.

[0006] Therefore, the efficient extraction and accurate classification of semi-structured data are the core links to realizing data assetization and supporting intelligent decision-making, and there is an urgent need for this in the field of engineering cost. Summary of the Invention

[0007] The technical problem to be solved by this invention is to provide an intelligent and accurate method for automatically extracting and classifying semi-structured engineering drawing information based on OCR recognition and large model classification, so as to realize the extraction of key information from engineering drawings, the output of structured tabular data, and data classification.

[0008] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0009] An automatic extraction and classification method for semi-structured engineering drawing information based on OCR recognition and large model classification includes the following steps:

[0010] S1 converts each page of the engineering drawing PDF document into RGB images to ensure that text, tables, charts and other elements are clearly distinguishable;

[0011] S2, based on the interactive annotation tool, collects the configuration information of each field region in the RGB image;

[0012] S3 intelligently cuts out sub-region images of each engineering drawing based on configuration information;

[0013] S4, based on OCR recognition, recognizes the digital or text information of the sub-region image and outputs structured tabular data;

[0014] S5: Based on the structured table data, each record and the classification knowledge base are input into the Embedding model for similarity matching, and the two classification data with the highest similarity for each record are filtered and output.

[0015] S6. To achieve accurate classification, each record in the structured table data and its two most similar classification data are input together into the large model for accurate classification, and the final classification result is obtained.

[0016] In step S1, the RGB image resolution is ≥300dpi.

[0017] Step S2 includes the following operations:

[0018] S201, Read the relative coordinates of the RGB image and a certain field region in the PDF, and calculate the pixel coordinates of the sub-region in the image; the width of the RGB image is W (unit: px) and the height is H (unit: px), and the relative coordinates of the certain field region in the PDF are (x1, y1, x2, y2), where x1 and x2 are the relative width ratios, and y1 and y2 are the relative height ratios, with values ​​ranging from... Pixel coordinates (X1, Y1, X2, Y2) satisfy the following formula:

[0019] S202 addresses potential rotation issues in engineering drawings (such as y-axis labels), by independently setting rotation angles of 90°, 180°, 270°, or 360° (i.e., 0°) for each sub-region image, ensuring that the numbers in the image remain horizontally and upright.

[0020] S203, interactively set the field name for each sub-region, then only the value corresponding to that field needs to be extracted at the end of the sub-region image, thus filtering out the original noise data to the greatest extent.

[0021] S204 outputs the region configuration information for each field. The region configuration information for each field includes the pixel coordinates, rotation angle, and field name of the sub-region.

[0022] Step S3 includes the following operations:

[0023] S301, obtain the corresponding region configuration information, traverse each field region, and crop out each sub-region image;

[0024] S302: Since the size of the sub-region images is generally small, borders are added to all sub-region images to adapt to the recognition requirements of OCR for single floating-point scenarios. This solves the problem of low recognition accuracy caused by insufficient image feature integrity and lack of contextual association when there are no borders. By strengthening regional saliency and feature integrity, the OCR recognition accuracy is improved.

[0025] S303: Traverse the original RGB images, perform S301 and S302 operations on each RGB image, and crop out the sub-region image of each RGB image.

[0026] Step S4 includes the following operations:

[0027] To adapt to the recognition of cost engineering drawing data, the following configurations were made for the basic OCR parameters to ensure that the data in the sub-region images can be correctly recognized;

[0028] S401, disable the text orientation classification function of OCR; before OCR processing, perform independent rotation correction on the target sub-region according to the rotation angle in the configuration information. This solution is suitable for scenarios where the text orientation of the sub-region image is upright. It can avoid the recognition error of easily confusing the numbers 6 and 9 when orientation classification is enabled, and improve the recognition accuracy while increasing the processing efficiency.

[0029] S402, Enable text detection function; a basic step in OCR, text regions must be located before content recognition. In practice, it has been found that disabling this function improves the recognition accuracy of sub-region images containing only a single floating-point number; while enabling it, to avoid a decrease in recognition accuracy in such scenarios, a border needs to be added to the sub-region image before inputting it into the OCR. This processing adapts to the OCR recognition needs of single floating-point number scenarios, solving problems such as incomplete image features and lack of contextual association caused by the absence of borders, thereby improving the low recognition accuracy.

[0030] S403 does not impose restrictions on the aspect ratio and height of the text region. Given that the sub-region images cropped from the original RGB image generally contain only a single floating-point number and have a small image size, keeping the aspect ratio and height unrestricted can prevent important text information from being mistakenly deleted by the filtering mechanism.

[0031] Step S5 includes the following operations:

[0032] S501, Prepare a classification standard knowledge base (such as including basic types, topographic and geological features and corresponding geological classifications), and extract key fields (such as basic types and topographic and geological features) from the OCR recognition results; among them, topographic and geological features need to be classified based on semantics.

[0033] S502, the key fields (such as basic type, terrain and geological features) of the data in the classification standard knowledge base and the OCR recognition results are respectively vector-embedded and converted into 1024-dimensional vectors with semantic representation capabilities; if we assume that the vector of a certain category in the knowledge base is... (This represents the total number of knowledge base categories). The vector of a specific record in the OCR recognition result is... (To record the total number);

[0034] S503, calculate the cosine similarity between each record in the OCR recognition result and each category in the classification standard knowledge base, using the following formula:

[0035]

[0036] in Let L2 norms be the L2 norms of the two vectors, respectively.

[0037] S504. Based on the calculated similarity value, select the two classification data with the highest similarity to the classification knowledge base for each record.

[0038] Step S6 includes the following operations:

[0039] In particular, during the similarity matching stage, the scores of the highest similarity category and the second highest similarity category are very close in most cases, so the category data corresponding to the highest score is not directly determined as the best classification result;

[0040] S601, construct an input sequence containing multi-level information, and input the original data to be classified (i.e., the structured record obtained by OCR recognition) and the two types of high similarity classification data recalled in the similarity matching stage (including their complete feature description in the knowledge base) into the large model;

[0041] S602, through carefully designed prompt word engineering (such as clearly defining the classification task objectives, feature matching rules, and output format requirements), guides the large model to deeply analyze text semantics, associate category features, and data attributes, and finally outputs a unique and accurate classification result.

[0042] The prompt word engineering includes clarifying the classification task objectives, feature matching rules, and output format requirements; the main model is the Qwen2.5 model.

[0043] The system implementing the above-mentioned automatic information extraction and classification methods mainly consists of an image acquisition module, an image recognition module, a similarity matching module, and a large model classification module. Each module is loosely coupled and integrated through API interfaces, and the performance of a single module can be expanded according to business needs, significantly improving the efficiency and accuracy of semi-structured document data processing.

[0044] The image acquisition module proceeds to steps S1 to S3, the image recognition module proceeds to step S4, the similarity matching module proceeds to step S5, and the large model classification module proceeds to step S6.

[0045] To address the current problems of low efficiency and poor accuracy in manually entering and classifying engineering drawing data, the inventors have developed an automatic extraction and classification method for semi-structured engineering drawing information based on OCR recognition and large-scale model classification. The method includes the following steps: S1, converting each page of the engineering drawing PDF document into RGB images; S2, collecting the configuration information of each field region in the RGB images using an interactive annotation tool; S3, intelligently cropping sub-region images from each engineering drawing based on the configuration information; S4, recognizing the numerical or textual information in the sub-region images using OCR recognition and outputting structured tabular data; S5, inputting each record and classification knowledge base into the Embedding model for similarity matching based on the structured tabular data, filtering and outputting the two most similar classification data for each record; S6, collaboratively inputting each record in the structured tabular data and its two most similar classification data into the large-scale model for accurate classification, obtaining the final classification result. This invention uses OCR recognition and large model classification to extract key information from engineering drawings, output structured tabular data, and classify data. It significantly reduces the degree of manual intervention, improves the accuracy of data extraction and classification, and overcomes the shortcomings of engineering drawing data processing, such as consuming a lot of manpower, data being prone to errors and omissions, and data classification being relatively cumbersome. Attached Figure Description

[0046] Figure 1 This is a flowchart illustrating the automatic extraction and classification method for semi-structured engineering drawings based on OCR recognition and large model classification, according to the present invention. Detailed Implementation

[0047] Example Technical Background

[0048] There are three major problems with the manual processing of existing semi-structured PDF geological maps: ① Field extraction relies on manual input, which is inefficient and prone to errors; ② Soil type classification relies on experience, and the consistency of judgment on fuzzy features (such as "silty clay, soft plastic") is poor; ③ The summary of engineering quantities requires manual calculation, which is time-consuming and difficult to adapt to diverse geological scenarios.

[0049] Therefore, the applicant implemented a fully automated solution by referring to the above-mentioned method and system for automatic extraction and classification of semi-structured engineering drawing information based on OCR recognition and large model classification. The specific process is as follows:

[0050] II. Specific Process

[0051] (a) Dynamic template configuration: Enables visual definition and flexible adaptation of fields.

[0052] 1. Field definition and coordinate configuration

[0053] The user selects the target field (such as "Serial Number" or "Topographical and Geological Overview") on the PDF page using a front-end visualization tool. The user then enters the field name, and the system automatically records the relative coordinates (with the top-left corner of the page as the origin, the x-axis pointing horizontally to the right, and the y-axis pointing vertically downwards; the coordinate value is the ratio of the distance from the origin to the total page length, ranging from 0 to 1), and generates a JSON configuration template. An example is shown below:

[0054]

[0055]

[0056]

[0057] Cross-line area

[0058]

[0059] 2. Special Text Processing

[0060] For vertically arranged numbers or text in drawings, the rotation angle (such as 90° / 270°) of the selected area can be set through the front-end tool. The system will automatically apply rotation correction in the subsequent recognition stage to ensure the accuracy of OCR recognition.

[0061] (II) PDF to Image Conversion and Region Extraction: Preprocessing for Structured Data Extraction

[0062] 1. PDF to Image Conversion

[0063] The system calls the PDFToImageConverter class (which can be implemented using existing open-source libraries such as PyMuPDF) to convert each page of the PDF into a 300dpi RGB image, ensuring text clarity.

[0064] 2. Region coordinate transformation and interception

[0065] Calculate the actual pixel area based on the relative coordinates in the template: Let the page pixel size be... Then the actual pixel coordinates corresponding to the relative coordinates (x, y) are: .

[0066] Example: Relative coordinates of the "Topography, Geological Overview" field The system extracts the rectangular area (280, 200, 560, 400) as the object to be identified.

[0067] (III) OCR Recognition and Result Integration: Converting Image Data into Structured Text

[0068] 1. Regional OCR Recognition

[0069] The RapidOCR engine (a publicly available OCR tool) is used to recognize the captured area image. The text direction classification function is turned off (because the template has a preset direction), the text detection and recognition function is enabled, and borders are added to single floating-point areas (such as "buried depth") to improve recognition accuracy.

[0070] 2. Results Examples and Integration

[0071] For example, the recognition result for "Topography and Geological Overview": : Powdered clay, grayish-yellow, wet, soft to malleable. Silty clay, yellowish-brown, hard plastic, containing a small amount of gravel and iron-manganese nodules;

[0072] "Foundation Type" identification result: "Cast-in pile";

[0073] The burial depth identification result is "3.0m".

[0074] The system uses the pandas library to consolidate the recognition results of 50 PDFs into an Excel spreadsheet, as shown in the example below:

[0075] Table 1 PDF Recognition Results

[0076]

[0077] (iv) Example of automatic geological type classification: intelligent judgment based on knowledge base and large model

[0078] 1. Data Preprocessing

[0079] The system uses pandas to parse the "topography and geological overview" fields, splits the depth and soil characteristics, and generates the following intermediate table:

[0080] Table 2. Pandas Intermediate Table Parsing

[0081]

[0082] 2. Knowledge base loading and embedding

[0083] (1) Load the geological standard knowledge base (Excel format), as shown in the example below:

[0084] Table 3 Geological Standards Knowledge Base

[0085]

[0086] (2) Call load_knowledge through the LLMClassifier class The embeddings function uses BGE The M3 model (a publicly available text embedding model, obtainable through official channels) generates a 1024-dimensional embedding vector for each "geological feature" and stores it in knowledge. The dictionary of embeddings.

[0087] 3. Similarity matching

[0088] For the preprocessed soil features (such as "fine sand, loose"), the BGE-M3 model is used to generate embedding vectors, and the matching degree with the knowledge base features is calculated using the following cosine similarity formula:

[0089]

[0090] Where Q is the feature vector to be classified, and K is the feature vector of the knowledge base. Example: The basic type is "drilled pile", the geological feature is "silty clay (plastic, soft plastic) soil", and the similarity with the corresponding feature in the knowledge base is 0.92, matching "ordinary soil".

[0091] 4. Fuzzy classification correction for large models

[0092] For fuzzy features with similarity (such as "silty clay, soft plastic"), the system generates prompt words (such as "based on the knowledge base 'drilled pile - silty clay (plastic, soft plastic) → ordinary soil,' classifying the soil and rock as 'residual soil, mainly grayish-yellow, plastic to hard, containing debris and plant roots and stems'), and calls the Qwen2.5 model (a public large language model, following the official API calling specifications) to output the classification results, ensuring accuracy.

[0093] (v) Automatic calculation of project quantities: Multi-dimensional summarization based on pandas

[0094] Based on the classification results and depth data, the system automatically summarizes the quantities of each type of work using pandas and generates a results table.

[0095] Table 4. Results of pandas

[0096]

[0097] In summary, the embodiments fully realize the automated processing from semi-structured drawings to structured engineering quantities, verify the effectiveness of the technical solution of the present invention, and achieve the following technical effects:

[0098] 1. Efficiency Improvement: The processing time for 50 data entries was reduced from 4 hours manually to 8 minutes, a 30-fold increase in efficiency;

[0099] 2. Accuracy: Through knowledge base matching and large model correction, the classification accuracy reaches 95%, which is higher than the 85% of manual classification.

[0100] 3. Scalability: When adding a new geological type (such as "sand and gravel, medium dense"), you only need to add the corresponding relationship of "basic type - geological feature - soil and rock classification" to the knowledge base. The system will automatically recognize it through the embedded model without modifying the core code.

Claims

1. A method for automatically extracting and classifying semi-structured engineering drawing information for engineering cost estimation based on OCR recognition and large model classification, characterized in that... Includes the following steps: S1 converts each page of the engineering drawing PDF document into an RGB image; S2, based on the interactive annotation tool, collects the configuration information of each field region in the RGB image; S3 intelligently cuts out sub-region images of each engineering drawing based on configuration information; S4, based on OCR recognition, recognizes the digital or text information of the sub-region image and outputs structured tabular data; S5: Based on the structured table data, each record and the classification knowledge base are input into the Embedding model for similarity matching, and the two classification data with the highest similarity for each record are filtered and output. S6 inputs each record in the structured table data and its two most similar classification data into the large model for accurate classification, and obtains the final classification result.

2. The information automatic extraction and classification method according to claim 1, characterized in that... In step S1, the RGB image resolution is ≥300dpi.

3. The information automatic extraction and classification method according to claim 1, characterized in that... Step S2 includes the following operations: S201, read the relative coordinates of the RGB image and a certain field region in the PDF, and calculate the pixel coordinates of the sub-region in the image; the width of the RGB image is W and the height is H, and the relative coordinates of the certain field region in the PDF are (x1, y1, x2, y2), where x1 and x2 are the relative width ratios, and y1 and y2 are the relative height ratios, with values ​​ranging from 0 to 1; the pixel coordinates (X1, Y1, X2, Y2) satisfy the following formula: X1 = x1 × W, Y1 = y1 × H, X2 = x2 × W, Y2 = y2 × H; S202 allows for independent setting of rotation angles of 90°, 180°, 270°, or 360° for each sub-region image, ensuring that the numbers in the image are displayed horizontally and upright. S203, interactively set the field names for each sub-region; S204 outputs the region configuration information for each field. The region configuration information for each field includes the pixel coordinates, rotation angle, and field name of the sub-region.

4. The information automatic extraction and classification method according to claim 1, characterized in that... Step S3 includes the following operations: S301, obtain the corresponding region configuration information, traverse each field region, and crop out each sub-region image; S302 adds borders to all sub-region images to adapt to the OCR recognition requirements for single floating-point scenarios; S303: Traverse the original RGB images, perform S301 and S302 operations on each RGB image, and crop out the sub-region image of each RGB image.

5. The method for automatic information extraction and classification according to claim 1, characterized in that... Step S4 includes the following operations: S401, disable the text direction classification function of OCR; S402, enable text detection function; S403 does not impose restrictions on the aspect ratio and height of the text area.

6. The method for automatic information extraction and classification according to claim 1, characterized in that... Step S5 includes the following operations: S501, Prepare a classification standard knowledge base and extract key fields from the OCR recognition results; S502, the key fields of the data in the classification standard knowledge base and the OCR recognition results are respectively vector-embedded and converted into 1024-dimensional vectors with semantic representation capabilities; let K be the vector of a certain category in the knowledge base. i (i=1,2,...,n, where n is the total number of knowledge base categories), the vector of a record in the OCR recognition result is R. j (j=1,2,...,m, where m is the total number of records); S503, calculate the cosine similarity between each record in the OCR recognition result and each category in the classification standard knowledge base, using the following formula: sim(R j ,K i )=(R j ·K i ) / (||R j ||·||K i ||), Among them, R j ·K i Let ||R| be the vector dot product. j ||、||K i || represents the L2 norm of the two vectors; S504. Based on the calculated similarity value, select the two classification data with the highest similarity to the classification knowledge base for each record.

7. The method for automatic information extraction and classification according to claim 1, characterized in that... Step S6 includes the following operations: S601, construct an input sequence containing multi-level information, and input the original data to be classified and the two types of highly similar classification data recalled in the similarity matching stage into the large model; S602, through prompt word engineering, guides a large model to deeply analyze text semantics, related category features and data attributes, and finally outputs a unique and accurate classification result.

8. The method for automatic information extraction and classification according to claim 7, characterized in that: The prompt word engineering includes clearly defining the classification task objectives, feature matching rules, and output format requirements; the large model is the Qwen2.5 model.

9. A system for implementing the information automatic extraction and classification method according to any one of claims 1 to 8, characterized in that... It mainly consists of an image acquisition module, an image recognition module, a similarity matching module, and a large model classification module. These modules are loosely coupled and integrated through API interfaces.

10. The system according to claim 9, characterized in that: The image acquisition module proceeds to steps S1 to S3, the image recognition module proceeds to step S4, the similarity matching module proceeds to step S5, and the large model classification module proceeds to step S6.

Images