A concrete detection report analysis system and method combining regular rules and spatial coordinate regression

Abstract

This invention discloses a concrete testing report parsing system and method combining regularization rules and spatial coordinate regression. The system includes a file scanning module, a report type identification module, a coordinate positioning and extraction module, a data cleaning module, a storage module, and an anomaly handling module. The method includes: recursively scanning the root directory and subfolders to obtain PDF files; extracting the first page text and determining the report type based on regular expression matching of a preset keyword set; loading a field coordinate mapping table according to the report type and accurately extracting text within a specified coordinate area; formatting data according to the field type using intelligent cleaning rules; capturing anomalies and continuing to process subsequent files; and associating and storing the structured data with file metadata for output. This invention combines coordinate positioning and regularization rules to achieve automatic identification of multiple report types, accurate field extraction, intelligent data cleaning, batch recursive processing, anomaly tolerance, and traceable output.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, specifically to a concrete inspection report parsing system and method that combines regularization rules and spatial coordinate regression. Background Technology

[0002] Data extraction from concrete testing reports is a core component of building materials engineering quality informatization. The concrete and its raw material testing report system includes eight types: concrete compressive strength testing, fly ash testing, sand and gravel aggregate testing, cement physical property testing, admixture testing, mix design, and ready-mixed concrete quality certificates. These reports are compiled according to national standards such as GB / T 50107-2010 and are characterized by diverse formats, standardized layouts, and fixed field positions. Key fields include: inspection number, sampling date, strength grade, physicochemical indicators, and test results. Enterprises need to extract structured data from massive amounts of PDFs to support scenarios such as quality statistical analysis, historical traceability, mix design optimization, and regulatory reporting. Existing mainstream methods include: Manual data entry or semi-automatic copying: Each test report is opened manually, and key fields are entered into the system, or copy-paste is used to assist with entry. While this method offers controllable accuracy, it is inefficient, costly, and prone to omissions, errors, and inconsistencies as data volume increases, making it unsuitable for engineering applications.

[0003] OCR-based full-text recognition technology converts PDF or scanned reports into editable text using optical character recognition (OCR) technology, and then extracts target fields through keyword matching. This method is effective in scenarios with simple layouts and continuous text, but in concrete testing reports, due to complex table lines, dense fields, and high numerical similarity, OCR recognition errors are significant, and different fields are easily confused, leading to unstable parsing accuracy.

[0004] Methods based on full-text rule matching: Some existing solutions directly perform regular expression matching or keyword positioning on the full-text extracted from the PDF to obtain the target data. However, these methods do not consider the spatial relationship of fields on the page. When there are multiple similar field names or values ​​on the same page, mismatch problems are likely to occur, especially in multi-table, multi-page reports, resulting in low reliability of the parsing results.

[0005] Template-based customized parsing solution: For a specific report template, the field positions are manually marked in advance and the corresponding parsing logic is written, thereby achieving a high parsing accuracy. Summary of the Invention

[0006] This invention addresses the shortcomings of existing technologies by providing a method and system for parsing concrete test reports that combines regularization rules and spatial coordinate regression. It aims to solve the technical problems in existing technologies, such as weak type recognition capability, poor field positioning accuracy, rigid data cleaning rules, insufficient batch processing capability, poor result traceability, high configuration and maintenance costs, and imperfect error handling.

[0007] To achieve the above objectives, the present invention adopts the following technical solution: A concrete testing report parsing system combining regularization rules and spatial coordinate regression includes a file scanning module 1, a report type identification module 2, a coordinate positioning and extraction module 3, a data cleaning module 4, a storage module 5, and an anomaly handling module 6. The file scanning module 1 obtains the testing report file in PDF format from an external storage medium. The report type identification module 2 is connected to the file scanning module 1, the coordinate positioning and extraction module 3 is connected to the report type identification module 2, the data cleaning module 4 is connected to the coordinate positioning and extraction module 3, the storage module 5 is connected to the data cleaning module 4, and the anomaly handling module 6 is connected to the file scanning module 1, the report type identification module 2, the coordinate positioning and extraction module 3, the data cleaning module 4, and the storage module 5, respectively.

[0008] Furthermore, the document scanning module 1 transmits the path signal and document data signal of the detection report file to the report type identification module 2; the report type identification module 2 transmits the report type identifier signal to the coordinate positioning and extraction module 3; the coordinate positioning and extraction module 3 transmits the field text signal to the data cleaning module 4; the data cleaning module 4 transmits the structured data signal to the storage module 5; the storage module 5 transmits the output data signal to the outside; when any module generates an anomaly, that module transmits the anomaly signal to the anomaly handling module 6 and transmits the hold signal to other modules.

[0009] Furthermore, the report type identification module 2 determines the report type based on a preset keyword set. This keyword set is configured with multiple feature keywords for each report type and supports regular expressions. The report type identification module 2 calculates the matching degree between the extracted text and the keyword set of each type. When the matching degree reaches a preset threshold, the report type is determined. If there is no type match, it is marked as unrecognized and the processing is skipped.

[0010] Furthermore, the coordinate positioning and extraction module 3 is configured with a field coordinate mapping table. The field coordinate mapping table uses the field name as the key and a quadruple containing the start point of the horizontal coordinate, the start point of the vertical coordinate, the end point of the horizontal coordinate, and the end point of the vertical coordinate as the value. For a detection report containing multiple pages, the coordinate mapping table further stores the coordinate configuration of different pages according to the page number index. The coordinate positioning and extraction module 3 calls the corresponding coordinate configuration according to the page number index to extract cross-page fields.

[0011] Furthermore, the data cleaning module 4 is configured with intelligent cleaning rules, including: date field cleaning rules, numeric field cleaning rules, number field cleaning rules, and text field cleaning rules; the date field cleaning rules use regular expressions to identify various date formats and uniformly convert them into preset standard date formats; the numeric field cleaning rules use regular expressions to extract the numbers and decimal points, and remove unit symbols and non-numeric characters; the number field cleaning rules remove spaces and newlines to maintain the continuity of the number string; and the text field cleaning rules remove leading and trailing whitespace characters to retain the original text content.

[0012] Furthermore, the structured data output by storage module 5 includes: independent files stored according to report type, and summary files that merge all types of data; the file metadata includes at least the file name, report type, folder path where the file is located, absolute file path, and extraction time.

[0013] Furthermore, the file scanning module 1 uses a recursive traversal method of the path library to automatically discover all files with specified extensions; during the processing, it calculates and outputs the processing progress in real time; after processing is completed, it counts and outputs the total number of files, the number of successfully processed files, the number of failed files, and the number of files skipped, and distributes them statistically according to report type.

[0014] Furthermore, the keyword set in the report type identification module 2 and the field coordinate mapping table in the coordinate positioning extraction module 3 are centrally defined and managed through a configuration dictionary; by modifying the configuration dictionary, report types and their corresponding identification keywords and field coordinate mapping relationships can be added, deleted, or adjusted.

[0015] This invention also provides a method for parsing concrete test reports that combines regularization rules and spatial coordinate regression, comprising the following steps: S1: File scanning module 1 recursively scans the specified root directory and all subfolders, automatically discovers and obtains all PDF format detection report files, and generates a list of files to be processed; S2: For each file to be processed, the report type identification module 2 extracts the homepage text and matches it based on a preset set of keywords containing regular expressions to automatically determine the type of the current report; S3: The coordinate positioning and extraction module 3 calls the pre-configured field coordinate mapping table according to the determined report type. This mapping table defines the rectangular area coordinates of different fields in the PDF page. Using the coordinate system of the PDF parsing library, the text content is accurately extracted within the rectangular area corresponding to each field. S4: Data cleaning module 4 applies preset intelligent cleaning rules to format the extracted text according to the type of the extracted fields, generating structured data; S5: When an exception occurs in any of the aforementioned steps during the processing of each file, the exception handling module 6 captures the exception and continues to process subsequent files without interrupting the overall process. S6: Storage module 5 associates and stores the structured data with the corresponding file metadata, and outputs it as a structured file.

[0016] Furthermore, step S5 also includes: recording the exception information to the list of failed files, and simultaneously outputting the log information of the processing process to the log file and the console; after the processing is completed, counting and outputting the total number of files, the number of successful processing, the number of failures, and the number of skips.

[0017] Compared with traditional solutions, the present invention has the following advantages: (1) The extraction efficiency is greatly improved. It takes 5-10 minutes to manually enter a single report, while this system processes a single report in less than 1 second and batch processes 1,000 reports in about 10-20 minutes, which is hundreds of times more efficient.

[0018] (2) The coordinate positioning method used in this invention has an accuracy of over 99%, avoiding visual fatigue and input errors caused by manual input. Combined with intelligent cleaning, the output data quality far exceeds that of manual and general OCR.

[0019] (3) This invention supports seamless switching of more than 8 report types, including concrete compressive strength test, fly ash test, sand and gravel aggregate test, cement physical property test, admixture test, mix design, and ready-mixed concrete quality certificate. The keyword recognition accuracy rate is greater than 95%, and all types can be processed in a single run.

[0020] (4) The method described in this invention has strong batch processing capabilities, supports recursive traversal of any folder structure, and automatically discovers all PDFs; the fault tolerance mechanism ensures long-term running stability and is suitable for projects with thousands to tens of thousands of files.

[0021] (5) Each data entry is associated with the complete file path and extraction time. When tracing quality issues, the original PDF can be quickly located, meeting the requirements of ISO9001, GB / T50107 and other standards for traceability of quality records.

[0022] (6) The maintenance cost of this invention is extremely low. The configurable design means that adding new report types or adjusting field coordinates only requires modifying the configuration dictionary, without changing the core logic; coordinate calibration can be completed with the help of visualization tools, and the technical threshold is low.

[0023] (7) The data output by this invention is in CSV format. This format is highly versatile and can be directly imported into Excel, databases, BI tools or Python / R for statistical analysis. It is also easy to integrate with enterprise information systems.

[0024] (8) The present invention has a wide range of applications and is applicable to all scenarios that require batch processing of building material inspection reports, such as concrete mixing plants, precast component plants, cement production enterprises, construction units, and third-party testing institutions. Attached Figure Description

[0025] This manual includes the following figures, which illustrate the following: Figure 1 This is a hardware structure block diagram of the system of the present invention.

[0026] Figure 2 This is a flowchart illustrating the method of the present invention.

[0027] The module includes: 1. Document scanning module; 2. Report type identification module; 3. Coordinate positioning and extraction module; 4. Data cleaning module; 5. Storage module; and 6. Anomaly handling module. Detailed Implementation

[0028] The specific embodiments of the present invention will be further described in detail below with reference to the accompanying drawings, in order to help those skilled in the art to have a more complete, accurate and in-depth understanding of the inventive concept and technical solution of the present invention, and to facilitate its implementation.

[0029] like Figure 1 As shown, this invention provides a concrete testing report parsing system that combines regularization rules and spatial coordinate regression. The system includes a document scanning module 1, a report type identification module 2, a coordinate positioning and extraction module 3, a data cleaning module 4, a storage module 5, and an anomaly handling module 6. These modules are connected via a data bus or control bus to collaboratively complete the entire processing flow from PDF file scanning to structured data output.

[0030] File scanning module 1 connects to external storage media. Its input is the user-specified root directory and all subfolders, and its output is a list of files to be processed, containing metadata such as the full file path, filename, folder path, and file size. This module uses the `rglob` method from the Python standard library `pathlib` to implement recursive traversal, supporting arbitrary depths of folder nesting and automatically discovering all ".pdf" or ".PDF" files without manual expansion. During traversal, file scanning module 1 calculates the scanning progress in real time, outputting progress information every 10 files processed for easy monitoring of long-running scanning tasks. For inaccessible files, such as those with insufficient permissions or those already in use, file scanning module 1 generates a permission exception signal and passes it to exception handling module 6, while simultaneously skipping that file and continuing the scan.

[0031] The report type identification module 2 is connected to the document scanning module 1. Its input is the PDF file path passed from the document scanning module 1, and its output is the report type identifier. This module has an internal configuration dictionary, using the report type name as the key and a keyword set and coordinate mapping table as values. The keyword set defines 5 to 10 characteristic keywords for each of the eight report types: concrete compressive strength test report, fly ash test report, sand and gravel aggregate test report, cement physical performance test report, admixture test report, mix design report, and ready-mixed concrete quality certificate, and supports regular expression matching. The identification algorithm uses a weighted matching mechanism: first, the pdfplumber library is used to extract the text content of the first page of the PDF; then, the extracted text is matched with the keyword sets for each type using regular expressions. The matching degree is determined by calculating the percentage of matched keywords relative to the total number of keywords in the keyword set. When the matching degree reaches a preset threshold, it is determined to be the report type; in this embodiment, the preset value is 80%. If the matching degree of multiple types exceeds the threshold, the type with the highest matching degree is selected; if the matching degree of all types is below the threshold, it is marked as unrecognized. After the identification is completed, the report type identification module 2 will pass the report type identifier to the coordinate positioning and extraction module 3.

[0032] The coordinate positioning and extraction module 3 is connected to the report type identification module 2. Its input is the report type identifier, and its output is a dictionary mapping from field names to the original text. For each standard PDF template of each report type, the coordinates of the quadruples of the rectangular area containing each key field are determined using the pdfplumber page coordinate system through visualization tools or programming. Specifically, after opening the PDF page, the page boundaries are obtained using `page.bbox`, the field positions are located using `page.extract_text`, and the field coordinate range is determined interactively or programmatically using the `page.within_bbox` method. The coordinate quadruples for each field have the top-left corner of the page as the origin, with the horizontal coordinate increasing to the right and the vertical coordinate increasing downwards, in points. After the coordinate configuration is completed, it is stored in the configuration dictionary according to the report type. The coordinate mapping table uses the field name as the key and the quadruples (x0, top, x1, bottom) as the value, where (x0, top) is the coordinate of the top-left corner of the rectangular area, and (x1, bottom) is the coordinate of the bottom-right corner of the rectangular area, with units consistent with the PDF coordinate system. For multi-page reports, the coordinate mapping table further stores the coordinate configurations of different pages according to page number indexes. The coordinate positioning and extraction module 3 uses the within_bbox method of the pdfplumber library for precise coordinate extraction: This method traverses all character objects on the page and checks whether the x0, top, x1, and bottom coordinates of each character fall completely within the target rectangular area. If they do, the character is collected, and finally, the characters are sorted by their x0 coordinates and concatenated into complete text.

[0033] The data cleaning module 4 is connected to the coordinate positioning and extraction module 3. This module automatically identifies the field type based on the field name and applies corresponding intelligent cleaning rules. The field type identification rules are as follows: if the field name contains keywords such as date, year-month-day, sampling date, and inspection date, it is identified as a date field; if the field name contains keywords such as intensity, rate, quantity, index, density, modulus, mud content, crushing index, water reduction rate, and bleeding rate ratio, it is identified as a numerical field; if the field name contains keywords such as inspection number, factory number, and report number, it is identified as a number field; otherwise, it is identified as a text field. After cleaning, the data cleaning module 4 transfers the cleaning results of all fields to the storage module 5.

[0034] Storage module 5 is connected to data cleaning module 4. This module first uses the `datetime.now().strftime` method to obtain the current timestamp, generating categorized filenames such as "Concrete Compressive Strength Test Report_20241225_143020.csv" and summary filenames such as "All Reports Summary_20241225_143020.csv". For each piece of structured data, storage module 5 outputs metadata fields including filename, report type, folder path, absolute file path, and extraction time, ensuring data traceability to the source file. Storage module 5 validates the data before writing, ensuring required fields are not empty, numeric fields are correctly formatted, and date fields conform to standard formats. If validation fails, the validation exception is passed to exception handling module 6.

[0035] The exception handling module 6 is connected to the file scanning module 1, report type identification module 2, coordinate location extraction module 3, data cleaning module 4, and storage module 5, respectively, to monitor the operating status of each module in real time, capture exception signals, and perform fault-tolerant processing. This module uses dual-channel logging, i.e., file and console logging, to capture exceptions that may occur in each module. After capturing an exception, the exception handling module performs the following operations: First, it writes the exception type, exception content, occurrence time, and involved file paths to the log file "pdf_processor.log" and simultaneously outputs it to the console, achieving dual-channel logging; then, it appends the complete path of the file to the failed file list "failed file_YYYYMMDD_HHMMSS.txt"; finally, it sends a hold signal to other modules, allowing them to continue processing the next file, ensuring the overall process is not interrupted. A statistical report is output after processing is complete.

[0036] like Figure 2 As shown, the present invention provides a method for parsing concrete test reports that combines regularization rules and spatial coordinate regression, comprising the following specific steps: S1: File scanning module 1 first performs an initialization operation to obtain the root directory path configured by the user. Then, this module uses the recursive traversal method pathlib.rglob, matching ".pdf" and ".PDF", to recursively traverse the root directory and all subfolders, automatically discovering all PDF files. During the traversal, the file scanning module records the complete path, filename, and folder path of each file, generating a list of files to be processed. For inaccessible files, the module generates an exception signal and passes it to exception handling module 6, while skipping that file and continuing the scan. After the scan is complete, file scanning module 1 transmits the list of files to be processed, along with the path and file data of each file, to report type recognition module 2.

[0037] Step S2: The report type identification module 2 sequentially identifies the type of each file in the list of files to be processed. Taking a concrete compressive strength test report as an example, this module first uses the pdfplumber library to extract the first page text content of the PDF file.

[0038] The report type identification module 2 has a pre-configured dictionary that defines a set of characteristic keywords for eight report types: concrete compressive strength test report, fly ash test report, sand and gravel aggregate test report, cement physical property test report, admixture test report, mix design report, and ready-mixed concrete quality certificate. Regular expressions are also supported. For example, the keyword set for a concrete compressive strength test report may include keywords such as "concrete compressive strength," "testing equipment," "age (d)," and "strength grade."

[0039] This module performs regular expression matching between the extracted homepage text and keyword sets of various types, calculating the matching degree. When the matching degree reaches a preset threshold of 80%, it is determined to be of that report type. Taking a concrete compressive strength test report as an example, its homepage text contains keywords such as "concrete compressive strength test report," "testing equipment: pressure testing machine," "age d: 28," and "strength grade: C30." The matching degree with the keyword set of "concrete compressive strength test report" reaches the threshold, so it is determined to be of that type. The report type identification module 2 transmits the identified report type identifier to the coordinate positioning and extraction module 3. If a file is marked as "unrecognized," the system records the file path to the skipped file list, which is recorded by the exception handling module 6, and then continues to process the next file.

[0040] Step S3: The coordinate positioning and extraction module 3 loads the corresponding field coordinate mapping table from the configuration dictionary based on the report type identifier. When adding a new report type, only the new key-value pairs need to be added to this dictionary; no modification to the main process code is required. The coordinate mapping table uses field names as keys and quadruples (x0, top, x1, bottom) as values, where (x0, top) is the coordinate of the top-left corner of the rectangular area, and (x1, bottom) is the coordinate of the bottom-right corner of the rectangular area, with units consistent with the PDF coordinate system. Taking a concrete compressive strength test report as an example, its coordinate mapping table can define the coordinate areas for fields such as test number, strength grade, compressive strength, and sampling date. For multi-page reports, the coordinate mapping table further stores the coordinate configurations of different pages according to page number indexes, and the system will automatically switch to the corresponding page for extraction based on the page number.

[0041] Taking a concrete compressive strength test report as an example, this module performs the following extraction operations: opens the PDF file to obtain the specified page, and sequentially calls the within_bbox method to extract text within each rectangular area, obtaining the original text of the test number, strength grade, compressive strength value, sampling date, etc. The coordinate positioning extraction module 3 then passes the extracted text data to the data cleaning module 4 in dictionary form, with field names as keys and original text as values.

[0042] S4: The data cleaning module 4 iterates through each field in the dictionary, automatically identifies the field type based on the semantics of the field name, and applies corresponding intelligent cleaning rules. Field type identification uses a keyword matching strategy, and the cleaning rules include cleaning date fields, numeric fields, number fields, and text fields.

[0043] Taking a concrete compressive strength test report as an example, the data cleaning module performs the following cleaning operations: For the test number field, the number cleaning rules are applied, using regular expressions to match all whitespace characters and replace them with empty strings to obtain a continuous number string; for the strength grade field, text cleaning rules are applied to remove leading and trailing whitespace and retain the original grade identifier; for the compressive strength field, numerical cleaning rules are applied, using regular expressions to extract the numbers and decimal parts, removing unit symbols to obtain a pure numerical value; for the sampling date field, date cleaning rules are applied to identify various date formats such as "YYYY / MM / DD" and "YYYY-MM-DD", and uniformly convert them to the standard "YYYY-MM-DD" format. After cleaning, the data cleaning module 4 transmits the cleaning results of all fields to the storage module 5 in dictionary form, with field names as keys and cleaned data as values.

[0044] S5: During the processing of each file, the exception handling module monitors the running status of each module in real time through a dual-channel structure. This module can capture various exception types and perform standardized fault-tolerant operations.

[0045] Taking a corrupted PDF file as an example, the coordinate positioning and extraction module throws a PDFSyntaxError exception when calling pdfplumber.open. This exception is immediately passed to the exception handling module 6 after being caught. The exception handling module 6 performs the following operations: First, it writes information such as the exception type, exception content, occurrence time, and involved file paths to the log file "pdf_processor.log" and simultaneously outputs it to the console, achieving dual-channel logging; then, it appends the complete path of the file to the list of failed files "failed files_YYYYMMDD_HHMMSS.txt"; finally, it sends a hold signal to other modules, instructing each module to continue processing the next file, ensuring the overall process is not interrupted.

[0046] The exception handling module 6 has three internal counters: a success counter, a failure counter, and a skip counter. The success counter increments by 1 for each successful processing operation; the failure counter increments by 1 for each exception caught; and the skip counter increments by 1 for each file skipped due to an unrecognized type. During batch processing, the exception handling module 6 records all failed files and outputs a statistical report upon completion.

[0047] Step S6: Storage module 5 associates and stores the cleaned structured data with file metadata. This module first obtains the current timestamp and generates a categorized output file "Concrete Compressive Strength Test Report_20241225_143020.csv" and a summary output file "All Reports Summary_20241225_143020.csv". For each piece of structured data, storage module 5 adds metadata fields such as filename, report type, folder path, absolute file path, and extraction time.

[0048] Taking a concrete compressive strength test report as an example, the CSV record output by storage module 5 contains all the business fields of the report, such as test number, strength grade, compressive strength, sampling date, and the aforementioned metadata fields. Storage module 5 writes to categorized CSV files and summary CSV files. Before writing, the data is validated to ensure that required fields are not empty, numeric fields are formatted correctly, and date fields conform to standard formats. If validation fails, the validation exception is passed to exception handling module 6.

[0049] After processing all files, the exception handling module 6 outputs a statistical report, displaying the total number of files, the number of successfully processed files, the number of failed files, and the number of skipped files, with statistical distribution by report type. A list of failed files and a detailed log file are provided for subsequent manual review and problem localization. When quality traceability is required, the original PDF file can be quickly located via the absolute file path field, meeting the traceability requirements of quality management standards.

[0050] This invention uses a modular system architecture to decouple functions such as document scanning, report type identification, coordinate location extraction, data cleaning, storage, and anomaly handling into independent modules. These modules work together through a standardized signal transmission mechanism, achieving a complete automated processing loop from PDF document scanning to structured data output.

[0051] This invention employs a recursive traversal mechanism to automatically discover all PDF files in a folder structure of any depth, eliminating the need for manual expansion of directories or selection of files one by one, thus significantly reducing manual operation time. For example, processing 1000 test reports in batches would take several days using traditional manual entry methods, while this invention's system can complete the entire process in 10 to 20 minutes, improving efficiency by hundreds of times and significantly liberating the productivity of quality inspection personnel.

[0052] In terms of accuracy, this invention leverages the standardized format and fixed field positions of concrete testing reports, employing coordinate mapping-driven precision field extraction technology. It uses the `within_bbox` method of the `pdfplumber` library to extract text within a specified rectangular area, effectively avoiding mismatch issues that easily occur in full-text regular expression matching. Combined with intelligent cleaning rules customized for the building materials industry, the accuracy of the output data is significantly superior to general OCR tools and pure regular expression matching schemes.

[0053] This invention supports seamless switching between more than eight report types, including concrete compressive strength testing, fly ash testing, sand and gravel aggregate testing, cement physical property testing, admixture testing, mix design, and ready-mixed concrete quality certificates. Through preset keyword sets and regular expression matching, the report type identification accuracy is greater than 95%, and all report types can be processed in a single run, achieving true "one-click extraction."

[0054] This invention employs the pathlib.rglob method to recursively traverse folders at arbitrary depths, automatically discovering all PDF files without omissions or blind spots. Through an exception handling mechanism, when a single file processing fails, only the failure information is recorded, and subsequent files are processed without interrupting the overall workflow. This ensures stability during long-term batch processing and is suitable for projects with thousands to tens of thousands of files.

[0055] This invention associates complete metadata such as filename, report type, folder path, absolute file path, and extraction time into the output CSV file, establishing a link between the extracted data and the original PDF file. In the event of quality issues, the original report can be quickly located via the file path, meeting the traceability requirements for quality records in standards such as ISO9001 and GB / T50107.

[0056] This invention uses a configuration dictionary to centrally manage the mapping relationship between identification keywords and field coordinates for report types. When adding a new report type or adjusting field coordinates, only the configuration dictionary needs to be modified without altering the core processing flow code. Coordinate calibration can be completed using the pdfplumber visualization tool, making it technically simple and easily accessible to general operations and maintenance personnel.

[0057] In summary, this invention, by combining regularization rules and spatial coordinate regression, effectively solves the technical problems of weak type recognition capability, poor field positioning accuracy, rigid data cleaning rules, insufficient batch processing capability, poor result traceability, high configuration and maintenance costs, and imperfect error handling in existing technologies. It achieves efficient, accurate, and automated batch parsing of concrete test reports, and is applicable to all scenarios requiring batch processing of building material test reports, such as concrete mixing plants, precast component factories, cement production enterprises, construction units, and third-party testing institutions. It has significant economic value and social benefits.

[0058] The present invention has been described above by way of example with reference to the accompanying drawings. Obviously, the specific implementation of the present invention is not limited to the above-described manner. Any non-substantial improvements made using the inventive concept and technical solution; or the direct application of the inventive concept and technical solution to other situations without modification, are all within the protection scope of the present invention.

Claims

1. A concrete testing report parsing system combining regularization rules and spatial coordinate regression, characterized in that, It includes a file scanning module (1), a report type identification module (2), a coordinate positioning and extraction module (3), a data cleaning module (4), a storage module (5), and an anomaly handling module (6). The file scanning module (1) obtains a PDF-formatted test report file from an external storage medium. The report type identification module (2) is connected to the file scanning module (1). The coordinate positioning and extraction module (3) is connected to the report type identification module (2). The data cleaning module (4) is connected to the coordinate positioning and extraction module (3). The storage module (5) is connected to the data cleaning module (4). The anomaly handling module (6) is connected to the file scanning module (1), the report type identification module (2), the coordinate positioning and extraction module (3), the data cleaning module (4), and the storage module (5), respectively.

2. The concrete test report parsing system combining regular rules and spatial coordinate regression according to claim 1, characterized in that, The file scanning module (1) transmits the path signal and file data signal of the detection report file to the report type identification module (2); the report type identification module (2) transmits the report type identification signal to the coordinate positioning extraction module (3); the coordinate positioning extraction module (3) transmits the field text signal to the data cleaning module (4); the data cleaning module (4) transmits the structured data signal to the storage module (5); the storage module (5) transmits the output data signal to the outside; when any module generates an abnormality, the module transmits the abnormality signal to the abnormality handling module (6) and transmits the hold signal to other modules.

3. A concrete test report parsing system combining regular rules and spatial coordinate regression according to claim 1 or 2, characterized in that, The report type identification module (2) determines the report type based on a preset keyword set. The keyword set is configured with multiple feature keywords for each report type and supports regular expressions. The report type identification module (2) calculates the matching degree between the extracted text and the keyword set of each type. When the matching degree reaches a preset threshold, the report type is determined. If there is no type match, it is marked as unrecognized and the processing is skipped.

4. A concrete test report parsing system combining regular rules and spatial coordinate regression according to claim 1 or 2, characterized in that, The coordinate positioning and extraction module (3) is configured with a field coordinate mapping table. The field coordinate mapping table uses the field name as the key and a quadruple containing the start point of the horizontal coordinate, the start point of the vertical coordinate, the end point of the horizontal coordinate, and the end point of the vertical coordinate as the value. For a detection report containing multiple pages, the coordinate mapping table further stores the coordinate configuration of different pages according to the page number index. The coordinate positioning and extraction module (3) calls the corresponding coordinate configuration according to the page number index to perform cross-page field extraction.

5. A concrete inspection report parsing system combining regular rules and spatial coordinate regression according to claim 1 or 2, characterized in that, The data cleaning module (4) is configured with intelligent cleaning rules, including: date field cleaning rules, numeric field cleaning rules, number field cleaning rules, and text field cleaning rules; the date field cleaning rules identify multiple date formats through regular expressions and convert them into preset standard date formats; the numeric field cleaning rules extract the numbers and decimal points through regular expressions and remove unit symbols and non-numeric characters; the number field cleaning rules maintain the continuity of the number string by removing spaces and newlines; the text field cleaning rules retain the original text content by removing leading and trailing whitespace characters.

6. A concrete inspection report parsing system combining regularization rules and spatial coordinate regression according to claim 1 or 2, characterized in that, The structured data output by the storage module (5) includes: independent files stored according to report type, and summary files that merge all types of data; the file metadata includes at least the file name, report type, folder path where the file is located, absolute file path, and extraction time.

7. A concrete inspection report parsing system combining regular rules and spatial coordinate regression according to claim 1 or 2, characterized in that, The file scanning module (1) uses the recursive traversal method of the path library to automatically discover all files with specified extensions; during the processing, it calculates and outputs the processing progress in real time; after the processing is completed, it counts and outputs the total number of files, the number of successfully processed files, the number of failed files and the number of skipped files, and performs statistical distribution according to report type.

8. A concrete inspection report parsing system combining regular rules and spatial coordinate regression according to claim 1 or 2, characterized in that, The keyword set in the report type identification module (2) and the field coordinate mapping table in the coordinate positioning extraction module (3) are centrally defined and managed through a configuration dictionary; by modifying the configuration dictionary, report types and their corresponding identification keywords and field coordinate mapping relationships can be added, deleted or adjusted.

9. A method for parsing concrete test reports based on any one of claims 1-8, combining regular rules and spatial coordinate regression, characterized in that, Includes the following steps: S1: The file scanning module (1) recursively scans the specified root directory and all subfolders, automatically discovers and obtains all PDF format detection report files, and generates a list of files to be processed; S2: Report type identification module (2) For each file to be processed, extract the homepage text, match it based on a preset set of keywords containing regular expressions, and automatically determine the type of the current report; S3: Coordinate positioning and extraction module (3) According to the determined report type, call the pre-configured field coordinate mapping table. The mapping table defines the rectangular area coordinates of different fields in the PDF page. Using the coordinate system of the PDF parsing library, the text content is accurately extracted within the rectangular area corresponding to each field. S4: Data cleaning module (4) applies preset intelligent cleaning rules to format the extracted text according to the type of the extracted fields, and generates structured data; S5: Exception handling module (6) captures the exception and continues to process subsequent files without interrupting the overall process when an exception occurs in any of the aforementioned steps during the processing of each file. S6: The storage module (5) associates the structured data with the corresponding file metadata and stores it, and outputs it as a structured file.

10. The method for parsing concrete test reports combining regularization rules and spatial coordinate regression according to claim 9, characterized in that, Step S5 further includes: recording the exception information to the failure file list, and simultaneously outputting the log information of the processing process to the log file and the console; after the processing is completed, counting and outputting the total number of files, the number of successful processing, the number of failures, and the number of skips.

Images