Data extraction method, electronic equipment and readable storage medium

Abstract

The invention discloses a data extraction method for a detection report, which comprises the following steps of: S1, reading a PDF (Portable Document Format) file, and converting the PDF file into a picture; s2, acquiring an identification code on the picture, analyzing a report type of the PDF file according to the identification code, and matching a corresponding data extraction rule based on the report type; s3, according to the data extraction rule, the text on the picture is converted into a machine-coded text, and corresponding text information is generated; and S4, verifying the text information, and outputting the structured data. The data extraction method is designed for the detection report of the constructional engineering, the report type of the PDF file can be analyzed according to the identification code on the picture converted from the PDF file, the data of the PDF file is extracted according to the data extraction rule set corresponding to the report type, the data processing efficiency and accuracy can be improved, and the data extraction efficiency is improved. And moreover, manual intervention is reduced, and the labor cost of data management of the detection report of the constructional engineering is reduced.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a data extraction method, an electronic device, and a readable storage medium. Background Art

[0002] Data management is one of the core links of construction projects. Different inspection units and different inspection projects will issue different inspection reports. With the development of information technology, these inspection reports are issued in the form of electronic reports of PDF files. The inspection reports record a large amount of key engineering data, which is crucial for engineering quality control and subsequent analysis. Since the existing PDF text extraction tools cannot accurately identify and extract table data or data in a specific format, the data processing of the inspection reports of construction projects now relies on manual reading and data entry, which makes the existing inspection report data processing method inefficient and error-prone. Summary of the invention

[0003] The technical problem to be solved by the present invention is to provide a method for extracting data from a test report to improve the data processing efficiency and accuracy of the test report of a construction project.

[0004] In order to solve the above technical problems, the technical solution adopted by the present invention is: a method for extracting data from a test report, comprising the following steps:

[0005] S1: Read a PDF file and convert the PDF file into a picture;

[0006] S2: Obtaining an identification code on the image, parsing a report type of the PDF file according to the identification code, and matching a corresponding data extraction rule based on the report type;

[0007] S3: According to the data extraction rule, convert the text on the image into machine-coded text to generate corresponding text information;

[0008] S4: Verify the text information and output structured data.

[0009] Furthermore, the step S2 also includes: performing location detection on the image according to the identification code.

[0010] Furthermore, the step S2 also includes: acquiring report information of the PDF file according to the identification code.

[0011] Furthermore, in the step S4, the text information is compared and verified, including: comparing the report information with the text information to determine whether the two are consistent; if they are consistent, outputting and storing structured data according to the text information; if they are inconsistent, the PDF file is transferred to manual processing.

[0012] Furthermore, the report information includes at least one of a testing unit, a report number, a report type, a report date and a report conclusion.

[0013] Furthermore, before step S1, the step of setting data extraction rules according to the report type of the PDF file is also included.

[0014] Furthermore, setting data extraction rules according to the report type of the PDF file specifically includes: obtaining fixed positions of each information to be extracted in the report template corresponding to the report type of the PDF file, and setting data extraction rules based on the fixed positions of the information to be extracted.

[0015] Furthermore, in the step S4, outputting the structured data includes: storing the structured data in a database.

[0016] Another object of the present invention is to provide an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned data extraction method when executing the program.

[0017] Another object of the present invention is to provide a computer-readable storage medium storing computer program instructions, which implement the above-mentioned data extraction method when executed by a processor.

[0018] The beneficial effects of the present invention are as follows: The beneficial effects of the present invention are as follows: The present invention designs a data extraction method for the inspection report of a construction project, which can parse the report type of the PDF file according to the identification code on the image converted from the PDF file, and then extract the data of the PDF file according to the data extraction rules set for the corresponding report type, and output structured data. The data extraction method provided by the present invention can improve the efficiency and accuracy of data processing, reduce manual intervention, and reduce the labor cost of data management of the inspection report of the construction project. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 A flowchart of a method for extracting data from a test report provided by the present invention in one embodiment;

[0020] Figure 2 The present invention provides a schematic structural diagram of an electronic device. DETAILED DESCRIPTION

[0021] In order to explain the technical content, achieved objectives and effects of the present invention in detail, the following is an explanation in combination with the implementation modes and the accompanying drawings.

[0022] Please refer to Figure 1, a method for extracting data from a test report, comprising the following steps:

[0023] S1: Read a PDF file and convert the PDF file into a picture;

[0024] S2: Obtaining an identification code on the image, parsing a report type of the PDF file according to the identification code, and matching a corresponding data extraction rule based on the report type;

[0025] S3: According to the data extraction rule, convert the text on the image into machine-coded text to generate corresponding text information;

[0026] S4: Verify the text information and output structured data.

[0027] From the above description, it can be seen that the beneficial effect of the present invention is that the present invention designs a data extraction method for the inspection report of the construction project, which can parse the report type of the PDF file according to the identification code on the picture converted from the PDF file, and then extract the data of the PDF file according to the data extraction rules set for the corresponding report type, and output structured data. The data extraction method provided by the present invention can improve the efficiency and accuracy of data processing, reduce manual intervention, and reduce the labor cost of data management of the inspection report of the construction project.

[0028] Furthermore, the step S2 also includes: performing location detection on the image according to the identification code.

[0029] From the above description, it can be seen that the data extraction method provided by the present invention sets data extraction rules according to the fixed position of the information to be extracted in the report template corresponding to the report type, and extracts text from the image of the area corresponding to the picture according to the data extraction rules; therefore, the reliability and accuracy of the data extraction rules can be guaranteed by positioning, detecting and correcting the image through the identification code.

[0030] Furthermore, the step S2 also includes: acquiring report information of the PDF file according to the identification code.

[0031] Furthermore, in the step S4, the text information is compared and verified, including: comparing the report information with the text information to determine whether the two are consistent; if they are consistent, outputting and storing structured data according to the text information; if they are inconsistent, the PDF file is transferred to manual processing.

[0032] From the above description, it can be seen that the identification code can not only locate and detect the images converted from the PDF file, but also verify the report content of the PDF file to ensure the consistency between the report type of the detection report in the PDF file and the data extraction rules, thereby ensuring the correctness of the extracted data.

[0033] Furthermore, the report information includes at least one of a testing unit, a report number, a report type, a report date and a report conclusion.

[0034] Furthermore, before step S1, the step of setting data extraction rules according to the report type of the PDF file is also included.

[0035] Furthermore, setting data extraction rules according to the report type of the PDF file specifically includes: obtaining fixed positions of each information to be extracted in the report template corresponding to the report type of the PDF file, and setting data extraction rules based on the fixed positions of the information to be extracted.

[0036] From the above description, it can be seen that the data extraction method provided by the present invention sets data extraction rules according to the fixed position of the information to be extracted in the report template corresponding to the report type, and extracts text from the image of the corresponding area in the picture according to the data extraction rules.

[0037] Furthermore, in the step S4, outputting the structured data includes: storing the structured data in a database.

[0038] From the above description, it can be seen that storing structured data in a database can facilitate subsequent data management and analysis.

[0039] Another object of the present invention is to provide an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned data extraction method when executing the program.

[0040] It can be seen from the above description that the electronic device provided by the present invention has at least all the beneficial effects of the above data extraction method.

[0041] Another object of the present invention is to provide a computer-readable storage medium storing computer program instructions, which implement the above-mentioned data extraction method when executed by a processor.

[0042] It can be seen from the above description that the computer-readable storage medium provided by the present invention has at least all the beneficial effects of the above data extraction method.

[0043] Embodiment 1

[0044] Please refer to Figure 1, Embodiment 1 of the present invention is:

[0045] In construction projects, different inspection units and different inspection projects will issue different inspection reports. Existing PDF text extraction tools cannot accurately identify and extract table data or data in specific formats of PDF files. Therefore, the data processing of existing construction project inspection reports relies on manual reading and data entry.

[0046] In order to solve the problem that the above-mentioned existing PDF text extraction tools cannot accurately identify and extract PDF files, this embodiment provides a data extraction method to meet the data extraction requirements of PDF files of various report types. The data extraction method includes the following steps:

[0047] S1: Read a PDF file and convert the PDF file into a picture;

[0048] S2: Obtaining an identification code on the image, parsing a report type of the PDF file according to the identification code, and matching a corresponding data extraction rule based on the report type;

[0049] S3: According to the data extraction rule, convert the text on the image into machine-coded text to generate corresponding text information;

[0050] S4: Verify the text information and output structured data.

[0051] The inspection report of the construction project contains an identification code, which can be a QR code or a barcode. The identification code carries the basic information of the inspection report (for example, inspection unit, report number, report type, report date, etc.). After the PDF file is converted into an image, the identification code on the image can be obtained. According to the identification code, the report type of the inspection report of the construction project in the PDF file can be obtained, so as to obtain the corresponding data extraction rules to extract the text information on the image.

[0052] In this embodiment, the step S1 further includes the step of setting data extraction rules according to the report type of the PDF file. Specifically, the data extraction rules can be set according to the following steps, obtaining the fixed positions of each information to be extracted in the report template corresponding to the report type of the PDF file, and setting the data extraction rules based on the fixed positions of the information to be extracted.

[0053] According to the report templates corresponding to various report types provided by the inspection unit, the fixed position of each engineering data in the PDF file can be analyzed and determined. This embodiment sets the data extraction rules based on this feature to ensure the reliability and accuracy of the data extraction method.

[0054] Correspondingly, when converting the text on the image into machine-coded text according to the data extraction rules, the image is divided into multiple areas according to the data extraction rules, and OCR conversion is performed on each area to convert the text of the image in each area into machine-coded text.

[0055] Since the data extraction rules in this embodiment are set based on the fixed positions of each engineering data in the PDF file, the positioning of the picture when extracting text information based on the picture has a great impact on the reliability of data extraction. In this embodiment, step S2 also includes: positioning the picture according to the identification code. In this embodiment, the picture is positioned by the identification code, and the identification code is used as a positioning point to ensure that the position relationship and size ratio of each area range in the page of the picture match the data extraction rules.

[0056] In this embodiment, the image converted from the PDF file can be quickly located through the identification code, thereby improving the data extraction efficiency and accuracy.

[0057] In this embodiment, the step S2 further includes: obtaining report information of the PDF file according to the identification code;

[0058] In the step S4, the text information is verified, including: comparing the report information with the text information to determine whether the two are consistent; if they are consistent, outputting and storing structured data according to the text information; if they are inconsistent, the PDF file is transferred to manual processing.

[0059] The above-mentioned report information includes at least one of the testing unit, report number, report type, report date and report conclusion.

[0060] When extracting data, the report information obtained according to the identification code can be compared with the corresponding text information extracted according to the data extraction rule to further verify that the PDF file corresponds to the selected data extraction rule. The data extraction method provided in this embodiment can verify the PDF file and the data extraction rule to ensure the consistency of the report type of the test report in the data extraction rule and the PDF file. Compared with manual data entry, the data extraction method provided in this embodiment can reduce the error probability and improve the accuracy of data processing.

[0061] In this embodiment, after the PDF file is transferred to manual processing, it also includes: sending a data extraction rule error warning to notify the operation and maintenance personnel to maintain and change the data extraction rules to adapt to new report templates or new data extraction requirements.

[0062] In step S4 of this embodiment, outputting structured data includes: storing the structured data in a database to facilitate subsequent data management and analysis operations.

[0063] The data extraction rules of the data extraction method provided in this embodiment may include data verification rules set according to industry standards and legal rules to calculate and verify structured data to ensure that the engineering data in the engineering inspection report meets the specified quality and safety requirements.

[0064] During the implementation of the data extraction method provided in this embodiment, when a data extraction rule is applied for the first time, in order to ensure the reliability and correctness of the data extraction rule, it is necessary to manually review the extracted engineering data.

[0065] In addition, when the data extraction method is applied to extract non-standardized engineering inspection reports, the report type of the PDF file of the non-standardized engineering inspection report parsed according to the identification code cannot match the corresponding data extraction rules; in response to the above situation, step S2 of this embodiment also includes: when the report type cannot match the corresponding data extraction rules, the PDF file is converted to manual processing.

[0066] Since the report templates of engineering inspection reports of various report types may change over time, the present embodiment further includes the step of regularly maintaining and updating data extraction rules to ensure that the data extraction rules adapt to new report templates.

[0067] In this embodiment, an electronic device is also provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned data extraction method when executing the program.

[0068] Among them, the computer program execution program will record the process of executing the above-mentioned data extraction method through the system log, ensuring the transparency and traceability of the computer program's execution of the above-mentioned data extraction method, and further ensuring the reliability and correctness of the data extraction method.

[0069] In summary, the present invention designs a data extraction method for the inspection report of a construction project, which can parse the report type of a PDF file according to the identification code carried by the PDF file, and then extract the data of the PDF file according to the data extraction rules set for the corresponding report type to output structured data. The data extraction method provided by the present invention can improve the efficiency and accuracy of data processing, reduce manual intervention, and reduce the labor cost of data management of the inspection report of the construction project.

[0070] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent transformations made using the contents of the present invention's specification and drawings, or directly or indirectly applied in related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for extracting data from a test report, characterized in that: The following steps are involved: S1: Read a PDF file and convert the PDF file into a picture; S2: Obtaining an identification code on the image, parsing a report type of the PDF file according to the identification code, and matching a corresponding data extraction rule based on the report type; S3: According to the data extraction rule, convert the text on the image into machine-coded text to generate corresponding text information; S4: Verify the text information and output structured data.

2. The data extraction method according to claim 1, characterized in that: The step S2 also includes: performing location detection on the image according to the identification code.

3. The data extraction method according to claim 1, characterized in that: The step S2 further includes: acquiring report information of the PDF file according to the identification code.

4. The data extraction method according to claim 3, characterized in that: In the step S4, the text information is verified, including: comparing the report information with the text information to determine whether the two are consistent; if they are consistent, outputting and storing structured data according to the text information; if they are inconsistent, the PDF file is transferred to manual processing.

5. The data extraction method according to claim 3, characterized in that: The report information includes at least one of a testing unit, a report number, a report type, a report date and a report conclusion.

6. The data extraction method according to claim 1, characterized in that: The step before step S1 also includes the step of setting data extraction rules according to the report type of the PDF file.

7. The data extraction method according to claim 6, characterized in that: Setting data extraction rules according to the report type of the PDF file specifically includes: obtaining fixed positions of each piece of information to be extracted in a report template corresponding to the report type of the PDF file, and setting data extraction rules based on the fixed positions of the information to be extracted.

8. The data extraction method according to claim 4, characterized in that: In the step S4, outputting the structured data includes: storing the structured data in a database.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the data extraction method according to any one of claims 1 to 8 is implemented.

10. A computer-readable storage medium storing computer program instructions, characterized in that: The computer program instructions implement the data extraction method according to any one of claims 1 to 8 when executed by a processor.

Images