Table extraction method for file without table line, storage medium and electronic equipment

By removing watermarks, converting them into Word files, replacing them with text boxes and grouping them, the problems of cell misalignment and content recognition errors in traditional methods are solved, and the accurate extraction of table data without table lines and tables is achieved. Integration ensures the accuracy and completeness of the data.

CN119067084APending Publication Date: 2024-12-03ZHANJIANG POWER SUPPLY BUREAU OF GUANGDONG POWER GRID CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411044263.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-31
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

Traditional format conversion methods have cell misalignment and content recognition errors in table data extraction without table lines, which affects the accuracy and usability of the data.

Method used

By removing the watermarking of the power grid project budget quota PDF file, converting it into a Word file, replacing the table contents into text boxes, analyzing the text box position information, and using the threshold method to group and sort, finally filling the text box content into Excel cells.

Benefits of technology

It solves the problems of cell misalignment and content identification errors, ensures the accuracy and completeness of the data, and provides reliable guarantees for the use of power grid project budget quota table data.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119067084A_ABST
    Figure CN119067084A_ABST
Patent Text Reader

Abstract

The invention provides a table extraction method for a table-line-free file, a storage medium and electronic equipment, and the method comprises the steps: carrying out the watermark removal processing of a power grid project budget quota PDF file, obtaining a first processing file, retrieving a page containing a table as a target page, summarizing the target page, and converting the target page into a Word file, obtaining a second processing file; replacing contents of all tables in the second processing file with a textbox containing the same text to obtain a third processing file; analyzing the third processing file to obtain position information of each textbox in the third processing file, and grouping and sorting according to the position information of each textbox by adopting a threshold method to obtain a plurality of textbox groups; and filling the textboxes into cells of the Excel according to a group sequence to generate the Excel file. According to the method, the problem that the accuracy and practicability of data are seriously influenced by cell dislocation and content identification errors caused by a traditional format conversion method is solved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of electrical engineering, and in particular to a table extraction method for a table-free line file, a computer-readable storage medium, and an electronic device. Background Art

[0002] With the rapid development of power grid projects, budget quota files contain a large amount of tabular data, which is crucial for project management and cost control. However, since many budget quota tables do not have table lines, traditional format conversion methods often lead to cell misalignment and content recognition errors, seriously affecting the accuracy and availability of data. These problems not only increase the workload of manual proofreading, but may also lead to deviations in project budget and management decisions.

[0003] To address the above problems, the existing solution is to use image processing methods to solve them. Image processing methods extract row and column information of tables through edge detection and connected domain analysis techniques. They are suitable for tables without table lines, but are not effective for files with complex backgrounds and high noise levels. Summary of the invention

[0004] The main purpose of the present application is to provide a table extraction method for a table line-free file, a computer-readable storage medium and an electronic device, so as to at least solve the problem that traditional format conversion methods often lead to cell misalignment and content recognition errors, seriously affecting the accuracy and practicality of the data.

[0005] In order to achieve the above-mentioned purpose, according to one aspect of the present application, a method for extracting tables from table-free line files is provided, comprising: removing watermarks from a power grid engineering budget quota PDF file to obtain a first processed file; retrieving pages containing tables in the first processed file as target pages, and aggregating and converting all the target pages into Word files to obtain a second processed file; replacing the contents of all tables in the second processed file with text boxes containing the same text to obtain a third processed file; parsing the third processed file to at least obtain position information of each text box in the third processed file, and grouping and sorting each text box using a threshold method according to the position information of each text box to obtain multiple text box groups; filling each text box into Excel cells in group order to generate an Excel file.

[0006] Optionally, the power grid engineering budget quota PDF file is processed to remove the watermark to obtain a first processed file, including: using PyMuPDF in Python to convert the pages in the power grid engineering budget quota PDF file into PNG format to obtain an image file; using OpenCV and NumPy to perform Fourier transform to convert the image file to the frequency domain to obtain an initial frequency domain image; identifying and filtering out specific frequency components corresponding to the watermark in the initial frequency domain image in the frequency domain to obtain a processed frequency domain image; performing an inverse Fourier transform on the processed frequency domain image to obtain a watermark-removed image, and saving the watermark-removed image as a PDF file to obtain the first processed file.

[0007] Optionally, the page containing the table in the first processing file is retrieved as the target page, and all the target pages are summarized and converted into Word files to obtain the second processing file, including: using the enumerate of the pdfplumber function library to traverse each page in the first processing file, and obtaining the page number and the page object at the same time; using the ET method to call the page.extract_tables() function to extract the table from the current page, and store the table in a tables list, and the tables list corresponds to the page one by one; when the tables list exists and the length is greater than 0, it is determined that the page corresponding to the tables list contains the table, and the page containing the table is determined as the target page; all the target pages are summarized and the first processing file is converted into a Word file using a format converter as the second processing file.

[0008] Optionally, the contents of all tables in the second processing file are replaced with text boxes containing the same text to obtain a third processing file, including: retrieving the contents of the cells in the second processing file; when there are no table lines in the cell contents, using line breaks to segment the cell contents to obtain segmented text, and using each segmented text as text box content; creating a text box shape and setting a text box style to obtain a text box, wherein the text box style includes the position and size information of the text box; adding the text box to the cell to obtain a Word file with mixed text and images as the third processing file.

[0009] Optionally, a text box shape is created, and a text box style is set to obtain a text box, including: creating a structured document tag element and an sdtContent element, and adding the sdtContent element to the structured document tag element to obtain a target sdt element, wherein the structured document tag element is used to represent a text box; the sdtContent element serves as a content container for the text box; creating a paragraph element, a paragraph attribute element, a text run element, and a text element, wherein the paragraph element is used to represent a paragraph of the text box, the paragraph attribute element is used to define the attributes of the paragraph, the text run element is used to represent a piece of text in the paragraph, the text element is used to set the text content to the passed-in text parameter, the paragraph attribute element is added to the paragraph element to obtain a paragraph processing element, and the text element is added to In the text running element, a first processing element is obtained, and the first processing element is added to the paragraph processing element to obtain a second processing element, and finally the second processing element is added to the sdtContent element to obtain a target sdtContent element; an sdtPr element, a w:sz element and a w:rPr element are created, the sdtPr element is used to set the properties of the text box, the w:sz element is used to set the font size of the text in the text box, and the w:rPr element is used to characterize the running properties of the text in the text box; according to the target sdt element, the target sdtContent element, the sdtPr element, the w:sz element and the w:rPr element, a target text box is obtained, and the target text box is added to a cell, and the target sdt element is added to the XML structure of the cell.

[0010] Optionally, the third processing file is parsed to obtain at least the position information of each text box in the third processing file, including: using an Xpath function to query the content in the text box, and using an etree function library to parse the XML content in the third processing file; taking the upper left corner of the text box as a base point, querying the XML node <wp:positionh>and <wp:positionv>, obtain the initial horizontal information and initial vertical information of the text box; convert the units of the initial horizontal information and the initial vertical information to obtain the horizontal position information and vertical position information of the text box in centimeters.

[0011] Optionally, the text boxes are grouped and sorted using a threshold method according to the position information of each text box to obtain a plurality of text box groups, including: determining a first target difference, the first target difference being the absolute value of the difference between the horizontal coordinate of the ith text box and the horizontal coordinate of the jth text box, the ith text box being the currently retrieved text box, and the jth text box being the text box that has been retrieved and grouped; when the first target difference is less than or equal to the threshold, the ith text box is added to the group to which the jth text box belongs, and when the first target difference is greater than the threshold, a new group is established based on the ith text box to obtain a plurality of row groups; the smallest row coordinate in each row group is added to the group to which the jth text box belongs. Using the second target difference as the representative coordinate of the group to perform bubble sort between the row groups; determining a second target difference, the second target difference is the absolute value of the difference between the ordinate of the ith text box and the ordinate of the jth text box, the ith text box is the text box currently being retrieved, and the jth text box is the text box that has been retrieved and grouped; when the second target difference is less than or equal to a threshold, adding the ith text box to the group to which the jth text box belongs, and when the second target difference is greater than the threshold, establishing a new group based on the ith text box to obtain multiple column groups; using the smallest ordinate in each column group as the representative coordinate of the group to perform bubble sort between the column groups.

[0012] Optionally, each of the text boxes is filled into an Excel cell in group order to generate an Excel file, including: obtaining a row group number and a column group number of each of the text boxes, wherein the row group number of the text box corresponds to the column number in the Excel file, and the column group number of the text box corresponds to the row number in the Excel file; and filling each of the text boxes into an Excel cell in group order according to the row group number and column group number of each of the text boxes to generate the Excel file.

[0013] According to another aspect of the present application, a table extraction device for a table-free line file is provided, including: a first processing unit, used to remove watermarks from a power grid engineering budget quota PDF file to obtain a first processed file; a second processing unit, used to retrieve pages containing tables in the first processed file as target pages, and aggregate and convert all of the target pages into Word files to obtain a second processed file; a third processing unit, used to replace the contents of all tables in the second processed file with text boxes containing the same text to obtain a third processed file; a fourth processing unit, used to parse the third processed file to at least obtain position information of each text box in the third processed file, and group and sort each text box by a threshold method according to the position information of each text box to obtain a plurality of text box groups; a fifth processing unit, used to fill each text box into an Excel cell in group order to generate an Excel file.

[0014] According to another aspect of the present application, a computer-readable storage medium is provided, the computer-readable storage medium comprising a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute any one of the table extraction methods for table-free line files.

[0015] According to another aspect of the present application, an electronic device is provided, comprising: one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include a method for executing any one of the above-mentioned table extraction methods for table-free line files.

[0016] Applying the technical solution of the present application, the table extraction method of the above-mentioned table-free line file first removes the watermark processing of the power grid engineering budget quota PDF file to obtain the first processing file; then retrieves the page containing the table in the first processing file as the target page, and summarizes and converts all the target pages into Word files to obtain the second processing file; then replaces the contents of all tables in the second processing file with text boxes containing the same text to obtain the third processing file; parses the third processing file to at least obtain the position information of each text box in the third processing file, and uses the threshold method to group and sort each text box according to the position information of each text box to obtain multiple text box groups; finally, fills each text box into the Excel cell in the order of the group to generate an Excel file. This method can solve the problem that the traditional format conversion method often leads to cell dislocation and content recognition errors, which seriously affects the accuracy and practicality of the data, realizes the effective extraction of the table without table line in the power grid engineering budget quota PDF file, overcomes the difficulties such as cell dislocation and content recognition errors when directly performing format conversion, ensures the accuracy and integrity of the data, and provides a reliable guarantee for the subsequent use of the power grid engineering budget quota table data. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings constituting part of the present application are used to provide a further understanding of the present application. The exemplary embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0018] Figure 1 A hardware structure block diagram of a mobile terminal for executing a table extraction method for a table-free line file provided in an embodiment of the present application is shown;

[0019] Figure 2 A schematic flow chart of a table extraction method for a table-free line file provided according to an embodiment of the present application is shown;

[0020] Figure 3 A schematic diagram of a table without table lines of a table extraction method for a table without table line file provided according to an embodiment of the present application is shown;

[0021] Figure 4 A schematic diagram of a Word document showing a table extraction method for a table-free line file provided according to an embodiment of the present application;

[0022] Figure 5 A schematic diagram of a flow chart of a method for sorting text boxes provided according to an embodiment of the present application is shown;

[0023] Figure 6 A table-free line table conversion result diagram of a table extraction method for a table-free line file provided according to an embodiment of the present application is shown;

[0024] Figure 7 A schematic flow chart of another method for extracting a table from a table-free line file provided according to an embodiment of the present application is shown;

[0025] Figure 8 A structural block diagram of a table extraction device for a table-free line file provided according to an embodiment of the present application is shown.

[0026] The above drawings include the following reference numerals:

[0027] 102, processor; 104, memory; 106, transmission device; 108, input and output devices. DETAILED DESCRIPTION

[0028] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0029] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0030] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application described here. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0031] As introduced in the background technology, with the rapid development of power grid projects in the prior art, a large amount of tabular data is contained in the budget quota files, which are crucial for project management and cost control. However, since many budget quota tables do not have table lines, traditional format conversion methods often lead to cell misalignment and content recognition errors, seriously affecting the accuracy and availability of data. These problems not only increase the workload of manual proofreading, but may also lead to deviations in engineering budgets and management decisions. In response to the above problems, the existing solution is to use image processing methods to solve them. The image processing method extracts the row and column information of the table through edge detection and connected domain analysis and other technologies. It is suitable for tables without table lines, but it is not effective for files with complex backgrounds and more noise. The embodiments of the present application provide a table extraction method for files without table lines, a computer-readable storage medium, and an electronic device.

[0032] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0033] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Taking running on a mobile terminal as an example, Figure 1 FIG. 1 is a hardware structure block diagram of a mobile terminal of a table extraction method for a table-free line file according to an embodiment of the present invention. Figure 1 As shown, the mobile terminal may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, wherein the mobile terminal may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the mobile terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.

[0034] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the table extraction method without table line files in the embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the mobile terminal via a network. Examples of the above network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof. The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the mobile terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0035] In this embodiment, a table extraction method for a table-free line file running on a mobile terminal, a computer terminal or a similar computing device is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0036] Figure 2 1 is a flow chart of a table extraction method for a table-free line file according to an embodiment of the present application. Figure 2 As shown, the method comprises the following steps:

[0037] Step S201, removing the watermark from the power grid project budget quota PDF file to obtain a first processed file;

[0038] Specifically, Fourier transform is used to identify and filter out specific frequency components of the watermark to achieve accurate watermark removal and ensure that the image quality in the PDF file is not damaged.

[0039] The process of removing the watermark from the power grid project budget quota PDF file to obtain the first processed file includes the following steps:

[0040] Step S301, using PyMuPDF in Python to convert the pages in the power grid project budget quota PDF file into PNG format to obtain an image file;

[0041] Step S302, using OpenCV and NumPy to perform Fourier transform, converting the above image file into the frequency domain to obtain an initial frequency domain image;

[0042] Step S303, identifying and filtering out the specific frequency components corresponding to the watermark in the initial frequency domain image in the frequency domain, to obtain a processed frequency domain image;

[0043] Step S304, performing inverse Fourier transform on the processed frequency domain image to obtain a watermark-removed image, and saving the watermark-removed image as a PDF file to obtain the first processed file.

[0044] Specifically, Fourier transform is used to identify and filter out specific frequency components of the watermark to achieve accurate watermark removal and ensure that the image quality in the PDF file is not damaged. Python scripts and multiple function libraries (PyMuPDF, OpenCV, NumPy, pdfplumber, etc.) are used to automatically process PDF files, reducing manual intervention and improving efficiency.

[0045] In some cases, the collected data on the entire process of production project cost includes investment estimation documents, bidding documents, contract documents, construction process data, completion data, and settlement data. The file formats are mainly Word, Excel, and PDF, and contain a large amount of semi-structured and unstructured power grid production cost data.

[0046] The Fourier transform and inverse Fourier transform use (2D IDFT) and (2D IDFT).

[0047] The formula for (2D DFT) is as follows:

[0048]

[0049] The formula for (2D IDFT) is as follows:

[0050]

[0051] In the formula, f(x,y) is the value of the original image at the coordinate (x,y), and F(u,v) is the value of the frequency domain image at the frequency domain (u,v). M and N are the height and width of the image, respectively. x and y are used to indicate the specific pixel position in the image. x and y are the coordinates in the image space domain. The value range of x is 0 to M-1, and the value range of y is 0 to N-1. u and v are the coordinates in the frequency domain. u and v are used to indicate the frequency index in the horizontal and vertical directions of the frequency component. The value range of u and v is the same as that of x and y.

[0052] Among them, identifying and filtering out specific frequency components corresponding to the watermark is to determine the specific frequency components of the watermark in the frequency domain through the periodic structure of the watermark, and then select a suitable mask to filter out these specific frequency components.

[0053] Step S202, retrieving the pages containing the table in the first processing file as target pages, and aggregating and converting all the target pages into Word files to obtain a second processing file;

[0054] Specifically, by using the pdfplumber function library enumerate and calling the page.extract_tables() function, the page number and page number object are automatically obtained and correspond one-to-one with the tables list, which improves efficiency and ensures accurate data extraction of tables without table lines.

[0055] The process of retrieving the page containing the table in the first processing file as the target page, and aggregating and converting all the target pages into Word files to obtain the second processing file includes the following steps:

[0056] Step S401, using enumerate of the pdfplumber function library to traverse each page in the first processing file, and obtaining the page number and page object;

[0057] Step S402, using the ET method to call the page.extract_tables() function, extracting tables from the current page, and storing the tables in a tables list, wherein the tables list corresponds to the page one by one;

[0058] Step S403, if the tables list exists and its length is greater than 0, determining that the page corresponding to the tables list contains a table, and determining the page containing the table as the target page;

[0059] Step S404: Summarize all the target pages and use a format converter to convert the first processed file into a Word file as the second processed file.

[0060] Specifically, by writing code to automatically extract tables from documents, a lot of time and energy can be saved. Using the pdfplumber function library combined with the ET method to extract and process tables can improve work efficiency, accuracy, and convenience, making processing tables in PDF documents more convenient and efficient.

[0061] In some examples, the watermarked PDF file is removed, and the enumerate function of the pdfplumber library is used to traverse each page in the PDF file, and the page number and page object are obtained at the same time. The ET method is used to identify the pages containing tables, such as Figure 3 As shown, summarize the pages containing tables and use a format converter to convert PDF to Word document.

[0062] The ET method refers to calling the page.extract_tables() function, which extracts tables from the current page and stores them in the tables list. Then, by checking the tables list, if the tables list exists and its length is greater than 0, it means that the current page contains one or more tables.

[0063] Step S203, replacing the contents of all tables in the second processed file with text boxes containing the same text, to obtain a third processed file;

[0064] Specifically, the cell content without table lines is divided according to line breaks, each divided text is used as the text box content, the text box shape is created, the text box style is set, and finally a Word document with mixed text and graphics is generated, ensuring accurate data extraction of the table without table lines and avoiding the problems of cell misalignment and content recognition errors in traditional methods.

[0065] The steps of replacing the contents of all tables in the second processing file with text boxes containing the same text to obtain a third processing file include the following steps:

[0066] Step S501, retrieving the content of the cell in the second processing file;

[0067] Step S502, when there is no table line in the content of the cell, the content of the cell is segmented by line breaks to obtain segmented texts, and each segmented text is used as the text box content;

[0068] Step S503, creating a text box shape and setting a text box style to obtain a text box, wherein the text box style includes information about the position and size of the text box;

[0069] Step S504, adding the text box to the cell to obtain a Word file containing mixed text and images as the third processed file.

[0070] Specifically, by combining a variety of technical means, the accuracy and stability of table extraction are optimized, ensuring the integrity and reliability of the data.

[0071] Among them, creating a text box shape and setting the text box style to obtain a text box includes the following steps:

[0072] Step S601, creating a structured document tag element and a sdtContent element, and adding the sdtContent element to the structured document tag element to obtain a target sdt element, wherein the structured document tag element is used to represent a text box; the sdtContent element is used as a content container of the text box;

[0073] Step S602, creating a paragraph element, a paragraph attribute element, a text run element and a text element, wherein the paragraph element is used to represent the paragraph of the text box, the paragraph attribute element is used to define the attributes of the paragraph, the text run element is used to represent a section of text in the paragraph, and the text element is used to set the text content as the input text parameter;

[0074] Step S603, adding the paragraph attribute element to the paragraph element to obtain a paragraph processing element, adding the text element to the text run element to obtain a first processing element, then adding the first processing element to the paragraph processing element to obtain a second processing element, and finally adding the second processing element to the sdtContent element to obtain a target sdtContent element;

[0075] Step S604, creating an sdtPr element, a w:sz element and a w:rPr element, wherein the sdtPr element is used to set the properties of the text box, the w:sz element is used to set the font size of the text in the text box, and the w:rPr element is used to represent the running properties of the text in the text box;

[0076] Step S605, obtain the target text box according to the target sdt element, the target sdtContent element, the sdtPr element, the w:sz element and the w:rPr element, add the target text box to the cell, and add the target sdt element to the XML structure of the cell.

[0077] Specifically, the properties of the text box can be set using the sdtPr element, the w:sz element, the w:rPr element, etc., which reduces manual intervention and improves efficiency.

[0078] In some instances, the cell contents are first retrieved, and for the cell contents without table lines, they are first split by line breaks, and then each split text is used as the text box content, a text box shape is created, and the text box style such as the position and size information of the text box is set, and finally the text box is added to the cell, thereby obtaining a Word document with mixed text and images; Figure 4 A schematic diagram of a Word document showing a table extraction method for a table-free line file provided according to an embodiment of the present application is shown, Figure 4 shown.

[0079] The specific steps to create a text box are as follows:

[0080] Create a text box shape: create an sdt element (structured document tag) to represent a text box; create an sdtContent element as the content container of the text box and add it to the sdt.

[0081] Create text content: Create a w:p element (paragraph) to represent the paragraph in the text box; create a w:pPr element (paragraph properties) and add it to the paragraph to define the properties of the paragraph; create a w:r element (run) to represent a piece of text in the paragraph; create a w:t element (text), set the text content to the passed in text parameter, add the w:t element to the w:r, add the w:r to the w:p, and finally add the w:p to the sdtContent.

[0082] Set the text box style: Create an sdtPr element to set the properties of the text box; create a w:sz element to set the font size to 24 (in half-point units); create a w:rPr element, which contains the run properties of the text, and add it to the text box.

[0083] Add a text box to a cell: Add the created sdt element to the XML structure of the cell.

[0084] Step S204, parsing the third processed file to obtain at least the position information of each text box in the third processed file, and grouping and sorting each text box using a threshold method according to the position information of each text box to obtain a plurality of text box groups;

[0085] Specifically, the etree library is used to parse the XML content of a Word document with mixed text and images, obtain the horizontal and vertical position information of the text box, use the Xpath function to query the content in the text box, and use the keyword matching method to clean up trivial text box information, which greatly reduces the workload of manual proofreading during PDF table extraction and improves work efficiency.

[0086] The third processing file is parsed to obtain at least the position information of each text box in the third processing file, including the following steps:

[0087] Step S701, using Xpath function to query the content in the text box, and using etree function library to parse the XML content in the third processing file;

[0088] Step S702, using the upper left corner of the text box as the base point, query the XML node <wp:positionh>and <wp:positionv>, get the initial horizontal information and initial vertical information of the text box;

[0089] Step S703: Convert the initial horizontal information and the initial vertical information into units to obtain the horizontal position information and the vertical position information of the text box in centimeters.

[0090] Specifically, the horizontal and vertical position information of the text box is obtained by querying the XML node <wp:positionh>and <wp:positionv>Get the horizontal and vertical position values ​​of the text box, and then convert EMU to centimeters through unit conversion. The keyword matching method is to summarize the high-frequency words in trivial information as keywords. If keywords appear in the text box, the text box is cleaned up. This greatly reduces the workload of manual proofreading during PDF table extraction and improves work efficiency.

[0091] The method of grouping and sorting the text boxes according to the position information of the text boxes by using a threshold method to obtain a plurality of text box groups includes the following steps:

[0092] Step S801, determining a first target difference, the first target difference being the absolute value of the difference between the horizontal coordinate of the ith text box and the horizontal coordinate of the jth text box, the ith text box being the currently searched text box, and the jth text box being the text box that has been searched and grouped;

[0093] Step S802, when the first target difference is less than or equal to a threshold, adding the i-th text box to the group to which the j-th text box belongs; when the first target difference is greater than the threshold, establishing a new group based on the i-th text box to obtain a plurality of row groups;

[0094] Step S803, taking the smallest row coordinate in each of the row groups as the representative coordinate of the group and performing bubble sorting among the row groups;

[0095] Step S804, determining a second target difference, the second target difference being the absolute value of the difference between the ordinate of the ith text box and the ordinate of the jth text box, the ith text box being the currently searched text box, and the jth text box being the already searched and grouped text box;

[0096] Step S805, when the second target difference is less than or equal to a threshold, adding the i-th text box to the group to which the j-th text box belongs; when the second target difference is greater than the threshold, creating a new group based on the i-th text box, to obtain multiple column groupings;

[0097] Step S806 , taking the smallest vertical coordinate in each of the above-mentioned column groups as the representative coordinate of this group and performing bubble sorting among the above-mentioned column groups.

[0098] Specifically, the threshold method searches for the position information of all text boxes in sequence. Each time the position information of a text box is searched, the absolute value is obtained by subtracting it from the text boxes that have been searched before. When the absolute value is less than or equal to the threshold, the two text boxes belong to the same group. The threshold is selected through data analysis and statistical methods. The specific steps are as follows: 20 groups of different table samples are randomly selected. First, the approximate range of the threshold is defined based on experience. Then this range is divided into 10 parts, and these 10 thresholds are applied to each group of samples in turn. Finally, the threshold with the best grouping effect is selected to improve the accuracy of table extraction.

[0099] In some examples, the data in the text box is grouped according to the horizontal and vertical coordinates using a threshold method, wherein the horizontal axis grouping process is performed.

[0100] like Figure 5 As shown, Figure 5 A method for sorting text boxes provided according to an embodiment of the present application is shown; for the grouped text boxes, the smallest coordinate in each group is used as the representative coordinate of the group and bubble sort is performed. The specific steps are as follows:

[0101] S11: Initialize the group list, and then query the i-th text box row axis coordinate marked as x i ;

[0102] S12: Determine whether the formula |x is satisfied i -x j |<=k, if it satisfies, add the text box to x j The group to which it belongs, if it does not meet the requirements, a new group will be created;

[0103] S13: Add text box to x j After the group it belongs to, iterate through the remaining text boxes;

[0104] S14: If the traversal is completed, the groups are numbered sequentially according to the minimum coordinates of each group, otherwise return to step S11.

[0105] Step S205, filling the above text boxes into Excel cells according to the group order to generate an Excel file.

[0106] Specifically, bubble sort is performed on the grouped groups, the row groups correspond to Excel column numbers, the column groups correspond to Excel row numbers, the text box contents are filled into Excel cells, and an Excel file is generated. This overcomes the problems of cell misalignment and content recognition errors during direct format conversion, ensures the accuracy and completeness of the data, and provides a reliable guarantee for the subsequent use of the budget quota table data for power grid projects.

[0107] The steps of filling the above text boxes into Excel cells in the order of groups to generate an Excel file include the following:

[0108] Step S901, obtaining the row group number and column group number of each of the text boxes, wherein the row group number of the text box corresponds to the column number in the Excel file, and the column group number of the text box corresponds to the row number in the Excel file;

[0109] Step S902, filling each of the text boxes into Excel cells in group order according to the row group number and column group number of each of the text boxes, to generate the Excel file.

[0110] Specifically, through automated processing and precise data extraction and conversion, accurate data extraction of tables without table lines is ensured, avoiding the problems of cell misalignment and content recognition errors in traditional methods, greatly reducing the workload of manual proofreading during PDF table extraction and improving work efficiency.

[0111] Among them, for the grouped text boxes, the smallest coordinate in each group is used as the representative coordinate of the group, and bubble sort is performed. For each text box, the group number obtained by the row position corresponds to the Excel column number, and the group number obtained by the column position corresponds to the Excel row number. Then the text box content is filled into the Excel cell to generate an Excel file. Figure 6 As shown, Figure 6 A table-free line table conversion result diagram of a table extraction method for a table-free line file provided according to an embodiment of the present application is shown.

[0112] The table extraction method of the above-mentioned table-free line file of the present application first removes the watermark processing of the power grid engineering budget quota PDF file to obtain the first processing file; then retrieves the page containing the table in the first processing file as the target page, and summarizes and converts all the target pages into Word files to obtain the second processing file; then replaces the contents of all tables in the second processing file with text boxes containing the same text to obtain the third processing file; parses the third processing file to at least obtain the position information of each text box in the third processing file, and uses the threshold method to group and sort each text box according to the position information of each text box to obtain multiple text box groups; finally, fills each text box into the Excel cell in the order of the group to generate an Excel file. This method can solve the problem that the traditional format conversion method often leads to cell dislocation and content recognition errors, which seriously affects the accuracy and practicality of the data, realizes the effective extraction of table-free line tables in the power grid engineering budget quota PDF file, overcomes the difficulties such as cell dislocation and content recognition errors when directly performing format conversion, ensures the accuracy and integrity of the data, and provides reliable guarantee for the subsequent use of power grid engineering budget quota table data.

[0113] In order to enable those skilled in the art to more clearly understand the technical solution of the present application, the implementation process of the table extraction method for table-free line files of the present application will be described in detail below in conjunction with specific embodiments.

[0114] This embodiment relates to a specific table extraction method for a table-free line file, such as Figure 7 As shown, the following steps are included:

[0115] Step S1: removing the watermark from the power grid project budget quota PDF file;

[0116] Step S2: remove the watermark from the PDF file, summarize the pages containing the table and convert the PDF into a Word document using a format converter;

[0117] Step S3: the contents of all tables in the converted Word document are replaced with text boxes containing the same text;

[0118] Step S4: parsing the position information of the text boxes and grouping and sorting them according to the position information;

[0119] Step S5: The text boxes are filled into the corresponding cells of Excel according to the groups to generate an Excel file.

[0120] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0121] The embodiment of the present application also provides a table extraction device for table-free line files. It should be noted that the table extraction device for table-free line files in the embodiment of the present application can be used to execute the table extraction method for table-free line files provided in the embodiment of the present application. The device is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware for a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation in hardware, or a combination of software and hardware, is also possible and conceivable.

[0122] The following is an introduction to the table extraction device for a table line-free file provided in an embodiment of the present application.

[0123] Figure 8 Schematic diagram of a table extraction device for a table-free line file according to an embodiment of the present application. Figure 8 As shown, the device includes: a first processing unit 10, a second processing unit 20, a third processing unit 30, a fourth processing unit 40 and a fifth processing unit 50, the first processing unit 10 is used to remove the watermark of the power grid engineering budget quota PDF file to obtain a first processing file; the second processing unit 20 is used to retrieve the page containing the table in the above-mentioned first processing file as the target page, and summarize and convert all the above-mentioned target pages into Word files to obtain a second processing file; the third processing unit 30 is used to replace the contents of all tables in the above-mentioned second processing file with text boxes containing the same text to obtain a third processing file; the fourth processing unit 40 is used to parse the above-mentioned third processing file to at least obtain the position information of each text box in the above-mentioned third processing file, and use the threshold method to group and sort each of the above text boxes according to the position information of each of the above text boxes to obtain a plurality of text box groups; the fifth processing unit 50 is used to fill each of the above text boxes into Excel cells in the order of the groups to generate an Excel file.

[0124] The table extraction device of the above-mentioned table-free line file of the present application includes: a first processing unit, a second processing unit, a third processing unit, a fourth processing unit and a fifth processing unit, wherein the first processing unit is used to remove the watermark of the power grid engineering budget quota PDF file to obtain a first processing file; the second processing unit is used to retrieve the page containing the table in the above-mentioned first processing file as the target page, and summarize and convert all the above-mentioned target pages into Word files to obtain a second processing file; the third processing unit is used to replace the contents of all tables in the above-mentioned second processing file with text boxes containing the same text to obtain a third processing file; the fourth processing unit is used to parse the above-mentioned third processing file to at least obtain the content in the above-mentioned third processing file The device can solve the problem that traditional format conversion methods often lead to cell misalignment and content recognition errors, which seriously affect the accuracy and practicality of data, realize the effective extraction of tables without table lines in the PDF file of power grid engineering budget quota, overcome the difficulties such as cell misalignment and content recognition errors when directly performing format conversion, ensure the accuracy and completeness of data, and provide reliable guarantee for the subsequent use of power grid engineering budget quota table data.

[0125] In some examples, the first processing unit includes a first conversion module, a second conversion module, a first processing module, and a second processing module. The first conversion module is used to use PyMuPDF in Python to convert the pages in the above-mentioned power grid engineering budget quota PDF file into PNG format to obtain an image file; the second conversion module is used to use OpenCV and NumPy to perform Fourier transform, convert the above-mentioned image file to the frequency domain, and obtain an initial frequency domain image; the first processing module is used to identify and filter out the specific frequency components corresponding to the watermark in the above-mentioned initial frequency domain image in the frequency domain, and obtain a processed frequency domain image; the second processing module is used to perform an inverse Fourier transform on the above-mentioned processed frequency domain image to obtain a watermark-removed image, and save the above-mentioned watermark-removed image as a PDF file to obtain the above-mentioned first processed file. The specific frequency components of the watermark are identified and filtered out by Fourier transform, so as to achieve accurate watermark removal and ensure that the image quality in the PDF file is not damaged. Through Fourier transform, the specific frequency components of the watermark are identified and filtered out to achieve accurate watermark removal, ensuring that the image quality in the PDF file is not damaged. Python scripts and multiple function libraries (PyMuPDF, OpenCV, NumPy, pdfplumber, etc.) are used to automatically process PDF files, reducing manual intervention and improving efficiency.

[0126] In some examples, the second processing unit includes a third processing module, a first storage module, a first determination module and a third conversion module, the third processing module is used to use the enumerate of the pdfplumber function library to traverse each page in the first processing file, and obtain the page number and page object at the same time; the first storage module is used to use the ET method to call the page.extract_tables() function to extract the table from the current page, and store the table in the tables list, and the tables list corresponds to the page one by one; the first determination module is used to determine that the page corresponding to the tables list contains a table when the tables list exists and the length is greater than 0, and determine the page containing the table as the target page; the third conversion module is used to summarize all the target pages and use the format converter to convert the first processing file into a Word file as the second processing file. By writing code to realize the automatic processing of table extraction in documents, a lot of time and energy can be saved, and the use of the pdfplumber function library combined with the ET method to extract and process tables can improve work efficiency, accuracy and convenience, making it more convenient and efficient to process tables in PDF documents.

[0127] In some examples, the third processing unit includes a first retrieval module, a first segmentation module, a fourth processing module and a fifth processing module. The first retrieval module is used to retrieve the content of the cell in the second processing file; the first segmentation module is used to segment the content of the cell using a line break when there is no table line in the content of the cell to obtain the segmented text, and each segmented text is used as the text box content; the fourth processing module is used to create a text box shape and set the text box style to obtain a text box, and the text box style includes the position of the text box and the size information of the text box; the fifth processing module is used to add the text box to the cell to obtain a Word file with mixed text and images as the third processing file. It ensures that the data extraction of the table without table lines is accurate, avoiding the problems of cell misalignment and content recognition errors in traditional methods.

[0128] In some instances, the fourth processing module includes a first creation module, a second creation module, a sixth processing module, a third creation module and a seventh processing module, wherein the first creation module is used to create a structured document tag element and an sdtContent element, and add the sdtContent element to the structured document tag element to obtain a target sdt element, wherein the structured document tag element is used to represent a text box; the sdtContent element serves as a content container for the text box; the second creation module is used to create a paragraph element, a paragraph attribute element, a text run element and a text element, wherein the paragraph element is used to represent a paragraph of the text box, the paragraph attribute element is used to define the attributes of the paragraph, the text run element is used to represent a paragraph of text in the paragraph, and the text element is used to set the text content to the passed-in text parameter; the sixth processing module is used to add the paragraph attribute element to the paragraph element to obtain a paragraph processing element, adding the above text element to the above text run element to obtain the first processing element, then adding the above first processing element to the above paragraph processing element to obtain the second processing element, and finally adding the above second processing element to the above sdtContent element to obtain the target sdtContent element; the third creation module is used to create sdtPr elements, w:sz elements and w:rPr elements, the above sdtPr element is used to set the properties of the above text box, the above w:sz element is used to set the font size of the text in the above text box, and the above w:rPr element is used to characterize the run properties of the text in the above text box; the seventh processing module is used to obtain the target text box according to the above target sdt element, the above target sdtContent element, the above sdtPr element, the above w:sz element and the above w:rPr element, and add the above target text box to the cell, and add the above target sdt element to the XML structure of the cell. The sdtPr element, w:sz element, and w:rPr element can be used to set the properties of the text box, reducing manual intervention and improving efficiency.

[0129] In some examples, the fourth processing unit includes a first parsing module, a first query module and an eighth processing module. The first parsing module is used to use the Xpath function to query the content in the text box and use the etree function library to parse the XML content in the third processing file. The first query module is used to use the upper left corner of the text box as a base point to query the XML node. <wp:positionh>and <wp:positionv>, and obtain the initial horizontal information and initial vertical information of the text box; the eighth processing module is used to convert the above initial horizontal information and the above initial vertical information into units, and obtain the horizontal position information and vertical position information of the above text box in centimeters. This greatly reduces the workload of manual proofreading during PDF form extraction and improves work efficiency.

[0130] In some instances, the fourth processing unit includes a second determination module, a first comparison module, a first sorting module, a third determination module, a second comparison module and a second sorting module, the second determination module is used to determine a first target difference, the first target difference is the absolute value of the difference between the horizontal coordinate of the i-th text box and the horizontal coordinate of the j-th text box, the i-th text box is the text box currently being retrieved, and the j-th text box is a text box that has been retrieved and grouped; the first comparison module is used to add the i-th text box to the group to which the j-th text box belongs when the first target difference is less than or equal to a threshold, and to establish a new group based on the i-th text box when the first target difference is greater than the threshold to obtain multiple row groups; the first sorting module is used to take the smallest row coordinate in each of the above row groups as the group of the present text box. The representative coordinates of the group are used to perform bubble sorting between the above-mentioned row groups; the third determination module is used to determine the second target difference, the above-mentioned second target difference is the absolute value of the difference between the vertical coordinates of the ith text box and the vertical coordinates of the jth text box, the above-mentioned ith text box is the text box currently being retrieved, and the above-mentioned jth text box is the text box that has been retrieved and grouped; the second comparison module is used to add the above-mentioned ith text box to the group to which the above-mentioned jth text box belongs when the above-mentioned second target difference is less than or equal to the threshold value, and to establish a new group based on the above-mentioned ith text box when the above-mentioned second target difference is greater than the above-mentioned threshold value, so as to obtain multiple column groups; the second sorting module is used to use the smallest vertical coordinate in each of the above-mentioned column groups as the representative coordinate of the group to perform bubble sorting between the above-mentioned column groups, thereby improving the accuracy of table extraction.

[0131] In some instances, the fifth processing unit includes a first acquisition module and a ninth processing module, the first acquisition module is used to obtain the row group number and the column group number of each of the above-mentioned text boxes, wherein the row group number of the above-mentioned text box corresponds to the column number in the above-mentioned Excel file, and the column group number of the above-mentioned text box corresponds to the row number in the above-mentioned Excel file; the ninth processing module is used to fill each of the above-mentioned text boxes into the Excel cell in group order according to the row group number and the column group number of each of the above-mentioned text boxes, and generate the above-mentioned Excel file, which greatly reduces the workload of manual proofreading during PDF table extraction and improves work efficiency.

[0132] The table extraction device for the table-free line file includes a processor and a memory. The first processing unit and the like are stored in the memory as program units, and the processor executes the program units stored in the memory to implement corresponding functions. The modules are all located in the same processor; or, the modules are located in different processors in any combination.

[0133] The processor contains a kernel, which calls the corresponding program unit from the memory. One or more kernels can be set, and the kernel parameters can be adjusted to solve the problem that traditional format conversion methods often lead to cell misalignment and content recognition errors, which seriously affect the accuracy and practicality of data.

[0134] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0135] An embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the table extraction method for the table-free line file.

[0136] Specifically, the table extraction method of the table-free line file includes:

[0137] Step S201, removing the watermark from the power grid project budget quota PDF file to obtain a first processed file;

[0138] Specifically, Fourier transform is used to identify and filter out specific frequency components of the watermark to achieve accurate watermark removal and ensure that the image quality in the PDF file is not damaged.

[0139] Step S202, retrieving the pages containing the table in the first processing file as target pages, and aggregating and converting all the target pages into Word files to obtain a second processing file;

[0140] Specifically, by using the pdfplumber function library enumerate and calling the page.extract_tables() function, the page number and page number object are automatically obtained and correspond one-to-one with the tables list, which improves efficiency and ensures accurate data extraction of tables without table lines.

[0141] Step S203, replacing the contents of all tables in the second processed file with text boxes containing the same text, to obtain a third processed file;

[0142] Specifically, the cell content without table lines is divided according to line breaks, each divided text is used as the text box content, the text box shape is created, the text box style is set, and finally a Word document with mixed text and graphics is generated, ensuring accurate data extraction of the table without table lines and avoiding the problems of cell misalignment and content recognition errors in traditional methods.

[0143] Step S204, parsing the third processed file to obtain at least the position information of each text box in the third processed file, and grouping and sorting each text box using a threshold method according to the position information of each text box to obtain a plurality of text box groups;

[0144] Specifically, the etree library is used to parse the XML content of a Word document with mixed text and images, obtain the horizontal and vertical position information of the text box, use the Xpath function to query the content in the text box, and use the keyword matching method to clean up trivial text box information, which greatly reduces the workload of manual proofreading during PDF table extraction and improves work efficiency.

[0145] Step S205, filling the above text boxes into Excel cells according to the group order to generate an Excel file.

[0146] Specifically, the grouped groups are bubble sorted, the row groups correspond to the Excel column numbers, the column groups correspond to the Excel row numbers, the text box contents are filled into the Excel cells, and an Excel file is generated, which greatly reduces the workload of manual proofreading during PDF table extraction and improves work efficiency.

[0147] An embodiment of the present invention provides a processor, and the processor is used to run a program, wherein the program executes the table extraction method of the table-free line file when running.

[0148] Specifically, the table extraction method of the table-free line file includes:

[0149] Step S201, removing the watermark from the power grid project budget quota PDF file to obtain a first processed file;

[0150] Specifically, Fourier transform is used to identify and filter out specific frequency components of the watermark to achieve accurate watermark removal and ensure that the image quality in the PDF file is not damaged.

[0151] Step S202, retrieving the pages containing the table in the first processing file as target pages, and aggregating and converting all the target pages into Word files to obtain a second processing file;

[0152] Specifically, by using the pdfplumber function library enumerate and calling the page.extract_tables() function, the page number and page number object are automatically obtained and correspond one-to-one with the tables list, which improves efficiency and ensures accurate data extraction of tables without table lines.

[0153] Step S203, replacing the contents of all tables in the second processed file with text boxes containing the same text, to obtain a third processed file;

[0154] Specifically, the cell content without table lines is divided according to line breaks, each divided text is used as the text box content, the text box shape is created, the text box style is set, and finally a Word document with mixed text and graphics is generated, ensuring accurate data extraction of the table without table lines and avoiding the problems of cell misalignment and content recognition errors in traditional methods.

[0155] Step S204, parsing the third processed file to obtain at least the position information of each text box in the third processed file, and grouping and sorting each text box using a threshold method according to the position information of each text box to obtain a plurality of text box groups;

[0156] Specifically, the etree library is used to parse the XML content of a Word document with mixed text and images, obtain the horizontal and vertical position information of the text box, use the Xpath function to query the content in the text box, and use the keyword matching method to clean up trivial text box information, which greatly reduces the workload of manual proofreading during PDF table extraction and improves work efficiency.

[0157] Step S205, filling the above text boxes into Excel cells according to the group order to generate an Excel file.

[0158] Specifically, the grouped groups are bubble sorted, the row groups correspond to the Excel column numbers, the column groups correspond to the Excel row numbers, the text box contents are filled into the Excel cells, and an Excel file is generated, which greatly reduces the workload of manual proofreading during PDF table extraction and improves work efficiency.

[0159] An embodiment of the present invention provides a device, the device including a processor, a memory, and a program stored in the memory and executable on the processor, and when the processor executes the program, at least the following steps are implemented:

[0160] Step S201, removing the watermark from the power grid project budget quota PDF file to obtain a first processed file;

[0161] Specifically, Fourier transform is used to identify and filter out specific frequency components of the watermark to achieve accurate watermark removal and ensure that the image quality in the PDF file is not damaged.

[0162] Step S202, retrieving the pages containing the table in the first processing file as target pages, and aggregating and converting all the target pages into Word files to obtain a second processing file;

[0163] Specifically, by using the pdfplumber function library enumerate and calling the page.extract_tables() function, the page number and page number object are automatically obtained and correspond one-to-one with the tables list, which improves efficiency and ensures accurate data extraction of tables without table lines.

[0164] Step S203, replacing the contents of all tables in the second processed file with text boxes containing the same text, to obtain a third processed file;

[0165] Specifically, the cell content without table lines is divided according to line breaks, each divided text is used as the text box content, the text box shape is created, the text box style is set, and finally a Word document with mixed text and graphics is generated, ensuring accurate data extraction of the table without table lines and avoiding the problems of cell misalignment and content recognition errors in traditional methods.

[0166] Step S204, parsing the third processed file to obtain at least the position information of each text box in the third processed file, and grouping and sorting each text box using a threshold method according to the position information of each text box to obtain a plurality of text box groups;

[0167] Specifically, the etree library is used to parse the XML content of a Word document with mixed text and images, obtain the horizontal and vertical position information of the text box, use the Xpath function to query the content in the text box, and use the keyword matching method to clean up trivial text box information, which greatly reduces the workload of manual proofreading during PDF table extraction and improves work efficiency.

[0168] Step S205, filling the above text boxes into Excel cells according to the group order to generate an Excel file.

[0169] Specifically, the grouped groups are bubble sorted, the row groups correspond to the Excel column numbers, the column groups correspond to the Excel row numbers, the text box contents are filled into the Excel cells, and an Excel file is generated, which greatly reduces the workload of manual proofreading during PDF table extraction and improves work efficiency.

[0170] The devices in this article can be servers, PCs, PADs, mobile phones, etc.

[0171] The present application also provides a computer program product, which, when executed on a data processing device, is suitable for executing a program for initializing at least the following method steps:

[0172] Step S201, removing the watermark from the power grid project budget quota PDF file to obtain a first processed file;

[0173] Specifically, Fourier transform is used to identify and filter out specific frequency components of the watermark to achieve accurate watermark removal and ensure that the image quality in the PDF file is not damaged.

[0174] Step S202, retrieving the pages containing the table in the first processing file as target pages, and aggregating and converting all the target pages into Word files to obtain a second processing file;

[0175] Specifically, by using the pdfplumber function library enumerate and calling the page.extract_tables() function, the page number and page number object are automatically obtained and correspond one-to-one with the tables list, which improves efficiency and ensures accurate data extraction of tables without table lines.

[0176] Step S203, replacing the contents of all tables in the second processed file with text boxes containing the same text, to obtain a third processed file;

[0177] Specifically, the cell content without table lines is divided according to line breaks, each divided text is used as the text box content, the text box shape is created, the text box style is set, and finally a Word document with mixed text and graphics is generated, ensuring accurate data extraction of the table without table lines and avoiding the problems of cell misalignment and content recognition errors in traditional methods.

[0178] Step S204, parsing the third processed file to obtain at least the position information of each text box in the third processed file, and grouping and sorting each text box using a threshold method according to the position information of each text box to obtain a plurality of text box groups;

[0179] Specifically, the etree library is used to parse the XML content of a Word document with mixed text and images, obtain the horizontal and vertical position information of the text box, use the Xpath function to query the content in the text box, and use the keyword matching method to clean up trivial text box information, which greatly reduces the workload of manual proofreading during PDF table extraction and improves work efficiency.

[0180] Step S205, filling the above text boxes into Excel cells according to the group order to generate an Excel file.

[0181] Specifically, the grouped groups are bubble sorted, the row groups correspond to the Excel column numbers, the column groups correspond to the Excel row numbers, the text box contents are filled into the Excel cells, and an Excel file is generated, which greatly reduces the workload of manual proofreading during PDF table extraction and improves work efficiency.

[0182] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order than here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0183] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0184] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0185] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0186] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0187] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0188] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0189] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0190] It should also be noted that the term "includes", "comprising" or any other variation thereof is intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of further restrictions, an element defined by the sentence "comprising a ..." does not exclude the existence of other identical elements in the process, method, commodity or device including the element.

[0191] From the above description, it can be seen that the above embodiments of the present application achieve the following technical effects:

[0192] 1) The table extraction method of the above-mentioned table-free line file of the present application first removes the watermark of the power grid engineering budget quota PDF file to obtain the first processing file; then retrieves the page containing the table in the first processing file as the target page, and summarizes and converts all the target pages into Word files to obtain the second processing file; then replaces the contents of all tables in the second processing file with text boxes containing the same text to obtain the third processing file; parses the third processing file to at least obtain the position information of each text box in the third processing file, and uses the threshold method to group and sort each text box according to the position information of each text box to obtain multiple text box groups; finally, fills each text box into the Excel cell in the order of the group to generate an Excel file. This method can solve the problem that the traditional format conversion method often leads to cell dislocation and content recognition errors, which seriously affects the accuracy and practicality of the data, realizes the effective extraction of table-free line tables in the power grid engineering budget quota PDF file, overcomes the difficulties such as cell dislocation and content recognition errors when directly performing format conversion, ensures the accuracy and integrity of the data, and provides reliable guarantee for the subsequent use of power grid engineering budget quota table data.

[0193] 2) The table extraction device of the above-mentioned table-free line file of the present application includes: a first processing unit, a second processing unit, a third processing unit, a fourth processing unit and a fifth processing unit. The first processing unit is used to remove the watermark of the power grid engineering budget quota PDF file to obtain a first processing file; the second processing unit is used to retrieve the page containing the table in the above-mentioned first processing file as the target page, and summarize and convert all the above-mentioned target pages into Word files to obtain a second processing file; the third processing unit is used to replace the contents of all tables in the above-mentioned second processing file with text boxes containing the same text to obtain a third processing file; the fourth processing unit is used to parse the above-mentioned third processing file to at least obtain the position information of each text box in the above-mentioned third processing file, and use the threshold method to group and sort each of the above text boxes according to the position information of each of the above text boxes to obtain multiple text box groups; the fifth processing unit is used to fill each of the above text boxes into Excel cells in the order of the groups to generate an Excel file. The device can solve the problem that traditional format conversion methods often lead to cell misalignment and content recognition errors, which seriously affect the accuracy and practicality of the data. It realizes the effective extraction of tables without table lines in the power grid engineering budget quota PDF file, overcomes the difficulties of cell misalignment and content recognition errors during direct format conversion, ensures the accuracy and completeness of the data, and provides a reliable guarantee for the subsequent use of power grid engineering budget quota table data.

[0194] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.< / wp:positionv> < / wp:positionh> < / wp:positionv> < / wp:positionh> < / wp:positionv> < / wp:positionh> < / wp:positionv> < / wp:positionh>

Claims

1. A table extraction method for a table-free line file, characterized in that: include: The power grid project budget quota PDF file is processed by removing the watermark to obtain a first processed file; Retrieving the pages containing tables in the first processing file as target pages, and aggregating and converting all the target pages into Word files to obtain a second processing file; Replacing the contents of all tables in the second processed file with text boxes containing the same text to obtain a third processed file; Parsing the third processing file to obtain at least position information of each text box in the third processing file, and grouping and sorting each text box using a threshold method according to the position information of each text box to obtain a plurality of text box groups; Fill each of the text boxes into the Excel cells in the order of groups to generate an Excel file.

2. The method according to claim 1, characterized in that The power grid project budget quota PDF file is processed to remove the watermark, and the first processed file is obtained, including: Use PyMuPDF in Python to convert the pages in the power grid project budget quota PDF file into PNG format to obtain an image file; Perform Fourier transform using OpenCV and NumPy to convert the image file into the frequency domain to obtain an initial frequency domain image; Identifying and filtering out specific frequency components corresponding to the watermark in the initial frequency domain image in the frequency domain to obtain a processed frequency domain image; The processed frequency domain image is subjected to inverse Fourier transformation to obtain a watermark-removed image, and the watermark-removed image is saved as a PDF file to obtain the first processed file.

3. The method according to claim 1, characterized in that Retrieving the pages containing the table in the first processing file as the target pages, and aggregating and converting all the target pages into Word files to obtain a second processing file, including: Use enumerate of the pdfplumber function library to traverse each page in the first processing file and obtain the page number and page object; Use the ET method to call the page.extract_tables() function to extract the table from the current page and store the table in the tables list, which corresponds to the page one by one; In the case that the tables list exists and has a length greater than 0, determining that the page corresponding to the tables list contains a table, and determining the page containing the table as a target page; All the target pages are aggregated and the first processed file is converted into a Word file using a format converter as the second processed file.

4. The method according to claim 1, characterized in that: The contents of all tables in the second processed file are replaced with text boxes containing the same text, to obtain a third processed file, including: Retrieving the contents of the cells in the second processed file; In the case where there is no table line in the content of the cell, the content of the cell is segmented using a line break to obtain segmented texts, and each segmented text is used as the text box content; Create a text box shape and set the text box style to obtain a text box, wherein the text box style includes the position of the text box and the size information of the text box; The text box is added to the cell to obtain a Word file with mixed text and images as the third processed file.

5. The method according to claim 4, characterized in that Create a text box shape and set the text box style to get the text box, including: Create a structured document tag element and a sdtContent element, and add the sdtContent element to the structured document tag element to obtain a target sdt element, wherein the structured document tag element is used to represent a text box; the sdtContent element serves as a content container of the text box; Create a paragraph element, a paragraph attribute element, a text run element and a text element, wherein the paragraph element is used to represent the paragraph of the text box, the paragraph attribute element is used to define the attributes of the paragraph, the text run element is used to represent a piece of text in the paragraph, and the text element is used to set the text content to the passed-in text parameter, Add the paragraph attribute element to the paragraph element to obtain a paragraph processing element, add the text element to the text run element to obtain a first processing element, then add the first processing element to the paragraph processing element to obtain a second processing element, and finally add the second processing element to the sdtContent element to obtain a target sdtContent element; Create an sdtPr element, a w:sz element, and a w:rPr element, wherein the sdtPr element is used to set the properties of the text box, the w:sz element is used to set the font size of the text in the text box, and the w:rPr element is used to characterize the running properties of the text in the text box; According to the target sdt element, the target sdtContent element, the sdtPr element, the w:sz element and the w:rPr element, a target text box is obtained, and the target text box is added to the cell, and the target sdt element is added to the XML structure of the cell.

6. The method according to claim 1, characterized in that Parsing the third processing file to obtain at least the position information of each text box in the third processing file includes: Use the Xpath function to query the content in the text box, and use the etree function library to parse the XML content in the third processing file; Using the upper left corner of the text box as the base point, query the XML node <wp:positionh>and <wp:positionv> , get the initial horizontal information and initial vertical information of the text box;< / wp:positionv> < / wp:positionh> The initial horizontal information and the initial vertical information are converted into units to obtain the horizontal position information and the vertical position information of the text box in centimeters.

7. The method according to claim 1, characterized in that The text boxes are grouped and sorted using a threshold method according to the position information of each text box to obtain a plurality of text box groups, including: Determine a first target difference, where the first target difference is an absolute value of a difference between a horizontal coordinate of an i-th text box and a horizontal coordinate of a j-th text box, where the i-th text box is a currently searched text box and the j-th text box is a text box that has been searched and grouped; When the first target difference is less than or equal to a threshold, the i-th text box is added to the group to which the j-th text box belongs; when the first target difference is greater than the threshold, a new group is established based on the i-th text box to obtain a plurality of row groups; Using the smallest row coordinate in each row group as the representative coordinate of the group to perform bubble sorting among the row groups; Determine a second target difference value, where the second target difference value is an absolute value of a difference between a ordinate of an i-th text box and a ordinate of a j-th text box, where the i-th text box is a text box currently being retrieved, and the j-th text box is a text box that has been retrieved and grouped; When the second target difference is less than or equal to a threshold, the i-th text box is added to the group to which the j-th text box belongs; when the second target difference is greater than the threshold, a new group is established based on the i-th text box to obtain a plurality of column groupings; The smallest vertical coordinate in each of the column groups is used as the representative coordinate of the group to perform bubble sorting among the column groups.

8. The method according to claim 1, characterized in that Fill each of the text boxes into the Excel cells in the order of groups to generate an Excel file, including: Obtaining a row group number and a column group number of each of the text boxes, wherein the row group number of the text box corresponds to the column number in the Excel file, and the column group number of the text box corresponds to the row number in the Excel file; The text boxes are filled into cells of Excel in group order according to the row group number and column group number of each text box to generate the Excel file.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the table extraction method for a table-free line file according to any one of claims 1 to 7.

10. An electronic device, characterized in that: include: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include a method for executing the table extraction method of the table-free line file described in any one of claims 1 to 7.