A high-efficiency extraction method of periodical literature table based on keyword and frame features
By using a keyword and border feature-based method, the table areas of journal articles are automatically identified, which solves the problems of low accuracy and insufficient automation in existing technologies. It achieves accurate extraction of tables with full borders and few borders and improves the accuracy of logical relationships.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- GUANGXI UNIV
- Filing Date
- 2022-11-24
- Publication Date
- 2026-04-17
AI Technical Summary
Existing technologies struggle to efficiently and automatically extract tables with few borders from journal articles, resulting in low extraction accuracy and a cumbersome process that fails to meet the demands of the big data era.
By establishing character and line sets, utilizing table keywords and border features, and combining language classification rules and layout features, table areas are automatically identified and bound, achieving fully automated extraction of table data.
It achieves accurate extraction of tables with full borders and few borders, improves the automation level of table data and the accuracy of logical relationships, and is applicable to Chinese and English documents, with wide applicability and selectivity.
Smart Images

Figure CN116070589B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of PDF file information extraction, and in particular to an efficient method for extracting journal literature tables based on keyword and border features. Background Technology
[0002] PDF (Portable Document Format) is a document format based on the PostScript language's image model. It accurately reproduces the characters, colors, and images of a document and is widely used in online electronic documents. It is an internationally accepted file format for the transmission and storage of journal articles. Journal articles contain a wealth of professional knowledge and experimental data, making them an important data source for current big data applications. Therefore, quickly obtaining and fully utilizing the effective information in journal articles is extremely important.
[0003] Tables, as a carrier of high information density in journal articles, often contain key information within the document. Currently, the number of academic documents is exploding, and manual retrieval can no longer keep up with the speed of information growth. Especially with the maturity of big data technology and its penetration into various industries, table data in journal articles, as a major source of big data in relevant fields, is receiving increasing attention from big data mining and applied research. The demand for extracting tables from journal articles (especially scientific and technological literature) is also becoming increasingly strong. Therefore, automatically extracting table data from PDFs has become an important part of big data applications, laying the foundation for data sharing and utilization in related fields.
[0004] Common document tables establish cell relationships using borders, with clear boundaries between cells (full-bordered tables). Currently, most table extraction methods target full-bordered tables with minimal redundant data, using borders to identify and distinguish data areas. However, journal articles often use tables with fewer borders (such as three-line tables) and contain a significant amount of non-table data, severely impacting the effectiveness of extraction methods for this type of table. A few extraction tools exist for tables with fewer borders, but these require multiple manual selections of the table area to improve accuracy, a cumbersome process that fails to meet the demands of automated and efficient table data acquisition in the era of big data.
[0005] Therefore, for tables in journal articles, a high-efficiency PDF table extraction method based on keywords and border features was designed. By utilizing the features of the beginning and end of the table, a table area is established to achieve fully automatic identification of tables with full borders and few borders, accurately obtain table data, and restore the row and column logical relationships between cells. Summary of the Invention
[0006] The purpose of this invention is to provide an efficient method for extracting journal literature tables based on keywords and border features, thereby solving the technical problems of low accuracy and low automation in the extraction of data from tables with few borders.
[0007] To achieve the above objectives, the technical solution adopted by the present invention is as follows:
[0008] An efficient method for extracting journal article tables based on keyword and bounding box features, the method comprising the following steps:
[0009] Step 1: Establish character sets and line sets, decode the PDF by page number, and obtain the page size, the text set of all characters on each page, the line set of all lines, and their information status parameters, including character font size, character horizontal and vertical coordinates, line height, and line horizontal and vertical coordinates;
[0010] Step 2: Define language classification rules. Divide PDF documents into Chinese documents and English documents, and create tables with keywords for Chinese and English documents respectively;
[0011] Step 3: Construct a set of table keywords Key, clean the characters using the characteristics of the table keywords, and obtain the table keywords after the initial cleaning;
[0012] Step 4: Construct a set of table borders Line, clean the lines using table border features, and obtain the table borders after the initial cleaning.
[0013] Step 5: To facilitate the differentiation of table areas, the keyword set Key is divided into the right column keyword set KeyR and the non-right column keyword set KeyNR according to whether the document is a two-column layout. The border set is divided into the right column border set LineR and the non-right column border set LineNR.
[0014] Step 6: Compare the vertical coordinates of the borders and keywords, bind the keywords belonging to the same table to the borders to form a set Table, and get the last border of each table;
[0015] Step 7: Determine the coordinates of the top left corner of the table area based on the horizontal and vertical coordinates of the keywords, and determine the coordinates of the bottom right corner of the table area based on the horizontal and vertical coordinates of the bottom right corner of the last border. The rectangular area of the table is then determined, and the table data is output.
[0016] Furthermore, in step 1, the horizontal and vertical coordinates of the character only represent the horizontal coordinate of the left side and the vertical coordinate of the top side of the character, without considering the coordinates of the right side and the bottom side. The line height is the default vertical height, and the horizontal and vertical coordinates of the line include the horizontal coordinate of the left side, the vertical coordinate of the top side, the horizontal coordinate of the right side, and the vertical coordinate of the bottom side of the line.
[0017] Further, in the step 2, the language classification rule is to obtain the characters with the largest font size and the second largest font size on the first page of the PDF. If the ASCII code values of these characters are all within the range of "\u4e00" - "\u9fa5", the document is a Chinese document; if the ASCII code values of these characters are all within [65, 90] or [97, 122], the document is an English document.
[0018] Further, in the step 2, the keyword for the table in the Chinese document is "Table" + table sequence number, and the keyword for the table in the English document is "Tab(le)" + table sequence number. Most of the strings that do not meet the conditions are removed using the keyword. One table keyword corresponds to one table.
[0019] Further, in the step 3, the steps for cleaning using the table keyword features include: for the keywords with overlapping coordinates caused by bold fonts, the duplicate coordinates and duplicate keywords are deleted by comparing the horizontal and vertical coordinates; for the keyword noises that appear in the body part of the body of the journal document, the keywords with larger font sizes are removed by comparing the font sizes.
[0020] Further, in the step 4, the steps for clarifying using the table frame line features include: deleting the lines with a large difference in the angle from the horizontal direction (such as vertical lines) and the too thick lines by comparing the line heights; judging whether the lines are in a spliceable state according to the horizontal and vertical coordinates of the lines, and splicing the split lines into complete lines; judging whether the lines overlap by the horizontal and vertical coordinates, and deleting the covered lines; counting the number of times the lines with the same horizontal coordinate appear, and deleting the lines with the number of times less than 3; counting the number of times the difference in the horizontal coordinates of the left and right endpoints of the lines, and deleting the lines with the number of times less than 3.
[0021] Further, in the step 5, the keyword and the frame line column division method are as follows: if the left horizontal coordinate of the keyword or the frame line is greater than half of the page width, the document is in a two-column layout format, the keyword is divided into the set KeyR, and the frame line is divided into the set LineR; otherwise, the document may be in a single-column layout format or a two-column layout format, the keyword is divided into the set KeyNR, and the frame line is divided into the set LineNR. Therefore, the right-column keyword set KeyR only represents the right-column keywords in the two-column layout, and the non-right-column keyword set KeyNR includes the left-column keywords in the two-column layout, the cross-column keywords in the two-column layout, and all the keywords in the single-column layout; the right-column frame line set LineR only represents the right-column frame lines in the two-column layout, and the non-right-column frame line set LineNR includes the left-column table frame lines in the two-column layout, the cross-column table frame lines in the two-column layout, and all the table frame lines in the single-column layout.
[0022]
[0023] ?
[0024] In equation (1), k i For the i-th keyword, x kiL Let l be the left horizontal coordinate of the k-th keyword, and w be the page width; in equation (2), l j For the j-th frame line, x ljL Let x be the x-coordinate of the left endpoint of the j-th frame line.
[0025] Furthermore, in step 6, the binding of keywords and borders is based on the columns, that is, the right column keyword set KeyR and the right column table border set LineR are combined to form the right column part, and then the keywords and borders are bound according to the vertical axis; the non-right column keyword set KeyNR and the non-right column table border set LineNR are combined to form the non-right column part, and then the keywords and borders are bound according to the vertical axis.
[0026] Furthermore, in step 6, the comparison between the frame line and the keyword's ordinate is as follows: if the frame line's ordinate is between the ordinate of the i-th and (i+1)-th keywords, then the frame line is bound to the i-th keyword; if the frame line's ordinate is less than the ordinates of all keywords, then the frame line is bound to the keyword with the smallest ordinate; if the frame line's ordinate is greater than the ordinates of all keywords, then the frame line is an isolated frame line and is deleted; if there is no frame line between the i-th and (i+1)-th keywords, then the i-th keyword is an isolated keyword and is deleted.
[0027] Furthermore, in step 6, the set Table includes several tables, each of which includes a key and several table borders.
[0028] Furthermore, in step 6, the vertical coordinates of the border lines in the table are compared to obtain the last border line of each table, which is used as the end position of each table.
[0029] Furthermore, in step 7, the coordinates of the top-left corner of the table area are the left horizontal coordinate and the top vertical coordinate. The method for determining the top-left corner coordinate of the table rectangle area based on the keyword coordinates is as follows: if the keyword belongs to the right column keyword set, then the left horizontal coordinate of the table is half the page width; if the keyword belongs to a keyword set other than the right column keyword set, the left horizontal coordinate of the table is 0. The top vertical coordinate of the table is equal to the vertical coordinate of the keyword.
[0030]
[0031] y TiT =y ki (4)
[0032] In equation (3), x TiL Let be the left horizontal coordinate of the i-th table, w be the page width, and k be the coordinate of the left horizontal coordinate of the i-th table. i For the i-th keyword; in equation (4), yTiT Let y be the top ordinate of the i-th table. ki This represents the ordinate of the i-th keyword.
[0033] Furthermore, in step 7, the coordinates of the lower right corner of the table area are the right horizontal coordinate and the lower vertical coordinate. The method for determining the coordinates of the lower right corner of the table area based on the right horizontal and vertical coordinates of the last frame line is as follows: the right horizontal coordinate of the table is the right horizontal coordinate of the last frame line, and the lower vertical coordinate of the table is the lower vertical coordinate of the last frame line.
[0034] x TiR =x LiR (5)
[0035] y TiB =y LiB (6)
[0036] In equation (5), x TiR Let x be the right x-coordinate of the i-th table. LiR Let y be the right x-coordinate of the last frame line of the i-th table; in equation (6), y TiB Let y be the bottom y-coordinate of the i-th table. LiB It represents the ordinate of the bottom edge of the last frame in the i-th table.
[0037] The present invention, by adopting the above-described technical solution, has the following beneficial effects:
[0038] This invention is designed specifically for the table borders and character characteristics of PDF journal documents. It identifies table areas using start keywords and end borders, achieving accurate extraction for both full-border and sparse-border tables, thus automating table extraction. Combined with tools, it enhances the accuracy of table logical relationships and is fully applicable to both Chinese and English documents. The system utilizes common keywords found in journal documents to accurately extract all tables. For journals in different academic fields, selective extraction of relevant tables can be achieved by adding appropriate table content keywords, further refining the selection. This provides broad applicability and diverse selectivity. Attached Figure Description
[0039] Figure 1 This is a flowchart of the method of the present invention.
[0040] Figure 2 This is a schematic diagram illustrating the related attributes of page characters and lines in this invention.
[0041] Figure 3 This is a schematic diagram of the largest and second largest characters in the journal font size according to the present invention.
[0042] Figure 4This is the table keyword set data structure of the present invention.
[0043] Figure 5 This is the table border set data structure of the present invention.
[0044] Figure 6 This is a schematic diagram of the splicable frame lines of the journal table of the present invention.
[0045] Figure 7 This is a schematic diagram of the journal table frame coverage of the present invention.
[0046] Figure 8 This is a schematic diagram of a column-based journal table with the same height according to the present invention.
[0047] Figure 9 This is a schematic diagram of the present invention, which binds table keywords and table borders based on the vertical axis.
[0048] Figure 10 This is the table set data structure of the present invention.
[0049] Figure 11 This is a schematic diagram illustrating the improvement of the left horizontal axis of the table area in this invention.
[0050] Figure 12 This is an example verification diagram of the present invention. Detailed Implementation
[0051] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and preferred embodiments. However, it should be noted that many details listed in the specification are merely to provide the reader with a thorough understanding of one or more aspects of the present invention, and these aspects of the invention can be implemented even without these specific details.
[0052] like Figure 1 As shown, this invention proposes an efficient method for extracting journal literature tables based on keyword and bounding box features. The method includes the following steps:
[0053] Step 1: Use tools such as PDFMiner.six to decode the PDF, obtaining the page width, text characters and lines within each page, and related attributes such as character font size, top-left corner coordinates of characters, line coordinates, and vertical height of lines. The meanings of these attributes are as follows: Figure 2 As shown.
[0054] Step 2: Use ASCII code values to distinguish between Chinese and English documents.
[0055] The ASCII codes for Chinese characters are located in the range of "\u4e00" to "\u9fa5", the ASCII codes for uppercase English letters are all located in [65, 90], and the ASCII codes for lowercase English letters are located in [97, 122]. Because the font size of article titles or important journal information (such as the journal name) in journal articles is usually the largest (e.g., ... Figure 3 As shown in the figure, the characters can be sorted from largest to smallest by font size. The top two characters by font size can be obtained, and the language of the document can be determined by comparing the range of their ASCII codes. If the ASCII codes of the obtained characters are in the range of "\u4e00" to "\u9fa5", the document is in Chinese. If the ASCII codes of the obtained characters are all in the range of [65, 90] or [97, 122], the document is in English.
[0056]
[0057] In equation (1), A f1 The ASCII code value corresponding to the largest character size, A f2 The ASCII code value corresponding to the second largest character in the font size.
[0058] After determining the language of the documents, create a table with "Table" + table number keywords for Chinese documents and a tab with "Tab(le)" + table number keywords for English documents as the identifier for the starting position of the table.
[0059] Step 3: Construct a set of table keywords Key, clean the characters using the characteristics of table keywords, and obtain the table keywords after the first cleaning.
[0060] Construct a key set Key as follows Figure 4 As shown, each keyword includes font size and horizontal and vertical coordinate attributes. For example, the i-th keyword ki includes a font size f. ki The left horizontal coordinate xkiL and the upper vertical coordinate y kiT .
[0061] Some keywords are in bold, causing the same keyword to appear repeatedly (noise characters). Since the coordinates of the noise characters almost overlap with the coordinates of the original characters, deleting the characters with duplicate coordinates will yield unique keywords.
[0062] Journal articles may also contain citations of "Table" + table number or "Tab(le)" + table number keywords (noise characters), which can affect the location of table keywords. In general journal articles, the font size of tables is smaller than that of the main text. Therefore, by comparing the font sizes, we can determine whether the keywords are noise characters and delete them.
[0063] Step 4: Construct a set of table borders called Line, clean the lines using table border features, and obtain the table borders after the initial cleaning.
[0064] Construct a table border set Line as follows Figure 5 As shown, each frame contains coordinate attributes, such as the j-th frame l. j Left x-coordinate ljL The upper ordinate y ljT The right-hand x-coordinate ljR and the lower ordinate y ljB .
[0065] Lines in journal articles include not only table borders but also other lines and even invisible lines. First, most irrelevant lines are filtered out based on the height characteristics of the table's horizontal borders. By obtaining the vertical height of the lines, lines with a vertical height greater than 1.5 are deleted, as are lines with a large angle difference from the horizontal direction (such as vertical and diagonal lines) and excessively thick lines.
[0066] A horizontal border in a table may be split into multiple shorter horizontal lines due to column divisions (e.g., ...). Figure 6 As shown, a horizontal frame line is split into seven lines after parsing. Adjacent split lines are connected end to end, and the complete frame line needs to be restored. Determine whether the right horizontal coordinate of the j-th line is approximately equal to the left horizontal coordinate of the (j+1)-th line, and then piece together the approximately equal lines.
[0067] Due to differences in journal standards, lines may be inserted multiple times at the same table border (e.g. Figure 7 As shown in the figure, after parsing the lines, multiple duplicate frames can be seen. By checking whether the horizontal coordinates of the frames are inclusive, the redundant frames can be removed.
[0068]
[0069] In equation (2), y ljT Let y be the upper ordinate of the j-th line. lhT Let x be the upper ordinate of the h-th line. ljL Let x be the left x-coordinate of the j-th line. lhL Let x be the left x-coordinate of the h-th line. lhR Let x be the right x-coordinate of the h-th line. ljR Let be the right x-coordinate of the j-th line. First, determine whether the two lines are at the same height (with a deviation of 0.5) based on their upper y-coordinates. If they are at the same height, then determine whether the h-th line is contained within the j-th line.
[0070] In journal articles, the number of horizontal borders in a table is usually greater than two. Based on this, we count the number of borders with the same horizontal coordinate and length on both sides, and retain the borders with a number greater than two.
[0071] Step 5: Arrange the keywords and borders into columns according to the layout.
[0072] This step only categorizes the right column in a two-column layout, treating the rest as non-right columns. The reason is that two-column layouts contain tables with overlapping vertical axes (such as...). Figure 8 As shown in the diagram, to avoid confusion in step 6 where the table area is divided by the vertical axis, we consider only the table in the right column to simplify the procedure. Other table formats do not have overlapping vertical axes (in single-column format journal articles, the vertical axes of the tables do not overlap; in two-column format journal articles, the vertical axes of the left column table and the cross-column table do not overlap), so the areas can be divided uniformly.
[0073]
[0074]
[0075] In equation (3), k i For the i-th keyword, x kiL Let l be the left horizontal coordinate of the i-th keyword, w be the page width, KeyR be the set of keywords in the right column, and KeyNR be the set of keywords not in the right column; in equation (4), l j For the j-th frame line, x ljL Let `KeyR` be the left endpoint x-coordinate of the j-th frame line, `LineR` be the set of right-column frame lines, and `LineNR` be the set of non-right-column frame lines. The left x-coordinates of the keywords and the left x-coordinates of the frame lines are compared to half the page width, resulting in a right-column set and a non-right-column set. The right-column keyword set `KeyR` only represents right-column keywords in a two-column layout, while the non-right-column keyword set `KeyNR` includes left-column keywords in a two-column layout, cross-column keywords in a two-column layout, and all keywords in a single-column layout. Similarly, the right-column frame line set `LineR` only represents right-column frame lines in a two-column layout, while the non-right-column frame line set `LineNR` includes left-column table frame lines in a two-column layout, cross-column table frame lines in a two-column layout, and all table frame lines in a single-column layout.
[0076] Step 6: Compare the vertical coordinates of the border lines and keywords, bind the keywords belonging to the same table to the border lines to form a set Table, and obtain the last border line of each table.
[0077] The correct table structure in this method is: a keyword contains several table borders. Based on this, by comparing the y-coordinate of the border with the y-coordinate of the keyword, if the y-coordinate of the border lies between the y-coordinate of the k-th and (k+1)-th keywords, then the border is bound to the k-th keyword, forming a table (e.g., ...). Figure 9 The four borders between Table 1 and Table 2 are bound to Table 1, and the three borders between Table 2 and Table 3 are bound to Table 2. If the y-coordinate of a border is less than the y-coordinate of all keywords, then the border is bound to the keyword with the smallest y-coordinate, forming a table (e.g., ...). Figure 9The three boxes under Table 3 are bound to Table 3; if the vertical coordinate of a box is greater than the vertical coordinate of all keywords, then that box has no keyword binding and should be deleted (e.g., ...). Figure 9 (The lines above Table 1 are not bound to keywords); After the border classification is completed, if there is no border between the k-th keyword and the (k+1)-th keyword, then the k-th keyword has no bound border and is deleted. The bound tables constitute a Table collection, such as... Figure 10 As shown.
[0078] The purpose of obtaining the last border line of each table is to determine the ending position of each table. Based on the comparison of the y-coordinates, the border line with the smallest y-coordinate is the last border line in each table.
[0079] Step 7: Combine the coordinates of the keyword and the last border line to determine the table area. Determine the top-left corner coordinates of the table area based on the top-left and bottom-right corner coordinates of the keyword, and determine the bottom-right corner coordinates of the table area based on the bottom-right corner coordinates of the last border line. The rectangular area of the table is then determined, and the table data is output.
[0080] The left horizontal coordinate of the keyword usually does not represent the left horizontal coordinate of the table. The area formed by combining its top-left corner coordinate with the bottom-right corner coordinate of the last border is as follows: Figure 11 As shown in the dark-bordered area, the left horizontal coordinate of the keyword needs to be extended further to the left. In collections not in the right column, the left horizontal coordinate of the keyword needs to be extended to the far left of the page (e.g., ...). Figure 11 Table 1 in the table shows the location where the horizontal coordinate is 0; in the right column set, the horizontal coordinate to the left of the keyword needs to be extended to half the page width (e.g., ...). Figure 11 Table 2 in the table, specifically the location at the horizontal axis w / 2. The extended table area is... Figure 11 The sum of light and dark areas in the image.
[0081]
[0082] y TiT =y ki (6)
[0083] In equation (5), x TiL Let be the left horizontal coordinate of the i-th table, w be the page width, and k be the coordinate of the left horizontal coordinate of the i-th table. i For the i-th keyword; in equation (6), y TiT Let y be the top ordinate of the i-th table. ki This represents the ordinate of the i-th keyword.
[0084] x TiR =x LiR (7)
[0085] y TiB =y LiB(8)
[0086] In equation (7), x TiR Let x be the right x-coordinate of the i-th table. LiR Let y be the right x-coordinate of the last frame line of the i-th table; in equation (8), y TiB Let y be the bottom y-coordinate of the i-th table. LiB It represents the ordinate of the bottom edge of the last frame in the i-th table.
[0087] After obtaining the left, right, top, and bottom x-coordinates of the table, use the Camelot tool to directly extract the table and save it as a .csv or .xlsx file.
[0088] Example verification such as Figure 12 As shown, (a) is the original Chinese document table, (b) is the extracted Chinese document table, (c) is the original English document table, and (d) is the extracted English document table.
[0089] Matters not covered in this invention are common knowledge.
[0090] This method is specifically designed for tables with few borders in journal articles, and is fully applicable to both Chinese and English literature. It automates table extraction and improves the accuracy of logical relationships within tables. Adding keywords further enables the filtering of tables within relevant fields, making it widely applicable.
[0091] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.
Claims
1. A method for efficient extraction of tables from periodical literature based on keyword and frame features, characterized in that: The method includes the following steps: Step 1: Create character sets and line sets, decode the PDF by page number, and obtain the page dimensions and the character sets for each page. Text Line collection Curve And its information status parameters, including character font size, character horizontal and vertical coordinates, line height, and line horizontal and vertical coordinates; Step 2: Define language classification rules. Divide PDF documents into Chinese documents and English documents, and create tables with keywords for Chinese and English documents respectively; Step 3: Construct a set of table keywords Key By using the characteristics of table keywords to clean characters, the table keywords after the initial cleaning are obtained. Step 4: Construct a set of table borders Line The table border features are used to clean the lines and obtain the table borders after the first cleaning. Step 5: To facilitate the differentiation of table areas, group the keywords according to whether the document is formatted as a two-column layout. Key Divided into right column keyword collection KeyR and non-right column keyword set KeyNR Set the frame lines Line Divided into right column border set LineR and non-right column border set LineNR ; Step 6: Compare the vertical axes of the borders and keywords, and bind the keywords belonging to the same table to the borders to form a set. Table And get the last border of each table; Step 7: Determine the coordinates of the top left corner of the table area based on the horizontal and vertical coordinates of the keywords, and determine the coordinates of the bottom right corner of the table area based on the horizontal and vertical coordinates of the bottom right corner of the last border. The rectangular area of the table is then determined, and the table data is output. In step 5, the column layout for keywords and borders is as follows: if the left horizontal coordinate of a keyword or border is greater than half the page width, the document will be in a two-column format, and the keyword will be assigned to a set. KeyR The border lines divide the set. LineR Otherwise, if the document is in single-column or double-column format, the keywords are divided into sets. KeyNR The border lines divide the set. LineNR Therefore, the right column keyword set KeyR This only refers to the keywords in the right column of a two-column layout, not the entire set of keywords in the right column. KeyNR This includes keywords in the left column of a two-column layout, keywords spanning multiple columns in a two-column layout, and all keywords in a single-column layout, as well as the right column border set. LineR This only refers to the right column border in a two-column layout, not the set of other right column borders. LineNR This includes the left column table border in a two-column layout, the cross-column table border in a two-column layout, and all table borders in a single-column layout; (1) (2) In equation (1), k i For the first i One keyword, x kiL For the first k The left x-coordinate of each keyword, w The width is the page width; in equation (2), l j For the first j Border lines, x ljL For the first j The x-coordinate of the left endpoint of the border line; In step 6, the keyword binding with the border is established based on the column division, specifically the keyword set in the right column. KeyR Collection of table borders in the right column LineR Combine them into the right column section, then bind keywords and borders according to the vertical axis, and include the non-right column keyword set. KeyNR Non-right column table border set LineNR Combine the non-right column sections, and then bind keywords and borders according to the vertical axis; The method for comparing the ordinates of the bounding box and the keywords is as follows: if the ordinate of the bounding box is located at the i-th and (i-th)-th... i If the vertical coordinate of the +1 keyword is between the two, then the box line is related to the first keyword. i For each keyword binding, if the y-coordinate of the bounding box is less than the y-coordinates of all keywords, the bounding box is bound to the keyword with the smallest y-coordinate. If the y-coordinate of the bounding box is greater than the y-coordinates of all keywords, the bounding box is an isolated bounding box and is deleted. i The keyword and the ( i If there is no border between the +1) keywords, then the 1st keyword... i The keyword is an isolated keyword; delete it. gather Table It includes several tables, each containing a keyword and several table borders; By comparing the vertical coordinates of the border lines in the table, we can obtain the last border line of each table and use it as the end position of each table.
2. The efficient method for extracting journal literature tables based on keywords and border features according to claim 1, characterized in that: In step 1, the horizontal and vertical coordinates of the character only represent the horizontal coordinate of the left side and the vertical coordinate of the top side of the character, without considering the coordinates of the right side and the bottom side. The line height is the default vertical height. The horizontal and vertical coordinates of the line include the horizontal coordinate of the left side, the vertical coordinate of the top side, the horizontal coordinate of the right side, and the vertical coordinate of the bottom side of the line.
3. The efficient method for extracting journal literature tables based on keywords and border features according to claim 1, characterized in that: In step 2, the language classification rule is to obtain the characters with the largest and second largest font sizes on the first page of the PDF. If the ASCII values of the characters are both in the range of \u4e00-\u9fa5, then the document is Chinese. If the ASCII values of the characters are both in the range of [65, 90] or [97, 122], then the document is English. Chinese literature table keywords are "table + table number", while English literature table keywords are "Tab" or "Table + table number". Keywords are used to eliminate most strings that do not meet the conditions, and one table keyword corresponds to one table.
4. The efficient method for extracting journal literature tables based on keywords and border features according to claim 1, characterized in that: Step 3, the process of cleaning using table keyword features includes: for keywords whose bolding causes coordinate overlap, duplicate coordinates and duplicate keywords are deleted by comparing the horizontal and vertical coordinates; for keyword noise appearing in the main text of journal articles, keywords with larger font sizes are removed by comparing the font sizes.
5. The efficient method for extracting journal literature tables based on keywords and border features according to claim 1, characterized in that: In step 4, the process of clearly defining the table border features includes: deleting lines with large differences in angle from the horizontal direction and excessively thick lines by comparing line heights; determining whether lines are splicable based on their horizontal and vertical coordinates; splicing the split lines into complete lines; determining whether lines overlap based on their horizontal and vertical coordinates; deleting covered lines; counting the number of times lines with the same horizontal coordinate appear; deleting lines with fewer than 3 occurrences; and counting the number of times the difference between the horizontal coordinates of the left and right endpoints of lines appears; deleting lines with fewer than 3 occurrences.
6. The efficient method for extracting journal literature tables based on keywords and border features according to claim 1, characterized in that: In step 7, the coordinates of the top left corner of the table area are the left horizontal coordinate and the top vertical coordinate. The way to determine the coordinates of the top left corner of the table rectangle area based on the keyword coordinates is as follows: if the keyword belongs to the right column keyword set, the left horizontal coordinate of the table is half the page width; if the keyword belongs to a non-right column keyword set, the left horizontal coordinate of the table is 0, and the top vertical coordinate of the table is equal to the keyword vertical coordinate. (3) (4) In equation (3), x TiL For the first i The left horizontal axis of each table, w For the width of the page, k i For the first i One keyword, in equation (4), y TiT For the first i The top vertical axis of the table, y ki Indicates the first i The vertical axis of each keyword.
7. The efficient method for extracting journal literature tables based on keywords and border features according to claim 1, characterized in that: In step 7, the coordinates of the bottom right corner of the table area are the right-side x-coordinate and the bottom-side y-coordinate. The method for determining the bottom right corner coordinates of the table area based on the x-coordinates and y-coordinates of the bottom right corner of the last border is as follows: the right-side x-coordinate of the table is the right-side x-coordinate of the last border, and the bottom-side y-coordinate of the table is the bottom-side y-coordinate of the last border. (5) (6) In equation (5), x TiR For the first i The right horizontal axis of each table, x LiR For the first i The right horizontal coordinate of the last frame in each table; in equation (6), y TiB For the first i The bottom vertical axis of each table, y LiB For the first i The vertical coordinate below the last frame of the table.
Citation Information
Patent Citations
PDF incomplete box line table extraction method, device and equipment and storage medium
CN112380812A