PDF text extraction method and system capable of maintaining text reading sequence, and device

Abstract

A PDF text extraction method and system capable of maintaining a text reading sequence, and a device. The method comprises: for a PDF text type, performing text extraction to obtain text block coordinate information; on the basis of the text block coordinate information, performing coordinate alignment and calculating coordinates, a row number row_no, a column number col_no, a row processing sequential number row_sequential_number and a column processing sequential number col_sequential_number of each text block; and on the basis of a text recovery type and calculation results, performing text recovery to obtain plain text content.

Description

PDF text extraction method, system and device for maintaining text reading order TECHNICAL FIELD

[0001] The present application relates to the field of computer data processing and text data processing, and particularly relates to a PDF text extraction method, system and device for maintaining text reading order. BACKGROUND

[0002] Recently, the rapid progress of artificial intelligence technologies such as large language models (LLM) represented by ChatGPT has had a profound impact on automatic question answering systems, machine translation, document processing and other fields, further promoting the development of question answering systems and knowledge assistants based on private documents. The question answering system based on private documents mainly uses the powerful natural language understanding and generation capabilities of LLM and other models to realize specific domain question answering and knowledge management functions on private documents such as PDF, Word and table, greatly simplifying document information extraction and knowledge management work.

[0003] PDF documents, as key information carriers in modern digital work and learning, are the main file formats for recording and transmitting information in important fields such as scientific research, academic reports, government documents, etc. In private document knowledge question answering and information extraction systems based on LLM and other artificial intelligence technologies, text understanding systems based on language models usually require pure text format information input and have certain requirements for the original order and semantic integrity of the text.

[0004] However, existing technologies in dealing with PDF documents usually rely on PDF instruction set parsing, OCR and other technologies to directly extract text information from PDF, but these processing methods have certain limitations. The main problems include:

[0005] (1) The extraction method based on PDF instruction parsing can only process original PDF documents. For scanned or photocopied PDF documents, the text content exists in the form of images and cannot be obtained in the PDF instructions, so the text content cannot be extracted;

[0006] (2) Although the extraction method based on OCR can extract text information, there may be some problems in recognizing the text, resulting in incomplete text extraction;

[0007] (3) The above two text extraction methods usually get pure text information, which loses the layout information of the PDF document. Especially when dealing with form-type PDF documents, the loss of this layout information will cause the text order of the entire form document to be disordered, which will further cause the LLM and other document understanding models to misunderstand the document information, resulting in incorrect processing results, which will bring great challenges to the document understanding and processing system based on LLM and other language models.

[0008] Therefore, how to maintain the layout information and text sequence information of the PDF document to the greatest extent while extracting the text from the PDF document to improve the understanding accuracy of the LLM and the like has become an urgent problem to be solved. SUMMARY

[0009] In view of the deficiencies of the prior art, the present application provides a PDF text extraction method and system for maintaining text reading order and equipment, which mainly aims at the layout information loss and text sequence disorder problem of PDF text extraction, and the core is to use the recognition result of PDF instruction analysis and OCR to restore the text sequence of document extraction to the greatest extent through the pre-set extraction method. The present scheme can maintain the integrity of the semantics and sequential arrangement of the extracted text, so that the text understanding and information extraction system based on the language model (such as LLM) can more accurately perform text understanding.

[0010] Specifically, the present application provides the following technical solutions:

[0011] On the one hand, the present application provides a PDF text extraction method for maintaining text reading order, which comprises:

[0012] S1, for the PDF text type, if it is an original PDF text, the BBox information of the PDF text is extracted; if it is a scanned PDF text, text extraction is performed to obtain text information; based on the BBox information and the text information, text block coordinate information is obtained;

[0013] S2, based on the text block coordinate information, coordinate alignment is performed to obtain aligned text block coordinates; the text blocks are traversed, and a vertical layout point positioning list L y and a horizontal layout point positioning list L x are formed; according to the vertical layout point positioning list L y , layout row segmentation is performed, and the row number row_no and the row processing number row_sequential_number of each text block are obtained; based on the horizontal layout point positioning list L x , column processing is performed on the text blocks, and the column number col_no and the column processing number col_sequential_number of each text block are obtained;

[0014] S3, based on the aligned text block coordinates, the row number row_no, the column number col_no, the row processing number row_sequential_number and the column processing number col_sequential_number, based on the text recovery type, text recovery is performed to obtain pure text content.

[0015] Preferably, in S2, the coordinate alignment mode is: converting the text block coordinate information into [x lt ,y lt ,x rb ,y rb ] format; wherein x lt represents the horizontal coordinate of the upper left corner of the text block, y lt represents the vertical coordinate of the upper left corner of the text block, x rb represents the horizontal coordinate of the lower right corner of the text block, and y rb represents the vertical coordinate of the lower right corner of the text block; wherein:

[0016] For the text block coordinate information obtained from the BBox information, each coordinate point is converted as follows: MediaBox_Height=MediaBox[3]-MediaBox[1]

[0017] wherein DPI (Dots Per Inch) represents the number of pixels per inch length of the PDF image, MediaBox represents the physical size of the PDF file, which is composed of 4 elements [0, 0, page_width, page_height], MediaBox[3] represents the value of the 3rd element in MediaBox, and MediaBox[1] represents the value of the 1st element in MediaBox; x new and y new represent the converted horizontal and vertical coordinates, and x old and y old represent the original horizontal and vertical coordinates.

[0018] For the text block coordinate information obtained from the text information, the x lt ,y lt ,x rb ,y rb values of the text block are directly extracted.

[0019] Preferably, the vertical layout point positioning list L y is obtained by traversing all text blocks and obtaining the vertical coordinates of the upper left corners to form the vertical layout point positioning list L y .

[0020] The horizontal layout point positioning list L x is obtained by traversing all text blocks and obtaining the horizontal coordinates of the upper left corners to form the horizontal layout point positioning list L y .

[0021] Preferably, the row number row_no and the row processing number row_sequential_number are set by traversing the vertical layout point positioning list Ly traversing the list of text blocks, calculating the height block_height of each text block; for the current text block upper left coordinate x i , obtaining the corresponding text block list, sorting the text blocks according to the height block_height, setting the value of row processing number row_sequential_number according to the sorting, wherein the text blocks with the same height block_height are set with the same row processing number row_sequential_number;

[0022] ascending order according to the row processing number row_sequential_number of the text blocks and the upper left coordinate x lt of the text blocks, and descending order according to the value of the height block_height; setting the row number row_no of each text block according to the sorted order.

[0023] Preferably, the setting mode of the column number col_no and the column processing number col_sequential_number is: traversing the horizontal layout point positioning list L x , calculating the width block_width of each text block;

[0024] for the current text block upper left coordinate x j , obtaining the corresponding text block list, sorting the text blocks according to the width block_width, setting the value of column processing number col_sequential_number according to the sorting, wherein the text blocks with the same width block_width are set with the same column processing number col_sequential_number;

[0025] ascending order according to the column processing number col_sequential_number of the text blocks and the upper left coordinate y lt of the text blocks, and descending order according to the value of the width block_width; setting the column number col_no of each text block according to the sorted order.

[0026] Preferably, in S3, the text recovery type includes: column type, form type and default type;

[0027] If the text recovery type is not specified, the recovery is performed according to the default type.

[0028] Preferably, the recovery according to the default type is:

[0029] The sum of row number row_no and column number col_no of all text blocks is calculated, sorted according to the sum, and the text is recovered according to the sorted order;

[0030] If the sum of row number row_no and column number col_no of multiple text blocks is equal, the difference between row number row_no and column number col_no is calculated, sorted according to the difference, and the text is recovered according to the sorted order;

[0031] If the difference between row number row_no and column number col_no of multiple text blocks is still equal, the text block with smaller row number row_no is arranged in front.

[0032] Preferably, if it is a column type, the text recovery method is:

[0033] The number of columns and the width ratio of each column are determined;

[0034] The left boundary line coordinates of each column are calculated based on the number of columns, width ratio and page width;

[0035] For the left boundary line of the first column, the coordinates of each point in the horizontal layout point positioning list L x are counted, and the smallest mode is taken as the left boundary line of the first column;

[0036] According to the left boundary line of each column, the text blocks with the same boundary line and similar width in the text block are found, spliced one by one according to the sorting order of row_no, and spliced according to the sorting order of col_no in different columns.

[0037] Preferably, if it is a form type, the text recovery method is:

[0038] If it is layout row first, the text blocks are spliced one by one according to the sorting order of row_no, and for different text blocks in the same layout row, they are spliced according to the sorting order of col_no; when two text blocks are adjacent and the text box spacing is less than a preset spacing threshold, text block merging is performed, otherwise no merging is performed;

[0039] If it is layout column first, the text blocks are spliced one by one according to the sorting order of col_no; for different text blocks in the same layout column, they are spliced according to the sorting order of row_no: if the row_no of two text blocks is continuous, and the left boundary line of the text block is equal or the right boundary line is equal, merging is performed, otherwise, no merging is performed.

[0040] Preferably, in the layout row first, the calculation method of the spacing threshold is:

[0041] In another aspect, the present application also provides a PDF text extraction system for maintaining text reading order, comprising:

[0042] a text extraction module, for PDF text type, if it is original PDF text, extracting BBox information of the PDF text; if it is scanned PDF text, performing text extraction to obtain text information; based on the BBox information and the text information, obtaining text block coordinate information;

[0043] a text processing module, based on the text block coordinate information, performing coordinate alignment to obtain aligned text block coordinates; traversing the text blocks and forming a vertical layout point positioning list L y and a horizontal layout point positioning list L x ; performing layout row segmentation according to the vertical layout point positioning list L y and obtaining row number row_no and row processing number row_sequential_number of each text block; based on the horizontal layout point positioning list L x , performing column processing on the text blocks and obtaining column number col_no and column processing number col_sequential_number of each text block;

[0044] a text recovery module, based on the aligned text block coordinates, the row number row_no, the column number col_no, the row processing number row_sequential_number and the column processing number col_sequential_number, based on text recovery type, performing text recovery to obtain pure text content.

[0045] In another aspect, the present application also provides a PDF text extraction device for maintaining text reading order, comprising a memory and a processor, wherein the processor invokes computer instructions stored in the memory to execute the PDF text extraction method for maintaining text reading order as described above.

[0046] Compared with the prior art, the present application has at least the following beneficial effects:

[0047] Compared with directly extracting text from native PDF instructions or using OCR to extract text from PDF images, the text extraction method proposed by the present application can maintain the layout information of the PDF to a certain extent, and further recover the text reading order of the PDF;

[0048] Based on the text extraction method proposed by the present application, the reading order of the text can be maintained to the greatest extent, and the accuracy of downstream application systems such as document question and answer based on LLM can be improved;

[0049] Compared with other more complex text reading order recovery models, the method proposed in the scheme is more concise, not only reduces the complexity of operation, but also processes different types of document processing requirements by using configurable rules, so that the scheme has high scalability. BRIEF DESCRIPTION OF DRAWINGS

[0050] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and those skilled in the art can obtain other drawings according to these drawings without creative labor.

[0051] Fig. 1 is a schematic diagram of the scheme framework of the embodiment of the present application;

[0052] Fig. 2 is a schematic diagram of the PDF file layout of the embodiment of the present application;

[0053] Fig. 3 is a schematic diagram of the method flow of the embodiment of the present application;

[0054] Fig. 4 is a schematic diagram of the text recovery mode of the embodiment of the present application;

[0055] Fig. 5 is a schematic diagram of the system architecture of the embodiment of the present application. DETAILED DESCRIPTION

[0056] The technical solutions in the embodiments of the present application will be described clearly and completely below in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0057] The scheme mainly aims at the problems of layout information loss and text order disorder in PDF text extraction. The core of the proposed PDF document text order recovery scheme is to use PDF instruction analysis and OCR recognition results to maximize the recovery of the text order of the extracted document. Through this technology, we can maintain the integrity of the semantic and order arrangement of the extracted text, so that the text understanding and information extraction system based on language model (such as LLM) can more accurately understand the text.

[0058] Specifically, we first use OCR technology and PDF instruction parsing technology to extract text from the PDF document, and then according to the text extraction result, the original order of the text is restored through the document layout recovery method and specific processing algorithm proposed in the scheme. The scheme not only improves the accuracy of text understanding, but also can be widely used in various scenarios that require text extraction and understanding.

[0059] The four parts of the architecture of the present scheme will be described in detail below in combination with the actual embodiments of FIG. 1, FIG. 3, and FIG. 2. The main processing process of the present scheme is divided into three modules: a text extraction module, a text processing module, and a text recovery module, and the detailed flow is shown in FIG. 3.

[0060] 1. Text extraction module

[0061] The text extraction part mainly extracts text from the PDF file to be processed, and the way to extract text will be determined according to the type of the PDF to be processed. The main processing flow is as follows:

[0062] 1.1. Determine the type of the input PDF text, whether it is a scanned or original PDF;

[0063] 1.2. If the PDF is an original PDF, call the PDF instruction parsing interface to extract the text and text BBox (Bounding Box, i.e. bounding box) information in the PDF. If the PDF is a scanned one, skip this step and go to step 1.3. Here, whether it is an original PDF document can be determined using existing tools, which will not be described here;

[0064] 1.3. Perform OCR text extraction on the PDF to obtain text information and text BBox information. In this embodiment, the specific extraction method of OCR can use the mature algorithm in the existing technology, which will not be described here.

[0065] For example, as shown in FIG. 2, a simulated PDF layout example is shown. For a native PDF file, the internal text can be extracted by instructions and the corresponding BBox information is given. For a photocopy or scanned PDF document, text blocks cannot be extracted by instructions, so OCR technology is mainly used for extraction.

[0066] Assuming that the original PDF document is as shown in FIG. 2, because different systems or methods process the information of BBox differently, it is assumed that the extracted coordinate information is described in the form of [x_bottomleft, y_bottomleft, x_topright, y_topright], that is, each block A, B, …, G in the file is represented by the left-bottom point coordinate and the right-top point coordinate. The above coordinate description is as follows: A: [92, 791, 296, 809] B: [92, 781, 296, 790] C: [324, 802, 528, 809] D: [324, 791, 396, 798] E: [420, 791, 528, 798] F: [324, 781, 396, 790] G: [420, 781, 528, 790]

[0067] Assuming that it is a scanned PDF document, because different OCR systems have different description methods, here it is described in the form of [[x_topleft, y_topleft], [x_topright, y_topright], [x_bottomright, y_bottomright], [x_bottomleft, y_bottomleft]]. For a scanned PDF document, the representation of each file block is as follows: A: [[383, 135], [1236, 135], [1236, 210], [383, 210]] B: [[383, 215], [1236, 215], [1236, 250], [383, 250]] C: [[1350, 135], [2200, 135], [2200, 165], [1350, 165]] D: [[1350, 180], [1650, 180], [1650, 210], [1350, 210]] E: [[1750, 180], [2200, 180], [2200, 210], [1750, 210]] F: [[1350, 215], [1650, 215], [1650, 215], [350, 215]] G: [[1750, 215], [2200, 215], [2200, 250], [1750, 250]]

[0068] At this point, the extraction of each text in the file is completed, and the corresponding coordinate information is retained.

[0069] 2. Processing module

[0070] This module mainly performs alignment processing on the extracted text information, merges the text extracted by OCR and the text obtained by parsing the PDF instructions, and outputs it in a unified format. Its main functions include:

[0071] (1) Coordinate alignment function, that is, converting the acquired text BBox coordinates into a coordinate system with the upper left corner as the origin. Each recognized text block is converted into [x lt ,y lt ,x rb ,y rb ] format, where [x lt ,y lt ] represents the coordinates of the upper left corner of the text BBox, [x rb ,y rb ] represents the coordinates of the lower right corner of the text BBox;

[0072] (2) Layout scanning function, that is, scanning text blocks from the upper left corner of the document in a certain order. For example, the text blocks can be scanned from left to right and from top to bottom to determine the layout information of the original document. The layout information definition is shown in Table 1.

[0073] Table 1 Layout information definition

[0074] 2.1 The coordinates need to be aligned.

[0075] The obtained text BBox coordinates are converted to a coordinate system with the upper left corner as the origin. Each recognized text block is converted to [x lt ,y lt ,x rb ,y rb ]Format:

[0076] For the OCR recognition result, the upper left corner is used as the coordinate origin, so the main conversion is to extract and convert [[x_topleft,y_topleft],[x_topright,y_topright],[x_bottomright,y_bottomright],[x_bottomleft,y_bottomleft]] into the above format;

[0077] For the BBox coordinates obtained from native PDF, each coordinate point needs to be converted according to the following formula: MediaBox_Height=MediaBox[3]-MediaBox[1]

[0078] Among them, DPI is usually set to 300, and MediaBox is the metadata of PDF, which defines the physical size of PDF. It consists of 4 elements [0,0,page_width,page_height] and can be directly obtained from PDF instructions. newx new x old y old x

[0079] As the PDF native coordinates above, according to the calculation results above, taking DPI = 300, MediaBox = [0, 0, 594.75, 841.5], the following conversion results can be obtained, with 1 decimal place for the calculation results: A: [383.3, 210.4, 1233.3, 135.4] B: [383.3, 252.1, 1233.3, 214.6] C: [1350.0, 164.6, 2200.0, 135.4] D: [1350.0, 210.4, 1650.0, 181.3] E: [1750.0, 210.4, 2200.0, 181.3] F: [1350.0, 252.1, 1650.0, 214.6] G: [1750.0, 252.1, 2200.0, 214.6]

[0080] According to the calculation method described in 2.1, the final conversion results are: A: [383.3, 135.4, 1233.3, 210.4] B: [383.3, 214.6, 1233.3, 252.1] C: [1350.0, 135.4, 2200.0, 164.6] D: [1350.0, 181.3, 1650.0, 210.4] E: [1750.0, 181.3, 2200.0, 210.4] F: [1350.0, 214.6, 1650.0, 252.1] G: [1750.0, 214.6, 2200.0, 252.1]

[0081] 2.2, first traverse all the text blocks and the corresponding BBox, get the vertical axis value of the upper left corner coordinate of each BBox, get the vertical layout point positioning list L y ; get the horizontal axis value of the upper left corner coordinate of each BBox, get the horizontal layout point positioning list L x ; exemplary, both lists are arranged in ascending order.

[0082] Next to the above embodiment, referring to FIG. 2, we can get: L y = [135.4, 181.3, 214.6] L x = [383.3, 1350.0, 1750.0].

[0083] 2.3, Layout line segmentation determination. Traverse the line, determine which text blocks to be included in a line, here need to calculate the height of each text block, that is the font size; judgment is divided into several cases: (1) the same coordinate value, the same font height, divided into a line; (2) the same coordinate value, different font height, set a certain space, as a line or as two lines; (3) the coordinates of two adjacent blocks are different, calculate the difference, if the tightness is high, it is divided into a line, otherwise it is divided into two lines; The tightness can be determined by comparing the distance and density between the upper and lower text lines. Combined with the above example, the specific calculation process of this step is as follows:

[0084] 2.3.1, according to L y =[y0,y1,y2,…,y m ]list traversal, according to the following rules for calculation:

[0085] 2.3.1.1, for each coordinate point in the list, find its corresponding text block, then calculate the height of each text block block_height, get the corresponding text height list, such as coordinate point y0 represents the uppermost coordinate point, corresponding to 2 text blocks, calculate the height of two text blocks;

[0086] Continue the above example:

[0087] For the results in the list L y =[135.4,181.3,214.6], such as y=135.4, the corresponding block is:

[0088] A: [383.3, 135.4, 1233.3, 210.4], according to the formula block_height=y rb -y lt Calculate block_height=210.4-135.4=75

[0089] C: [1350.0, 135.4, 2200.0, 164.6], block_height=164.6-135.4=29.2

[0090] For y=181.3, the corresponding block is calculated according to the similar formula as follows:

[0091] D: [1350.0, 181.3, 1650.0, 210.4], block_height=29.1

[0092] E: [1750.0, 181.3, 2200.0, 210.4], block_height=29.1

[0093] For y = 214.6, the corresponding blocks are:

[0094] B: [383.3, 214.6, 1233.3, 252.1], block_height = 37.5

[0095] F: [1350.0, 214.6, 1650.0, 252.1], block_height = 37.5

[0096] G: [1750.0, 214.6, 2200.0, 252.1], block_height = 37.5

[0097] 2.3.2, continue to L y = [y0, y1, y2,..., y m ] list, and calculate according to the following rules:

[0098] 2.3.2.1, for the current coordinate point y i , there is a corresponding text block list [textb0, textb1, textb2,..., textb b ], where textb0 represents the first text block corresponding to the current coordinate point y i , textb b represents the b+1th text block corresponding to the current coordinate point y i , and the rest is similar; then, according to the text height, the layout row is divided, the text blocks are sorted according to the height of the text blocks, and the division rules are as follows:

[0099] The text blocks are processed in order of small to large according to the text height, and a mark row_sequential_number is set for each text block;

[0100] The row_sequential_number mark is incremented from 0 in the global scope of the page;

[0101] Text blocks with the same text height are set with the same processing sequence number row_sequential_number;

[0102] Alternatively, in order to maintain the stability of the text box positioning error processing, a global text box can be used to count the text height, and a error value Δ can be calculated based on the statistical information, within the error value range, the text blocks can be set to the same processing sequence number row_sequential_number;

[0103] Continue with the above example:

[0104] For y = 135.4, the corresponding text block list is [A, C], the two text blocks have different heights, A is greater than C, then the row_sequential_number of A block is numbered as 0, and the row_sequential_number of C block is numbered as 1;

[0105] Then, for y = 181.3, numbered from 2, the corresponding text block list is [D, E], because the text heights are the same, the same row_sequential_number is set, which is set as 2;

[0106] For y = 214.6, numbered from 3, the corresponding text block list is [B, F, G], and the text block heights are the same, so the row_sequential_number is set as 3.

[0107] 2.3.3, After the above calculation, for each text block, we get the following information {x lt ,y lt ,row_sequential_number,block_height}, then all text blocks are sequentially numbered row_no from 1 according to the following rules:

[0108] According to the ascending order of row_sequential_number, x lt value;

[0109] According to the descending order of block_height value;

[0110] Continue the above example:

[0111] After the calculation, the information of all text blocks is as follows: A: {383.3, 135.4, 0, 75} B: {383.3, 214.6, 3, 37.5} C: {1350.0, 135.4, 1, 29.2} D: {1350.0, 181.3, 2, 29.1} E: {1750.0, 181.3, 2, 29.1} F: {1350.0, 214.6, 3, 37.5} G: {1750.0, 214.6, 3, 37.5}

[0112] Next, sort according to the above example rules, and give each text block row_no sequential numbering, the results are as follows: A: {383.3, 135.4, 0, 75}, row_no = 1; C: {1350.0, 135.4, 1, 29.2}, row_no = 2; D: {1350.0, 181.3, 2, 29.1}, row_no = 3; E: {1750.0, 181.3, 2, 29.1}, row_no = 4; B: {383.3, 214.6, 3, 37.5}, row_no = 5; F: {1350.0, 214.6, 3, 37.5}, row_no = 6; G: {1750.0, 214.6, 3, 37.5}, row_no = 7;

[0113] 2.4, all text blocks are processed in column direction, the specific process is as follows:

[0114] 2.4.1, according to the list L x = [x0, x1, x2,..., x m ] traversal, according to the following rules for processing:

[0115] 2.4.1.1, for each coordinate point in the list, find its corresponding text block, and then calculate the width of each text block block_width, get the corresponding text width list, such as coordinate point x0 represents the rightmost coordinate point, corresponding to 2 text blocks, calculate the width of two text blocks;

[0116] Continue the above example:

[0117] For the results in the list L x = [383.3, 1350.0, 1750.0], such as x = 383.3, its corresponding block is:

[0118] A: [383.3, 135.4, 1233.3, 210.4], the text block width calculation method is block_width = x rb -x lt , block_width = 1233.3-383.3 = 850;

[0119] B: [383.3, 214.6, 1233.3, 252.1], text block width block_width = 1233.3-383.3 = 850;

[0120] For x = 1350.0, according to the similar calculation method, its corresponding text block is:

[0121] C: [1350.0, 135.4, 2200.0, 164.6], text block width block_width = 850;

[0122] D: [1350.0, 181.3, 1650.0, 210.4], text block width block_width = 300;

[0123] F: [1350.0, 214.6, 1650.0, 252.1], text block width block_width = 300;

[0124] For x = 1750.0, the corresponding text block is:

[0125] E: [1750.0, 181.3, 2200.0, 210.4], text block width block_width = 450

[0126] G: [1750.0, 214.6, 2200.0, 252.1], text block width block_width = 450

[0127] 2.4.2, continue to L x = [x0, x1, x2,..., x t ] list traversal, according to the following rules for calculation:

[0128] 2.4.2.1, for the current coordinate point x j , corresponds to a text block list [textb0, textb1, textb2,..., textb c ], here textb0 represents the first text block corresponding to the current coordinate point x j , textb c represents the c+1th text block corresponding to the current coordinate point x j , c+1 is the total number of all text blocks corresponding to the current coordinate point x j , the rest is similar, according to the text width to layout line division, according to the width of the text block, these text blocks are sorted, the division rule is as follows:

[0129] According to the order from small to large, the text width of these text blocks is processed, and a mark col_sequential_number is set for each text block; col_sequential_number mark starts from 0 and increases in the global scope of the page;

[0130] The text blocks with the same text width are set with the same processing serial number col_sequential_number;

[0131] Alternatively, in order to keep the stability of the text box positioning error processing, the text width statistics can be performed based on the global text box, and an error value Δ can be calculated based on the statistical information, within the error value range, the text block can be set to the same processing serial number col_sequential_number;

[0132] Continue with the above example:

[0133] For x = 383.3, the corresponding text block list is [A, B], the width is equal, and the col_sequential_number number is set to 0;

[0134] For x = 1350.0, the corresponding text block list is [C, D, F], the width is different, and the col_sequential_number number is set to 2, 1, and 1, respectively;

[0135] For x = 1750.0, the corresponding text block list is [E, G], the width is the same, and the col_sequential_number number is set to 3;

[0136] 2.4.3, after the above calculation, for each text block, we get the following information {x lt ,y lt ,col_sequential_number,block_width}, then all the regions are sequentially numbered col_no according to the following rules:

[0137] Exemplarily, according to the ascending order of col_sequential_number, y lt value; according to the descending order of block_width value;

[0138] Continue with the above example, after the calculation, the information of all text blocks is as follows: A: {383.3, 135.4, 0, 850} B: {383.3, 214.6, 0, 850} C: {1350.0, 135.4, 2, 850} D: {1350.0, 181.3, 1, 300} E: {1750.0, 181.3, 3, 450} F: {1350.0, 214.6, 1, 300} G: {1750.0, 214.6, 3, 450}

[0139] After sorting and numbering according to the rules, the result is as follows:

[0140] A: {383.3, 135.4, 0, 850}, col_no number = 1;

[0141] B: {383.3,214.6,0,850}, col_no = 2;

[0142] D: {1350.0,181.3,1,300}, col_no = 3;

[0143] F: {1350.0,214.6,1,300}, col_no = 4;

[0144] C: {1350.0,135.4,2,850}, col_no = 5;

[0145] E: {1750.0,181.3,3,450}, col_no = 6;

[0146] G: {1750.0,214.6,3,450}, col_no = 7;

[0147] 2.5, By the above two steps, each text block has information as shown in Table 2.

[0148] Table 2 Text block information

[0149] 3, a text recovery module, which mainly according to the layout information obtained previously, according to the specific rules of text processing, and then output can ensure a certain layout information and reading order of pure text content, as shown in Figure 4, the main function and processing mode as follows:

[0150] (1) text recovery function, that is, according to the pre-defined rules, the text with layout markers are restored to pure text reading order; exemplary, the main processing flow of the recovery process is as follows:

[0151] 3.1, rule acquisition, that is, from the rule base to obtain the specific processing method and rule of text processing for each layout line, usually different rules can adapt to different document types:

[0152] such as for the column document, its processing rule is preferably as follows:

[0153] set the number of columns column_num, and the approximate width ratio of each column;

[0154] According to the setting, calculate the left boundary line coordinate x_left of each column;

[0155] Exemplarily, taking the equal two-column as an example, the left boundary line of the second column is calculated, that is, For the calculation of the first column, all L xThe minimum mode of each coordinate point of the list is taken as x_left1, i.e. the mode coordinate point is calculated, and if there are multiple coordinate points with little difference, the minimum coordinate point is taken as the boundary line;

[0156] For example, for a form document, the processing rule is preferably as follows:

[0157] Whether the layout row is merged first or the layout column is merged first is set;

[0158] For the layout row, when the text sequence is restored, the layout row is merged first to determine which text blocks in the same layout row can be spliced and output as a row;

[0159] For the layout column, the layout column is merged first to determine which different text blocks can be spliced and output across rows.

[0160] Exemplarily, if there is no special document type specified, or a specific processing algorithm and rule is selected, the default processing mode is adopted.

[0161] 3.2. The default processing mode is as follows:

[0162] The row_no+col_no value of all text blocks is calculated, and the order from small to large is sorted, and if the row_no+col_no value is the same, the next step is entered;

[0163] The difference between row_no and col_no is calculated, and the smaller difference is arranged in front, and if the difference is the same, the smaller row_no is arranged in front;

[0164] When outputting, different text blocks occupy different rows, i.e. the output of the next text block is a line break and occupies a row;

[0165] 3.3. If the column document is specified and the specific column rule is adopted, the predetermined column boundary line is used to find the text blocks with the same boundary line and similar width in the text blocks, and the row_no is sorted from small to large for splicing one by one. According to the size of col_no, from small to large, different columns are spliced;

[0166] 3.4. If the form document is specified and the specific processing rule is adopted, the following processing is performed:

[0167] If the layout row is preferred, the layout row row_no is sorted in order, and the text blocks are spliced one by one; for different text blocks in the same layout row, the col_no is spliced in order:

[0168] If two text blocks are adjacent, i.e. the distance between two text blocks is less than a certain threshold, then perform text block concatenation output as a line (i.e. concatenate directly after adding 4 spaces at the end of the previous text block); otherwise output as different lines. The threshold here can be pre-set or set according to the density of the document text:

[0169] One method is to calculate the distance of all text blocks, and select the 50% quantile of the numerical value as the threshold;

[0170] Or, for a layout row with more than 2 text blocks, calculate the average interval distance in the layout row as the threshold, and the calculation method is:

[0171] Or, for a layout row with 2 text blocks, the threshold is 1 / 3 of the layout row width;

[0172] Otherwise, do not perform concatenation output, and output as different lines in the order of row_no;

[0173] If it is layout column first, then according to the order of layout column col_no, decide which text blocks can be concatenated and output; for text blocks in the same layout column, judge according to the order of row_no:

[0174] If the row_no of two text blocks is continuous, and the left boundary line of the text block is equal or the right boundary line is equal, then perform concatenation output as a line (i.e. concatenate directly after adding 4 spaces at the end of the previous text block); otherwise output as different lines;

[0175] Otherwise, do not perform concatenation, and concatenate output as different lines in the order of col_no.

[0176] 3.5, output pure text content;

[0177] After the above text judgment and rule judgment, perform text concatenation and merging to complete the output of the text content. Continue with the above example, get the numbering information of each text box as follows:

[0178] A: row_no = 1, col_no = 1, sum = 2, difference = 0;

[0179] C: row_no = 2, col_no = 5, sum = 7, difference = 3;

[0180] D: row_no = 3, col_no = 3, sum = 6, difference = 0;

[0181] E: row_no = 4, col_no = 6, sum = 10, difference = 2;

[0182] B: row_no=5, col_no=2, sum=7, difference=3;

[0183] F: row_no=6, col_no=4, sum=10, difference=2;

[0184] G: row_no=7, col_no=7, sum=14, difference=0;

[0185] Then, based on the above judgment and splicing steps, the order is [A-D-C-B-E-F-G], and different text blocks occupy different rows. The final text output effect is as follows: A D C B E F G

[0186] Take the processing rule of a specified form class as an example:

[0187] If the layout is row first, the splicing order is A-C-D-E-B-F-G; here D and E have the same row_sequential_number and belong to the same layout row. Suppose the threshold is 110, the two text blocks are actually adjacent and the distance is 1750-1650=100, so E text is spliced after D text block using 4 spaces. Then B, F and G are in the same layout row, B is adjacent to F, but the distance is 1350-1233.3=116.7, which is greater than the preset threshold 110, so B is output separately, and F is output after B; F and G belong to the same layout row and are adjacent with a distance of 1750-1650=100, which is less than the preset threshold, so F and G are spliced using spaces and output as a row. Therefore, the final output layout is as follows: A C D E B F G

[0188] If the layout is column first, the splicing order is A-B-D-F-C-E-G; here A and B are in the same layout column, but the row_no is not continuous, so A is output first and B is not spliced; according to the col_no order, B is processed, B is not in the same layout column as the remaining blocks, so B is output on a new line; then D is processed according to the col_no order, and F is in the same layout column as D, but the row_no is not continuous, so D and F are output on two different lines, i.e. D is output on a new line and F is output on a new line; C is processed according to the order, and C is not in the same layout column as any block, so C is output on a new line; E is processed according to the order, and G is in the same layout column as E, but the row_no is not continuous, so E and G are output on a new line. The final text output effect should be as follows: A B D F C E G

[0189] In yet another embodiment, the scheme of the present application can also be implemented in a system manner, as shown in FIG. 5, which includes:

[0190] The text extraction module, for the PDF text type, if it is an original PDF text, extracts the BBox information of the PDF text; if it is a scanned PDF text, performs text extraction to obtain text information; based on the BBox information and the text information, obtains text block coordinate information;

[0191] The text processing module, based on the text block coordinate information, performs coordinate alignment to obtain aligned text block coordinates; traverses the text blocks and forms a vertical layout point positioning list L y and a horizontal layout point positioning list L x ; according to the vertical layout point positioning list L y performs layout row segmentation and obtains row numbers row_no and row processing numbers row_sequential_number of each text block; based on the horizontal layout point positioning list L x performs column processing on the text blocks and obtains column numbers col_no and column processing numbers col_sequential_number of each text block;

[0192] The text recovery module, based on the aligned text block coordinates, the row numbers row_no, the column numbers col_no, the row processing numbers row_sequential_number and the column processing numbers col_sequential_number, performs text recovery based on a text recovery type to obtain pure text content.

[0193] In addition, the system can also include a text data acquisition module for acquiring an original PDF file or a scanned PDF file.

[0194] In addition, the system can also include an output module for directly displaying the recovered text content or connecting with a personal digital terminal (mobile phone, digital tablet, notebook computer, etc.) and transmitting the recovered text to the personal digital terminal for display to the user.

[0195] In addition, the present application can also be realized by an electronic device, and a processor in the electronic device calls computer instructions to execute the steps of each method in the above-mentioned embodiments of the present application.

[0196] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by a computer program instructing related hardware, and the program can be stored in a computer readable storage medium, and when the program is executed, it can include the processes of the above-mentioned embodiments of each method. The storage medium can be a magnetic disc, an optical disc, a read-only memory (ROM) or a random access memory (RAM), etc.

[0197] It should be pointed out finally that the above embodiments are only used to illustrate the technical solutions of the present application, but not to limit the same; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can still be modified, or some technical features therein can be replaced equivalently; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A PDF text extraction method that preserves the reading order of text, characterized by, The method comprises: S1, for a PDF text type, if it is an original PDF text, BBox information of the PDF text is extracted; if it is a scanned PDF text, text extraction is performed to obtain text information; based on the BBox information and the text information, text block coordinate information is obtained; S2, based on the text block coordinate information, performing coordinate alignment to obtain aligned text block coordinates; traversing the text blocks and forming a vertical layout point positioning list L y and a horizontal layout point positioning list L x ; according to the vertical layout point positioning list L y , performing layout row segmentation to obtain row numbers row_no and row processing numbers row_sequential_number of each text block; based on the horizontal layout point positioning list L x , performing column processing on the text blocks to obtain column numbers col_no and column processing numbers col_sequential_number of each text block; S3, based on the aligned text block coordinates, row numbers row_no, column numbers col_no, row processing numbers row_sequential_number and column processing numbers col_sequential_number, based on a text recovery type, text recovery is performed to obtain pure text content.

2. The method of claim 1, wherein, In the S2, the coordinate alignment mode is: converting the text block coordinate information into [x lt ,y lt ,x rb ,y rb ] format; wherein, x lt represents the horizontal coordinate of the upper left corner of the text block, y lt represents the vertical coordinate of the upper left corner of the text block, x rb represents the horizontal coordinate of the lower right corner of the text block, and y rb represents the vertical coordinate of the lower right corner of the text block; wherein: For the text block coordinate information obtained from the BBox information, each coordinate point is converted as follows: MediaBox_Height = MediaBox[3] - MediaBox[1] wherein DPI represents the number of pixels per inch of length of the PDF image, MediaBox represents the physical size of the PDF file, which is composed of 4 elements [0, 0, page_width, page_height], MediaBox[3] represents the value of the 3rd element in the MediaBox, and MediaBox[1] represents the value of the 1st element in the MediaBox;x new , y new represent the converted horizontal and vertical coordinates, and x old , y old represent the horizontal and vertical coordinates before conversion. For the text block coordinate information obtained by the text information, the x lt ,y lt ,x rb ,y rb value of the text block is directly extracted.

3. The method of claim 1, wherein, The row number row_no and the row processing number row_sequential_number are set in the following way: according to the vertical layout point positioning list L y The traversal is performed, and the height block_height of each text block is calculated; for the current text block upper left corner vertical coordinate y i , the corresponding text block list is obtained, the text blocks are sorted according to the height block_height, and the value of the row processing number row_sequential_number is set according to the sorting, wherein the text blocks with the same height block_height are set with the same row processing number row_sequential_number; According to the row processing number row_sequential_number of the text block and the left upper corner horizontal coordinate x of the text block lt The row numbers row_no of the text blocks are set in the order of ascending arrangement and descending arrangement according to the values of the heights block_height.

4. The method of claim 1, wherein, The column number col_no and the column processing number col_sequential_number are set in the following manner: according to the horizontal layout point positioning list L x A traversal is performed to calculate the width block_width of each text block. For the current text block upper left corner horizontal coordinate x j , get the corresponding text block list, sort the text blocks according to the width block_width, set the value of the row processing number col_sequential_number according to the sorting, wherein the text blocks with the same width block_width set the same row processing number col_sequential_number; According to the line processing number col_sequential_number and the top-left vertical coordinate y of the text block lt The column numbers col_no of the text blocks are set in the order of ascending and descending values of the width block_width.

5. The method of claim 1, wherein, In the S3, the text recovery type comprises a column type, a form type and a default type; If the text recovery type is not specified, recovery is performed according to the default type.

6. The method of claim 5, wherein, The recovery mode according to the default type is: The sum of the row numbers row_no and the column numbers col_no of all text blocks is calculated, the sum is sorted, and the text is recovered according to the sorting order; If the sum of the row numbers row_no and the column numbers col_no of multiple text blocks is equal, the difference between the row numbers row_no and the column numbers col_no is calculated, the difference is sorted, and the text is recovered according to the sorted order; If the difference between the row numbers row_no and the column numbers col_no of multiple text blocks is still equal, the text block with a smaller row number row_no is arranged in front.

7. The method of claim 5, wherein, If it is the column type, the text recovery mode is: The number of columns and the width ratio of each column are determined; The left boundary line coordinates of each column are calculated based on the number of columns, the width ratio and the page width; Wherein, for the left boundary line of the first column, statistics are made on each coordinate point of the transverse layout point positioning list L x , and the minimum mode among them is taken as the left boundary line of the first column. According to the left boundary line of each column, text blocks with the same boundary line and similar width are found in the text block, and are spliced according to the sorting order of row_no and the sorting order of col_no.

8. The method of claim 5, wherein, If it is the form type, the text recovery mode is: If it is layout row priority, text blocks are spliced according to the sorting order of row_no, and for different text blocks in the same layout row, they are spliced according to the sorting order of col_no; when two text blocks are adjacent and the text box spacing is less than a preset spacing threshold, text block merging is performed, otherwise, no merging is performed; If it is layout column priority, text blocks are merged according to the sorting order of col_no; for different text blocks in the same layout column, they are judged according to the sorting order of row_no: if the row numbers row_no of two text blocks are continuous and the left boundary line of the text block is equal or the right boundary line is equal, merging is performed, otherwise, no merging is performed.

9. A PDF text extraction system that preserves the reading order of text, characterized in that, The system comprises: A text extraction module, for a PDF text type, if it is an original PDF text, BBox information of the PDF text is extracted; if it is a scanned PDF text, text extraction is performed to obtain text information; based on the BBox information and the text information, text block coordinate information is obtained; The text processing module performs coordinate alignment based on the text block coordinate information to obtain aligned text block coordinates; traverses the text blocks and forms a vertical layout point positioning list L y and a horizontal layout point positioning list L x ; performs layout row segmentation according to the vertical layout point positioning list L y , and obtains row numbers row_no and row processing numbers row_sequential_number of each text block; performs column processing on the text blocks based on the horizontal layout point positioning list L x , and obtains column numbers col_no and column processing numbers col_sequential_number of each text block; The text recovery module recovers the text based on the aligned text block coordinates, row number row_no, column number col_no, row processing number row_sequential_number and column processing number col_sequential_number, and based on a text recovery type, to obtain pure text content.

10. A PDF text extraction device that preserves the reading order of text, characterized by, The device includes a memory and a processor, which invokes computer instructions stored in the memory to execute the PDF text extraction method for maintaining text reading order according to any one of claims 1-8.

Images