Method for extracting and restoring Chinese and English texts and Arabic numerals in Uyghur language PDF document

Abstract

This invention provides a method for extracting and restoring Chinese and English text and Arabic numerals from Uyghur PDF documents. It utilizes PDFMiner to extract all element information from the PDF document, thereby extracting all text content. Then, all text is sorted line by line and formatted from right to left. All Chinese and English characters, Arabic numerals, and other text characters sorted from left to right are selected and then sorted again from left to right. Finally, all characters are inserted into a Word document according to their corresponding format, thus ensuring a higher degree of consistency between the reconstructed Uyghur document and the original document. This invention can automatically detect and extract characters represented from left to right in Uyghur PDF documents and then insert them into the corresponding positions in a DOCX file in a left-to-right order, thereby ensuring that the layout of the target document is as consistent as possible with the original document.

Description

Technical Field

[0001] The present invention relates to computer algorithms, content parsing and reconstruction of PDF documents, and in particular to a method for extracting and restoring Chinese and English text and Arabic numerals in Uyghur PDF documents. Background Technology

[0002] PDF is one of the most widely used document formats, primarily for file exchange and printing, and cannot interact with other computer programs. With the widespread application of PDF in finance, scientific research, education, and other fields, automatically recognizing PDF documents, extracting useful data, and reconstructing them into easily editable Word documents has become a significant concern. PDF documents mainly consist of text, images, tables, formulas, etc., and among these, the quality of text content reproduction, as a primary form of presentation, has a crucial impact on the overall rendering effect of the PDF document.

[0003] However, during the extraction and restoration of PDF document content, unlike documents with Chinese and English text arranged from left to right, Uyghur PDF text characters are arranged from right to left. Sometimes, Uyghur PDF documents contain a mixture of Chinese and English characters, Arabic numerals, and other characters arranged from left to right, which can cause some characters in the restored Word document to appear out of order, thus affecting the document's readability.

[0004] To address the aforementioned issues, this invention focuses on how to extract and restore left-to-right text, including Chinese and English characters and Arabic numerals, from Uyghur PDF documents, and then restore it in a Word document, thereby ensuring that the layout of the target document is as consistent as possible with the original document. Summary of the Invention

[0005] In view of the shortcomings of the prior art described above, the method of this invention addresses the problem of the inability to effectively restore Chinese and English characters, Arabic numerals, and other text in Uyghur PDF documents. First, PDFMiner is used to extract all element information from the Uyghur PDF, thereby extracting all text information. Then, all Uyghur text characters are formatted from right to left. After formatting, Chinese and English characters, Arabic numerals, and other text characters ordered from left to right are selected and rearranged from left to right. Finally, all characters are inserted into a Word document, resulting in a Uyghur document with higher fidelity after rearrangement.

[0006] To achieve the above and other related objectives, the present invention provides a method for extracting and restoring Chinese and English text and Arabic numerals from a Uyghur PDF document, comprising the following steps:

[0007] S1) Input a PDF file, use PDFMiner to parse the content in the text, and obtain information about all elements in the PDF;

[0008] S2) Filter the information of all elements in the PDF and extract the information corresponding to the text content. During extraction, the element with the element type "char" is the content corresponding to the text.

[0009] S3) After obtaining the text elements of all pages in all PDF documents, merge the lines of the text;

[0010] S4) If it is a Uyghur character, the order of all Uyghur characters is reversed; otherwise, the order of Chinese, English, Arabic, etc. characters is consistent with their x0 coordinate value in the PDF document, that is, from left to right. Chinese, English, Arabic, etc. characters can be arranged in normal order.

[0011] S41) Detect all characters in each line and find consecutive characters from left to right in each line. The method for character detection is as follows:

[0012] If the character's encoding range meets the following conditions, then the character is displayed from left to right:

[0013] \u4e00<=c<=\u9fff #Chinese characters

[0014] \u0021<=c<=\u007e #English character

[0015] \u00c0<=c<=\u02af #Latin character

[0016] \u1e00<=c<=\u1eff

[0017] If the character 'c' is an Arabic numeral or belongs to one of the following characters: , , ! :, then the character is also displayed from left to right.

[0018] S42) Combine consecutive characters from left to right in each line to form a text block text_unit. The text of the text block is a combination of all characters from left to right.

[0019] S43) After obtaining the character content and coordinate position of the text block, reverse the order of all characters in the text block;

[0020] S5) Combine all characters from left to right into a text block, reverse the characters in the text block, merge all characters in each line, and obtain the line text and coordinate position of each line text. The text line coordinate calculation method is as follows: calculate the coordinate position of the line text (l_x0,l_y0,l_x1,l_y1), where the coordinate position l_x0 of the leftmost line text is the coordinate x0 of the leftmost character, the coordinate position l_y0 of the bottom of the line text is the y0 coordinate of all characters in the line with the highest frequency, the coordinate l_x1 of the rightmost line text is the x1 coordinate of the rightmost character in the line, and the coordinate l_y1 of the top of the line text is the y1 coordinate of all characters in the line with the highest frequency.

[0021] S6) After obtaining the text of each line, reverse the order of all characters in each line of text to obtain a text line in which all characters are sorted from right to left. This text line is the text line corresponding to the Uyghur document.

[0022] S7) Insert the text line obtained in step S6) into the corresponding line coordinate position in the DOCX document in a right-to-left format.

[0023] Optionally, the PDFMiner mentioned in step S1) is a PDF file parsing system based on the PDFMiner library. This system can parse the content of all images, tables and text in a PDF document and their coordinate position information on the page. For text, the system can obtain information such as font size, color, font type and whether it is bold.

[0024] Optionally, the content corresponding to the text mentioned in step S2) can be obtained by acquiring all the character content in the text, as well as the coordinate position (x0, y0, x1, y1) of the character in the document, the font size of the character, the color of the character, whether the character is italic, and whether the character is bold. Here, the coordinates of the character x0 represent the leftmost coordinate position of the character, x1 represent the rightmost coordinate position, y0 represent the bottom coordinate position, and y1 represent the top coordinate position. The unit of the coordinates is pixel value.

[0025] Optionally, during the text merging process described in step S3), all characters on different pages are first sorted according to the coordinate position y0 at the bottom of the character. After sorting, all characters are divided into different rows according to the threshold α = 6. That is, if the difference in the y0 coordinates between characters is less than 6, then the two characters are in the same row.

[0026] Optionally, in step S4), the Uyghur characters in each line parsed by PDFMiner are sorted according to the size of the x0 value.

[0027] Optionally, the text block coordinate calculation method in step S42) is as follows: calculate the coordinate position (u_x0, u_y0, u_x1, u_y1) of the text block, where the leftmost coordinate position u_x0 of the text block is the coordinate x0 of the leftmost character, the bottom coordinate position u_y0 of the text block is the y0 coordinate of all characters in the text block with the highest frequency, the rightmost coordinate u_x1 of the text block is the coordinate x1 of the rightmost character in the text block, and the top coordinate u_y1 of the text block is the y1 coordinate of all characters in the text block with the highest frequency.

[0028] Optionally, after the text block is reversed in step S43), all characters displayed from left to right in the Uyghur PDF document are also sorted from right to left.

[0029] Optionally, during the text block merging described in step S5), the text block text_unit is represented as an independent character.

[0030] As described above, the present invention provides a method for extracting and restoring Chinese and English text and Arabic numerals from Uyghur PDF documents, which has the following beneficial effects: The method of the present invention uses PDFMiner to extract all element information from the PDF document, and then extracts all text content from it. Then, all text is sorted by line and formatted from right to left. During formatting, all Chinese and English characters, Arabic numerals, and other text characters sorted from left to right are selected and sorted according to left-to-right order. Finally, all characters are inserted into a WORD document according to their corresponding format, thereby ensuring a higher consistency between the content of the reconstructed Uyghur document and the original document. The present invention provides a method for extracting and restoring Chinese and English characters, Arabic numerals, and other text from left to right in Uyghur PDF documents. It can automatically detect and extract characters represented from left to right in Uyghur PDF documents, and then insert them into the corresponding positions in the DOCX file in a left-to-right order, thereby ensuring that the layout of the target document is as consistent as possible with the original document. Attached Figure Description

[0031] Figure 1 A schematic diagram illustrating a method for extracting and restoring Chinese and English text and Arabic numerals from a Uyghur PDF document, as provided in one embodiment; Detailed Implementation

[0032] The following specific examples illustrate the implementation of the method of the present invention. Those skilled in the art can easily understand other advantages and effects of the method of the present invention from the content disclosed in this specification. The method of the present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the method of the present invention. It should be noted that, unless otherwise specified, the following embodiments and features in the embodiments can be combined with each other.

[0033] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the method of the present invention. Therefore, the drawings only show the components related to the method of the present invention and are not drawn according to the actual number, shape and size of the components. In actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0034] Please see Figure 1 As shown, the method of the present invention provides a method for extracting and restoring Chinese and English text and Arabic numerals from a Uyghur PDF document, including the following steps:

[0035] S1) Input a PDF file, use PDFMiner to parse the content in the text, and obtain information about all elements in the PDF;

[0036] S2) Filter the information of all elements in the PDF and extract the information corresponding to the text content. During extraction, the element with the element type "char" is the content corresponding to the text.

[0037] S3) After obtaining the text elements of all pages in all PDF documents, merge the lines of the text;

[0038] S4) If it is a Uyghur character, the order of all Uyghur characters is reversed; otherwise, the order of Chinese, English, Arabic, etc. characters is consistent with their x0 coordinate value in the PDF document, that is, from left to right. Chinese, English, Arabic, etc. characters can be arranged in normal order.

[0039] S41) Detect all characters in each line and find consecutive characters from left to right in each line. The method for character detection is as follows:

[0040] If the character's encoding range meets the following conditions, then the character is displayed from left to right:

[0041] \u4e00<=c<=\u9fff #Chinese characters

[0042] \u0021<=c<=\u007e #English character

[0043] \u00c0<=c<=\u02af #Latin character

[0044] \u1e00<=c<=\u1eff

[0045] If the character 'c' is an Arabic numeral or belongs to one of the following characters: , , ! :, then the character is also displayed from left to right.

[0046] S42) Combine consecutive characters from left to right in each line to form a text block text_unit. The text of the text block is a combination of all characters from left to right.

[0047] S43) After obtaining the character content and coordinate position of the text block, reverse the order of all characters in the text block;

[0048] S5) Combine all characters from left to right into a text block, reverse the characters in the text block, merge all characters in each line, and obtain the line text and coordinate position of each line text. The text line coordinate calculation method is as follows: calculate the coordinate position of the line text (l_x0,l_y0,l_x1,l_y1), where the coordinate position l_x0 of the leftmost line text is the coordinate x0 of the leftmost character, the coordinate position l_y0 of the bottom of the line text is the y0 coordinate of all characters in the line with the highest frequency, the coordinate l_x1 of the rightmost line text is the x1 coordinate of the rightmost character in the line, and the coordinate l_y1 of the top of the line text is the y1 coordinate of all characters in the line with the highest frequency.

[0049] S6) After obtaining the text of each line, reverse the order of all characters in each line of text to obtain a text line in which all characters are sorted from right to left. This text line is the text line corresponding to the Uyghur document.

[0050] S7) Insert the text line obtained in step S6) into the corresponding line coordinate position in the DOCX document in a right-to-left format.

[0051] In an exemplary embodiment, the PDFMiner mentioned in step S1) is a PDF file parsing system based on the PDFMiner library. This system can parse the content of all images, tables, and text in a PDF document and their coordinate positions on the page. For text, the system can obtain the font size, color, font type, and whether it is bold. The content corresponding to the text mentioned in step S2) can obtain all the character content in the text, as well as the coordinate positions (x0, y0, x1, y1) of the character in the document, the font size, color, whether the character is italic, and whether the character is bold. Here, the coordinates x0 represent the leftmost coordinate position, x1 represent the rightmost coordinate position, y0 represent the bottom coordinate position, and y1 represent the top coordinate position. The units of the coordinates are pixel values. When merging text in step S3), all characters on different pages are first sorted according to the bottom coordinate position y0 of the characters. After sorting, all characters are divided into different rows according to a threshold α = 6. That is, if the difference in y0 coordinates between characters is less than 6, then the two characters are in the same row. In step S4), the Uyghur characters in each line parsed by PDFMiner are sorted according to their x0 values. In step S42), the text block coordinates are calculated as follows: the coordinates (u_x0, u_y0, u_x1, u_y1) of the text block are calculated. The leftmost coordinate u_x0 is the x0 coordinate of the leftmost character; the bottom coordinate u_y0 is the y0 coordinate of the most frequent character in the block; the rightmost coordinate u_x1 is the x1 coordinate of the rightmost character; and the top coordinate u_y1 is the y1 coordinate of the most frequent character. In step S43), after the text blocks are reversed, all characters displayed from left to right in the Uyghur PDF document are also sorted from right to left. In step S5), when merging text blocks, the text block `text_unit` is represented as an independent character.

[0052] In summary, the present invention provides a method for extracting and restoring Chinese and English text and Arabic numerals from Uyghur PDF documents, which has the following beneficial effects: The method utilizes PDFMiner to extract all element information from the PDF document, thereby extracting all text content. Then, all text is sorted by line and formatted from right to left. During formatting, all Chinese and English characters, Arabic numerals, and other text characters ordered from left to right are selected and sorted accordingly. Finally, all characters are inserted into a Word document according to their corresponding formats, thus ensuring a higher degree of consistency between the reconstructed Uyghur document and the original document. This invention provides a method for extracting and restoring left-to-right text, including Chinese and English characters and Arabic numerals, from Uyghur PDF documents. It can automatically detect and extract characters from left to right in Uyghur PDF documents and then insert them into the corresponding positions in the DOCX file in a left-to-right order, thereby ensuring that the layout of the target document is as consistent as possible with the original document.

[0053] The above embodiments are merely illustrative of the principles and effects of the method of the present invention, and are not intended to limit the method of the present invention. Any person skilled in the art can modify or change the above embodiments without departing from the spirit and scope of the method of the present invention. Therefore, all equivalent modifications or changes made by those skilled in the art without departing from the spirit and technical concept disclosed in the method of the present invention should still be covered by the claims of the method of the present invention.

Claims

1. A method for extracting and restoring Chinese and English text and Arabic numerals from a Uyghur PDF document, including the following steps: S1) Input a PDF file, use PDFMiner to parse the content in the text, and obtain information about all elements in the PDF; S2) Filter the information of all elements in the PDF and extract the information corresponding to the text content. During extraction, the element with the element type "char" is the content corresponding to the text. S3) After obtaining the text elements of all pages in all PDF documents, merge the lines of the text; S4) If it is a Uyghur character, the order of all Uyghur characters is reversed; otherwise, the order of Chinese, English, and Arabic characters is consistent with their x0 coordinate values ​​in the PDF document, that is, from left to right, and the Chinese, English, and Arabic characters are arranged in normal order. S41) Detect all characters in each line and find the consecutive characters from left to right in each line; S42) Combine consecutive characters from left to right in each line to form a text block text_unit. The text of the text block is a combination of all characters from left to right. S43) After obtaining the character content and coordinate position of the text block, reverse the order of all characters in the text block; S5) Combine all characters from left to right into a text block, reverse the characters in the text block, merge all characters in each line, and obtain the line text and coordinate position of each line text. The text line coordinate calculation method is as follows: calculate the coordinate position of the line text (l_x0,l_y0,l_x1,l_y1), where the coordinate position l_x0 of the leftmost line text is the coordinate x0 of the leftmost character, the coordinate position l_y0 of the bottom of the line text is the y0 coordinate of all characters in the line with the highest frequency, the coordinate l_x1 of the rightmost line text is the x1 coordinate of the rightmost character in the line, and the coordinate l_y1 of the top of the line text is the y1 coordinate of all characters in the line with the highest frequency. S6) After obtaining the text of each line, reverse the order of all characters in each line of text to obtain a text line in which all characters are sorted from right to left. This text line is the text line corresponding to the Uyghur document. S7) Insert the text line obtained in step S6) into the corresponding line coordinate position in the DOCX document in a right-to-left format; When merging text as described in step S3), firstly, all characters on different pages are sorted according to the coordinate position y0 of the bottom of the character. After sorting, all characters are divided into different rows according to the threshold α = 6. That is, if the difference between the y0 coordinates of the characters is less than 6, then the two characters are in the same row. In step S4), the Uyghur characters in each line parsed by PDFMiner are sorted according to the size of the x0 value; The text block coordinate calculation method described in step S42) is as follows: calculate the coordinate position (u_x0, u_y0, u_x1, u_y1) of the text block, where the leftmost coordinate position u_x0 of the text block is the coordinate x0 of the leftmost character, the bottom coordinate position u_y0 of the text block is the y0 coordinate of all characters in the text block with the highest frequency, the rightmost coordinate u_x1 of the text block is the coordinate x1 of the rightmost character in the text block, and the top coordinate u_y1 of the text block is the y1 coordinate of all characters in the text block with the highest frequency.

2. The method for extracting and restoring Chinese and English text and Arabic numerals from a Uyghur PDF document according to claim 1, characterized in that, Step S1) describes PDFMiner as a PDF file parsing system based on the PDFMiner library. This system parses the content of all images, tables, and text in a PDF document and their coordinate positions on the page. For text, the system obtains information such as font size, color, font type, and whether it is bold.

3. The method for extracting and restoring Chinese and English text and Arabic numerals from a Uyghur PDF document according to claim 1, characterized in that, Step S2) The content corresponding to the text is obtained by acquiring all the character content in the text, as well as the coordinate position (x0, y0, x1, y1) of the character in the document, the font size of the character, the color of the character, whether the character is italic, and whether the character is bold. Among them, the coordinates of the character x0 represent the leftmost coordinate position of the character, x1 represent the rightmost coordinate position, y0 represent the bottom coordinate position, and y1 represent the top coordinate position. The unit of coordinates is pixel value.

4. The method for extracting and restoring Chinese and English text and Arabic numerals from a Uyghur PDF document according to claim 1, characterized in that, After the text block is reversed as described in step S43), all characters displayed from left to right in the Uyghur PDF document are also sorted from right to left.

5. The method for extracting and restoring Chinese and English text and Arabic numerals from a Uyghur PDF document according to claim 1, characterized in that, When merging text blocks as described in step S5), the text block text_unit is represented as an independent character.

Images