A double-layer PDF generation and proofreading method and device

Abstract

The present invention provides a double-layer PDF generation and proofreading method and device, which performs secondary processing on the OCR engine recognition results, organizes the recognition results into logically complete paragraphs, and then uses paragraphs as units to perform error detection and error correction on paragraph sentences from the two aspects of character granularity and word granularity, and finally further generates a double-layer PDF document according to the error correction results. The device mainly includes an OCR recognition engine, a storage module, a paragraph synthesis module, an error detection and correction module, and an output module.

Description

Technical Field

[0001] The present invention relates to the technical field of computer information processing, and in particular to technologies related to double-layer PDF generation, production and proofreading. Background Art

[0002] A double-layer PDF (Portable Document Format) file is a PDF file with a multi-layer structure. Its file content contains both a text layer and an image layer, and the positions of the text layer and the image layer correspond to each other. It is based on a scanned PDF, and has undergone decontamination, correction, and OCR (optical character recognition, that is, using software to recognize scanned images as text). The text from OCR is then made into a transparent text layer and covered on the original scanned image layer, so it is called a "double-layer PDF". Compared with a pure scanned PDF, this type of PDF can be retrieved, copied, and exported for text, so it is also called a "searchable scanned PDF". In this way, an index database can be established based on text for scientific management.

[0003] However, there is a certain error rate in OCR recognition results, and the OCR effect is not very good. It often happens that a word appears on the scanned PDF, but not when searching; or the content copied and pasted from the double-layer PDF has many typos. This is all caused by inaccurate OCR recognition results. Summary of the invention

[0004] The purpose of the present invention is to provide a double-layer PDF generation and proofreading method and device to improve the accuracy of double-layer PDF text.

[0005] In order to achieve the above-mentioned purpose, the present invention provides a double-layer PDF generation and proofreading method, which performs secondary processing on the OCR engine recognition results, organizes the recognition results into logically complete paragraphs, and then uses paragraphs as units to perform error detection and error correction on paragraph sentences from both character granularity and word granularity, and finally further generates a double-layer PDF document based on the error correction results.

[0006] The above-mentioned double-layer PDF generation and proofreading method includes: 1) the OCR recognition engine module recognizes the image and outputs the recognition result; 2) the recognition result is filtered to obtain the recognition result meta information; the recognition result meta information includes the text block content, the circumscribed rectangle coordinates of all text blocks, and the text block score; 3) the text block is recognized to synthesize the text paragraph; according to the circumscribed rectangle coordinates of the text block, each text block is merged into a text paragraph, and a new text paragraph and the circumscribed rectangle coordinates of the paragraph are generated; 4) the text paragraph is error detected; 4.1) the error detection first uses the Chinese word segmenter to segment the word, detects errors from the character granularity and the word granularity, and integrates the doubts of these two granularities. 4.2) The error detection results are stored and persisted in the database to provide error location information for subsequent manual intervention and correction; 4.3) The error detection results are read, all suspected error locations are traversed, and the words in the error locations are replaced with sound-similar and shape-similar dictionaries. Then, the sentence perplexity is calculated through the language model, and the replacement results of all suspected error locations in the candidate set are compared and sorted to obtain the optimal corrected word; 4.4) The error correction results and sorting information are stored and persisted in the database to provide correction suggestions for subsequent manual intervention and correction; 5) Error correction; including automatic correction and / or manual intervention correction; 6) A double-layer PDF file is generated based on the correction results.

[0007] The above-mentioned double-layer PDF generation and proofreading method, wherein, includes: the step 2) includes: 2.1) obtaining the document MD5 value: according to the identification image document path, reading the document content, and calculating the document MD5 value according to the document content; 2.2) the recognition result is a json array, the recognition result is filtered, and the text block content, text block circumscribed rectangle coordinates, and text block scores of all text blocks are obtained, and the document MD5 value obtained in step 2.1 is used as the primary key to store the recognition result meta-information into the library; the upper left corner of the circumscribed rectangle of a text block is used as the origin (0, 0), the rightward extension line is used as the X-axis, and the downward extension line is used as the Y-axis to establish a coordinate system, and the coordinates of the circumscribed rectangle of the filtered text block are expressed as the coordinates of the upper left corner and the lower right corner of the rectangle in the coordinate system.

[0008] The above-mentioned double-layer PDF generation and proofreading method, wherein the step 3) includes: 3.1) finding the X value of the upper left corner coordinate of the circumscribed rectangle of the leftmost text block, recorded as X1; 3.2) finding the X value of the upper left corner coordinate of the circumscribed rectangle of the rightmost text block, recorded as X2; 3.3) traversing all recognition results, taking the X value of the upper left corner coordinate of the circumscribed rectangle of each text block, and assembling the text blocks with X values ​​between X1 and X2 into a row when the Y values ​​are the same, and sorting all the rows in ascending order according to the Y values ​​of the upper left corner coordinate of the circumscribed rectangle of the text block; 3.4) finding the row with the paragraph header; 3.5) assembling the rows into paragraphs according to the principle that the first character of the paragraph will be indented by two according to the paragraph document typesetting style.

[0009] The above-mentioned double-layer PDF generation and proofreading method includes: in the step 3.4), traversing all lines, calculating the difference between the X value of the upper left corner coordinate of the circumscribed rectangle of the start text block of each line and X1, if the difference is non-zero, identifying the line as a paragraph header, otherwise identifying the line as a normal line.

[0010] In the above-mentioned double-layer PDF generation and proofreading method, in the step 3.5), all lines are traversed, and if the current line is a paragraph header, the traversal continues backward until the next line is a paragraph header, all these lines are assembled into a paragraph, and then the next cycle is restarted until all lines are processed.

[0011] In the above-mentioned double-layer PDF generation and proofreading method, in said step 5), if the system is provided with an automatic error correction program, the system automatically corrects the detected errors.

[0012] In the above-mentioned double-layer PDF generation and proofreading method, in the step 5), if the system is set to manually intervene in text correction, the proofreading page is displayed for manual correction; the proofreading page is divided into two parts, one part is the original image, and the other part is a document restored based on the coordinates of the circumscribed rectangle of the recognized text and the coordinates of the circumscribed rectangle of the detected error text, wherein the area containing the detected error text will be highlighted in different colors, so that the user can immediately find the error text area and check it, and when the user clicks on the error text area, the system will give a correction suggestion, and the user can double-click the correction suggestion to update the error text area; when the user clicks on the error text area, the corresponding coordinate area of ​​the original image is also highlighted, so that the user can compare the original image with the recognition result.

[0013] In the above-mentioned double-layer PDF generation and proofreading method, the step 6) includes: 6.1) updating the content of all text paragraphs and the coordinates of the circumscribed rectangles of the text paragraphs that have been modified correctly by the user to the database for persistence; 6.2) reading the latest content of all text paragraphs and the circumscribed rectangles of the text paragraphs of the current recognized document, and outputting the double-layer PDF file in combination with the system-set text size.

[0014] Another technical solution provided by the present invention is a double-layer PDF generation and proofreading device, comprising the following modules: an OCR recognition engine module, which recognizes an image and outputs a recognition result; a storage module, which stores recognition result meta-information and error detection results; a paragraph synthesis module, which merges each text block into a text paragraph according to the content of each text block of the OCR recognized document and its circumscribed rectangular coordinates, and generates a new text paragraph and the circumscribed rectangular coordinates of the paragraph; an error detection and correction module, which applies Chinese lexical analysis to the synthesized text paragraph to detect text paragraphs with lexical errors, stores relevant text paragraph information, and further corrects text paragraphs with lexical errors; an output module, which outputs a double-layer PDF file according to the original image, the content of the recognized text paragraph, the circumscribed rectangular coordinates of the recognized text paragraph and the font size.

[0015] Compared with the prior art, the beneficial technical effects of the present invention are:

[0016] During the OCR recognition process, there are a small number of text recognition errors. The traditional proofreading method can only be to manually read the original text and check it bit by bit. Before proofreading, it is impossible to determine where the document itself is wrong, and the manual error checking and proofreading method is inefficient. The advantages of the double-layer PDF generation and proofreading method and device of the present invention are: secondary processing of the text output by OCR, Chinese lexical analysis, automatic inspection of errors in the recognized text, and support for manual intervention inspection and correction, and displaying the original text and the recognition result in the form of visual comparison, so that the text recognition errors are clear at a glance; at the same time, relevant modification suggestions can be given according to the error detection results, which greatly improves the proofreading efficiency; finally, the double-layer PDF is output according to the proofreading results, so that the output double-layer PDF file and the effect of the device during proofreading can achieve the effect of what you see is what you get, which improves the proofreading efficiency and output accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a flow chart of a double-layer PDF generation and proofreading method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0018] The following will be combined Figure 1 The double-layer PDF generation and proofreading method and device of the present invention are further described in detail.

[0019] The double-layer PDF generation and proofreading method of the present invention performs secondary processing on the OCR engine recognition results, organizes the recognition results into logically complete paragraphs, and then uses the paragraphs as units to perform error detection and error correction on the paragraph sentences from both the character granularity and the word granularity, and finally further generates a double-layer PDF document according to the correction results.

[0020] Figure 1 It is a flow chart of a double-layer PDF generation and proofreading method according to an embodiment of the present invention.

[0021] See also Figure 1 The double-layer PDF generation and proofreading method of this embodiment includes the following steps:

[0022] Step 1: The OCR recognition engine module recognizes the image;

[0023] The image to be recognized is input into the OCR recognition engine module, which calls the trained Chinese, letter and number models for detection, recognition and other processing, and outputs the recognition results;

[0024] Step 2: Store the recognition result meta information;

[0025] Filter the recognition results and store them in a persistent database. This step can be divided into two steps:

[0026] Step 2.1: Get the document MD5 value: Read the document content according to the recognized image document path, and calculate the document MD5 value based on the document content. This value will be used as the primary key for all subsequent information storage;

[0027] Step 2.2: The recognition result is a json array. Filter the recognition result to obtain the text block content, text block circumscribed rectangle coordinates, and text block scores of all text blocks. These information elements are called recognition result meta-information. Use the document MD5 value obtained in step 2.1 as the primary key to store the recognition result meta-information in the database.

[0028] Take the upper left corner of the circumscribed rectangle of a text block as the origin (0, 0), extend the line to the right as the X-axis, and extend the line downward as the Y-axis to establish a coordinate system. The coordinates of the circumscribed rectangle of the filtered text block are expressed by the coordinates of the upper left and lower right corners of the rectangle in the coordinate system.

[0029] Step 3: Identify text blocks and synthesize text paragraphs;

[0030] According to the coordinates of the circumscribed rectangle of the text block, the text blocks are merged into a text paragraph, and a new text paragraph and the circumscribed rectangle coordinates of the paragraph are generated. The purpose of this step is to make the recognized text blocks form a whole with semantic relevance, so as to prepare for the next step of error detection, as follows:

[0031] Step 3.1: Find the X value of the upper left corner coordinate of the circumscribed rectangle of the leftmost text block, record it as X1;

[0032] Since the recognition result array itself is arranged in ascending order, get the first 20 items in the recognition result (if the recognition result is less than 20 items, take all of them), scan from the data set (the data set consisting of the first 20 items) to find the minimum value of X, which is the X value of the upper left corner coordinate of the circumscribed rectangle of the leftmost text block;

[0033] Step 3.2: Find the X value of the upper left corner coordinate of the circumscribed rectangle of the rightmost text block, recorded as X2;

[0034] Since the recognition result array itself is arranged in ascending order, obtain the last 20 items in the recognition result (if the recognition result is less than 20 items, take all of them), scan the data set to find the maximum value of X, which is the X value of the upper left corner coordinate of the circumscribed rectangle of the rightmost text block;

[0035] Step 3.3: Assemble the text blocks into rows;

[0036] Traverse all recognition results, take the X value of the upper left corner coordinate of each text block's circumscribed rectangle, and assemble the text blocks with X values ​​between the leftmost value X1 and the rightmost value X2 into a row when the Y values ​​are the same. Store all the same row data into a row array, and sort the row array in ascending order according to the Y value of the upper left corner coordinate of the text block's circumscribed rectangle.

[0037] Step 3.4: Find the line where the paragraph header exists;

[0038] Traverse all lines and calculate the difference between the X value of the start text block of each line and the leftmost value X1 obtained in step 3.1. If the difference is non-zero, the line is marked as a paragraph header, otherwise it is marked as a normal line;

[0039] Step 3.5: According to the paragraph document layout style, the first character of a paragraph will be indented by two, and the lines are assembled into paragraphs;

[0040] Traverse all lines. If the current line is a paragraph header, continue traversing backward until the next line is a paragraph header. Assemble all these lines into a paragraph, and then restart the next cycle until all lines are processed;

[0041] Step 4: Perform error detection on the text paragraph;

[0042] Perform text error correction on the synthesized text paragraph to obtain the detection result. This step is mainly to further improve the accuracy of text recognition. If the system is set to automatically perform text error correction, the system will automatically correct the detected errors. This step is divided into two steps:

[0043] Step 4.1: Error detection;

[0044] Error detection is first performed through Chinese word segmentation. If there are typos in the sentences of a paragraph, the segmentation results may have segmentation errors. In this way, errors can be detected from both the character granularity and the word granularity. The suspected error results of these two granularities are integrated to form a candidate set of suspected error positions.

[0045] Step 4.2: The error detection results are stored in a persistent database to provide error location information for subsequent manual intervention and correction;

[0046] Step 4.3: Error correction;

[0047] Error correction is to read the error detection results from the system, traverse all suspected error positions, and use the sound-similar and shape-similar dictionaries to replace the words in the error positions. Then, the sentence perplexity is calculated through the language model, and all candidate set replacement results are compared and sorted to obtain the optimal correction word.

[0048] Step 4.4: The error correction results and sorting information are stored and persisted in the database to provide correction suggestions for subsequent manual intervention;

[0049] Step 5: Text proofreading;

[0050] If the system is set to manually intervene in text correction, the proofreading page will be displayed for manual correction; the proofreading page is divided into two parts, one is the original image, and the other is the document restored based on the coordinates of the circumscribed rectangle of the recognized text and the circumscribed rectangle of the detected error text. The area containing the detected error text will be highlighted in different colors, so that the user can immediately find the error text area and check it. When the user clicks on the error text area, the system will give correction suggestions. The user can double-click the correction suggestion to update the error text area; when the user clicks on the error text area, the corresponding coordinate area of ​​the original image is also highlighted, so that the user can compare the original image with the recognition result.

[0051] If the system is set to automatically perform text error correction, the system will automatically correct the detected errors and step 5 can be omitted, or the system can be set to automatically perform text error correction and text proofreading at the same time;

[0052] Step 6: Generate a double-layer PDF file;

[0053] Output a double-layer PDF file based on all text paragraph contents, text paragraph circumscribed rectangle coordinates and set font size after user proofreading; specifically including:

[0054] Step 6.1: Update all the text paragraph contents and the coordinates of the circumscribed rectangle of the text paragraphs that have been modified correctly by the user to the database for persistence;

[0055] Step 6.2: Read all the latest text paragraph contents and the circumscribed rectangle coordinates of the current recognized document, and output a double-layer PDF file in combination with the system-set text size.

[0056] The present invention also discloses a double-layer PDF generation and proofreading device, comprising the following modules:

[0057] OCR recognition engine module, which recognizes images based on existing mature market technologies and outputs recognition results;

[0058] Storage module, which mainly stores document recognition result meta information and error detection result information, which are required in document display, proofreading and double-layer PDF generation stages;

[0059] The paragraph synthesis module combines the text in adjacent rectangles of each text block and its bounding rectangle from the OCR-recognized document into a text paragraph based on the coordinate position relationship, and generates a new text paragraph and the coordinates of the bounding rectangle of the paragraph;

[0060] The error detection and correction module applies Chinese lexical analysis to the synthesized text paragraphs to detect text paragraphs with lexical errors, stores relevant text paragraph information, and further corrects the text paragraphs;

[0061] The output module outputs a double-layer PDF file based on the original image, the recognized text paragraph content, the recognized text paragraph circumscribed rectangle coordinates and the font size.

[0062] The method can realize high-efficiency OCR proofreading and output high-precision double-layer PDF documents, so that the output double-layer PDF files and the effect of the device during proofreading can achieve the effect of what you see is what you get, thereby improving the proofreading efficiency and output accuracy.

[0063] Obviously, those skilled in the art can make various changes and modifications to the invention without departing from the spirit and scope of the invention. Thus, if these modifications and variations of the invention fall within the scope of the claims of the invention and their equivalents, the invention is also intended to include these modifications and variations.

Claims

1. A double-layer PDF generation and proofreading method, characterized in that: Perform secondary processing on the OCR engine recognition results, organize the recognition results into logically complete paragraphs, and then perform error detection and error correction on the paragraph sentences from both the character granularity and word granularity. Finally, generate a double-layer PDF document based on the error correction results. Specifically include: 1) The OCR recognition engine module recognizes the image and outputs the recognition result; 2) Filter the recognition results to obtain recognition result meta information; The recognition result meta-information includes the text block content, the coordinates of the text block circumscribed rectangle, and the text block score of all text blocks; 3) Identify text blocks and synthesize text paragraphs; Merge the text blocks into text paragraphs according to the circumscribed rectangle coordinates of the text blocks, and generate a new text paragraph and the circumscribed rectangle coordinates of the paragraph; 4) Perform error detection on text paragraphs; 4.1) Error Detection First, the Chinese word segmenter is used to segment words, detect errors at both the character and word granularity, and integrate the suspected error results of these two granularities to form a candidate set of suspected error positions; 4.2) The error detection results are stored and persisted in the database to provide error location information for subsequent manual intervention and correction; 4.3) Read the error detection results, traverse all suspected error positions, and use the sound-similar and shape-similar dictionaries to replace the words in the wrong position. Then calculate the sentence perplexity through the language model, compare and sort the replacement results of all suspected error positions in the candidate set, and get the optimal correction word; 4.4) Error correction results and sorting information are stored and persisted to provide correction suggestions for subsequent manual intervention; 5) Error correction; Including automatic error correction and / or manual error correction; 6) Generate a double-layer PDF file based on the error correction results.

2. The double-layer PDF generation and proofreading method according to claim 1, characterized in that: The step 2) comprises: 2.1) Get the MD5 value of the document: Read the document content according to the identified image document path, and calculate the MD5 value of the document according to the document content; 2.2) The recognition result is a json array. Filter the recognition result to obtain the text block content, text block circumscribed rectangle coordinates, and text block scores of all text blocks. Use the document MD5 value obtained in step 2.1 as the primary key to store the recognition result meta information in the database; A coordinate system is established with the upper left corner of the circumscribed rectangle of a text block as the origin (0, 0), the rightward extension line as the X-axis, and the downward extension line as the Y-axis. The coordinates of the circumscribed rectangle of the filtered text block are expressed as the coordinates of the upper left and lower right corners of the rectangle in the coordinate system.

3. The double-layer PDF generation and proofreading method as claimed in claim 2, characterized in that: The step 3) comprises: 3.1) Find the X value of the upper left corner coordinate of the circumscribed rectangle of the leftmost text block, record it as X1; 3.2) Find the X value of the upper left corner coordinate of the circumscribed rectangle of the rightmost text block, recorded as X2; 3.3) Traverse all recognition results, take the X value of the upper left corner coordinate of each text block's circumscribed rectangle, and assemble the text blocks with X values ​​between X1 and X2 into a row if the Y values ​​are the same. Sort all rows in ascending order according to the Y value of the upper left corner coordinate of the text block's circumscribed rectangle; 3.4) Find the line where the paragraph header exists; 3.5) According to the paragraph document layout style, the first character of the paragraph will be indented by two, and the lines are assembled into paragraphs.

4. The double-layer PDF generation and proofreading method as claimed in claim 3, characterized in that: In the step 3.4), all the lines are traversed, and the difference between the X value of the upper left corner coordinate of the circumscribed rectangle of the start text block of each line and X1 is calculated. If the difference is non-zero, the line is identified as a paragraph header, otherwise, the line is identified as a normal line.

5. The double-layer PDF generation and proofreading method as claimed in claim 4, characterized in that: In the step 3.5), all the lines are traversed, and if the current line is a paragraph header, the traversal continues backward until the next line is a paragraph header, all these lines are assembled into a paragraph, and then the next cycle is restarted until all the lines are processed.

6. The double-layer PDF generation and proofreading method according to claim 1, characterized in that: In step 5), if the system is set with an automatic error correction program, the system automatically corrects the detected errors.

7. The double-layer PDF generation and proofreading method according to claim 1, characterized in that: In the step 5), if the system is set to manually intervene in text correction, the proofreading page will be displayed for manual correction; the proofreading page is divided into two parts, one part is the original image, and the other part is the document restored based on the coordinates of the circumscribed rectangle of the recognized text and the coordinates of the circumscribed rectangle of the detected error text, wherein the area containing the detected error text will be highlighted in different colors, so that the user can immediately find the error text area and check it, and when the user clicks on the error text area, the system will give correction suggestions, and the user can double-click the correction suggestions to update the error text area; when the user clicks on the error text area, the corresponding coordinate area of ​​the original image is also highlighted, so that the user can compare the original image and the recognition result area conveniently.

8. The double-layer PDF generation and proofreading method according to claim 1, characterized in that: The step 6) comprises: Step 6.1: Update all the text paragraph contents and the coordinates of the circumscribed rectangles of the text paragraphs that have been modified correctly by the user to the database for persistence; Step 6.2: Read all the latest text paragraph contents and the circumscribed rectangle coordinates of the current recognized document, and output a double-layer PDF file in combination with the system-set text size.

9. A double-layer PDF generation and proofreading device, characterized in that: Used to execute the double-layer PDF generation and proofreading method according to any one of claims 1 to 8, the double-layer PDF generation and proofreading device comprises the following modules: OCR recognition engine module, which recognizes the image and outputs the recognition result; A storage module, storing recognition result meta information and error detection results; The paragraph synthesis module combines the text blocks into text paragraphs according to the contents of the text blocks and the coordinates of the circumscribed rectangles of the OCR-recognized documents, and generates a new text paragraph and the coordinates of the circumscribed rectangle of the paragraph; An error detection and correction module applies Chinese lexical analysis to the synthesized text paragraphs to detect text paragraphs with lexical errors, stores relevant text paragraph information, and further corrects the text paragraphs with lexical errors; The output module outputs a double-layer PDF file based on the original image, the recognized text paragraph content, the recognized text paragraph circumscribed rectangle coordinates and the font size.

Images