Bidding document title structure extraction method based on OCR engine

Abstract

The invention discloses a bidding document title structure extraction method based on an OCR engine. The bidding document title structure extraction method comprises the steps that a bidding document in a pdf format is converted into an image file; performing text content recognition on the bidding document image file by utilizing OCR (Optical Character Recognition); sorting bidding document character recognition results according to the text block position information of the OCR recognition results, and obtaining bidding document text contents; performing title structure extraction on the paragraph content of the bidding document text; optimizing the title paragraphs by using header and footer rules and title features; the method solves the problems of low efficiency and strong manual dependence in the prior art, can quickly and accurately extract the title information of the document, improves the title extraction efficiency and accuracy, and is suitable for various types of documents.

Description

Technical Field

[0001] The invention belongs to the technical field of table content extraction, and in particular relates to a method for extracting the title structure of a tender document based on an OCR engine. Background Art

[0002] There are mainly two methods for extracting the title structure of bidding documents: rule-based methods and machine learning-based methods. The rule-based method usually requires a series of rules to be defined in advance to match the features of the title, and then matching and extracting according to the rules. This method is simple and intuitive, but requires a lot of manual participation and has high requirements for the format and structure of the document. The machine learning-based method automatically learns the features of the title by building a model, and then extracts the title according to the model. This method can effectively reduce manual intervention, but requires a lot of labeled data and training time. Therefore, it is necessary to design a bidding document title structure extraction method based on an OCR engine to solve the above problems. Summary of the invention

[0003] The technical problem to be solved by the present invention is to provide a method for extracting the title structure of a bid document based on an OCR engine, which is used to solve the problems of low efficiency and strong manual dependence of the existing technology. It has the characteristics of being able to quickly and accurately extract the title information of the document, improving the efficiency and accuracy of title extraction, and being suitable for various types of documents.

[0004] In order to achieve the above technical effects, the technical solution adopted by the present invention is: A method for extracting the title structure of a tender document based on an OCR engine comprises the following steps: S1, convert the tender document file in pdf format into an image file; S2, using OCR recognition to identify the text content of the tender document image file; S3, sorting out the text recognition results of the tender document according to the text block position information of the OCR recognition result, and obtaining the text content of the tender document; S4, extracting the title structure of the paragraph content of the bid document; S5, optimize the title paragraph using header and footer rules and title features.

[0005] Preferably, in step S1, the bid document file in PDF format is converted into an image file using the ffmpeg tool, and the image file is processed to obtain a clean image file; the specific method is: Open the command line interface and enter the command to convert the PDF file into image format; Remove stamps or interfering information: Use image processing algorithms to locate the content of the seal image on the base image; Optimize the seal text in the seal area according to the seal features and remove interference information; Use RGB different level processing tools to clean up the stamp content in the image and obtain clean image content.

[0006] Preferably, in step S2, using OCR recognition technology to perform text content recognition on the tender document image file includes: Select a suitable OCR recognition tool or library to perform text recognition operations on image files and extract text information.

[0007] Preferably, in step S3, arranging the text recognition results of the tender document according to the text block position information of the OCR recognition results, and obtaining the text content of the tender document includes: According to the location information of the text blocks, the tender text content is sorted and extracted to ensure accuracy and completeness.

[0008] Preferably, in step S4, extracting the title structure of the paragraph content of the bid document includes: Through text processing, title information is extracted from text paragraphs and a title structure is established for subsequent processing and analysis.

[0009] Preferably, in step S5, optimizing the title paragraph by using the header and footer rules and the title features includes optimizing the title paragraph according to the header and footer rules and the title features.

[0010] Preferably, in step S4, paragraph title extraction and level classification are performed, and the specific method includes: Paragraph title discovery, using the directory location area to locate and extract title extraction; When there is no table of contents, the title is extracted based on the title features, which include the difference in character size between the upper and lower paragraphs, the paragraph position area, the character height, the difference in the number of characters between the previous and next paragraphs, and the paragraph number; Paragraph title level label classification and recognition, using SVM classification algorithm to classify paragraph title level, paragraph sequence level and paragraph vector analysis.

[0011] The beneficial effects of the present invention are as follows: The present invention provides a method for extracting the title structure of a tender document based on an OCR engine, which can quickly and accurately extract the title information of the document, improve the efficiency and accuracy of title extraction, and is applicable to various types of documents. At the same time, the present invention can also reduce manual intervention, improve the automatic processing capability, and improve the efficiency of document processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 It is a flowchart of this application; Figure 2is a schematic diagram of an original image file to be processed in an embodiment of the present application; Figure 3 is a schematic diagram of an image file after removing interference information such as a seal in an embodiment of the present application; Figure 4 It is a schematic diagram of the system architecture in an embodiment of the present application. DETAILED DESCRIPTION

[0013] Embodiment 1: like Figure 1 As shown, a method for extracting the title structure of a tender document based on an OCR engine comprises the following steps: S1, convert the tender document file in pdf format into an image file; S2, using OCR recognition to identify the text content of the tender document image file; S3, sorting out the text recognition results of the tender document according to the text block position information of the OCR recognition result, and obtaining the text content of the tender document; S4, extracting the title structure of the paragraph content of the bid document; S5, optimize the title paragraph using header and footer rules and title features.

[0014] Preferably, in step S1, the bid document file in PDF format is converted into an image file using the ffmpeg tool, and the image file is processed to obtain a clean image file; the specific method is: Open the command line interface and enter the command to convert the PDF file into image format; Remove stamps or interfering information: Use image processing algorithms to locate the content of the seal image on the base image; Optimize the seal text in the seal area according to the seal features and remove interference information; Use RGB different level processing tools to clean up the stamp content in the image and obtain clean image content.

[0015] Preferably, in step S2, using OCR recognition technology to perform text content recognition on the tender document image file includes: Select a suitable OCR recognition tool or library to perform text recognition operations on image files and extract text information.

[0016] Preferably, in step S3, arranging the text recognition results of the tender document according to the text block position information of the OCR recognition results, and obtaining the text content of the tender document includes: According to the location information of the text blocks, the tender text content is sorted and extracted to ensure accuracy and completeness.

[0017] Preferably, in step S4, extracting the title structure of the paragraph content of the bid document includes: Through text processing, title information is extracted from text paragraphs and a title structure is established for subsequent processing and analysis.

[0018] Preferably, in step S5, optimizing the title paragraph by using the header and footer rules and the title features includes optimizing the title paragraph according to the header and footer rules and the title features.

[0019] Preferably, in step S4, paragraph title extraction and level classification are performed, and the specific method includes: Paragraph title discovery, using the directory location area to locate and extract title extraction; When there is no table of contents, the title is extracted based on the title features, which include the difference in character size between the upper and lower paragraphs, the paragraph position area, the character height, the difference in the number of characters between the previous and next paragraphs, and the paragraph number; Paragraph title level label classification and recognition, using SVM classification algorithm to classify paragraph title level, paragraph sequence level and paragraph vector analysis.

[0020] Embodiment 2: like Figure 2 As shown, this embodiment provides a specific application process of a method for extracting the title structure of a tender document based on an OCR engine, comprising the following steps: Step 1: Use the ffmpeg tool to convert the bid document file in pdf format into an image file; and perform image processing on the image file to obtain a clean image file. First, through the command line interface, enter the command to convert the bid document file in pdf format into an image format, for example: convert input.pdf into a series of image files, where each page of the pdf file is an image file; execute the command ffmpeg -i input.pdf output.png; then use image processing technology to pre-process the image file and remove stamps or interference information.

[0021] Furthermore, the process of removing the seal or interference information is as follows: Use image processing algorithms to locate the content of the seal image on the base image; Optimize the seal text in the seal area according to the seal features and remove interference information; Use RGB different level processing tools to clean up the stamp content in the image and obtain clean image content.

[0022] Step 2: Use OCR recognition technology to identify the text content of the tender document image file; Select a suitable OCR recognition tool or library to perform text recognition on the image file and extract text information.

[0023] For example, the OCR recognition JSON result of a certain text block is {'ret': [{'rect': {'left': 417, 'top': 133, 'width': 241, 'height': 65}, 'word': 'Transaction completed', 'charset': [{'rect': {'left': 417, 'top': 133, 'width': 50, 'height': 64}, 'prob': 0.999989152, 'word': 'Tra'}, {'rect': {'left': 475, 'top': 133, 'width': 54, 'height': 64}, 'prob': 0.9998947382, 'word': 'ns'}, {'rect': {'left': 538, 'top': 133, 'width': 52, 'height': 64}, 'prob': 0.9999819994, 'word': 'act'}, {'rect': {'left': 598, 'top': 133, 'width': 61, 'height': 64}, 'prob': 0.9999876022, 'word': 'ion'}, {'rect': {'left': 417, 'top': 133, 'width': 50, 'height': 64}, 'prob': 0.999989152, 'word': 'com'}, {'rect': {'left': 475, 'top': 133, 'width': 54, 'height': 64}, 'prob': 0.9998947382, 'word': 'ple'}, {'rect': {'left': 538, 'top': 133, 'width': 52, 'height': 64}, 'prob': 0.9999819994, 'word': 'ted'}], 'line_probability': {'average': 0.9999634027,'min': 0.9998947382, 'variance': 1.577323161e-09}}]}.

[0024] Step 3: Organize the text recognition results of the tender according to the position information of the text blocks in the OCR recognition results, and obtain the tender text content. According to the position information of the text blocks, organize and extract the tender text content to ensure accuracy and integrity.

[0025] As Figure 2 shown, the conversion of a receipt picture in a sample page description of a certain tender into text content is as shown in Table 1 below:

[0026] Table 1: Text information converted from the picture.

[0027] Step 4: Extract the title structure from the content of the bid document text paragraphs. Use text processing technology to extract the title information in the text paragraphs and establish the title structure to facilitate subsequent processing and analysis; Paragraph title discovery, using the directory location area to locate and extract title extraction; When there is no table of contents, title extraction is performed based on title features, such as the difference in character size between the upper and lower paragraphs, paragraph position area (center), character height, difference in the number of characters between the previous and next paragraphs, and paragraph sequence number; Paragraph title level label classification and recognition, using SVM classification algorithm to classify paragraph title level, paragraph sequence level, and paragraph vector analysis (total score and total).

[0028] The title structure of a document is as follows, where the serial number after the title text is the title level, 1 is the level 1 title, 2 is the level 2 title, and so on.

[0029] Step 5: Figure 3 As shown in the figure, the title paragraph is optimized by using the header and footer rules and title features. According to the header and footer rules and title features, the title paragraph is optimized to ensure that the document title structure is clear and easy to understand; the page number '-5-' in the footer is removed.

[0030] Embodiment three: like Figure 4 As shown, this embodiment provides a chart and table content extraction system based on an OCR engine, and the specific architecture is: a document format conversion module, a document content recognition module, a paragraph discovery module, a paragraph title structure extraction module, and a paragraph title structure optimization module; the system is used to execute the described method for extracting the title structure of a bid document based on an OCR engine.

Claims

1. A method for extracting the title structure of a tender document based on an OCR engine, characterized in that: The following steps are involved: S1, convert the tender document file in pdf format into an image file; S2, using OCR recognition to identify the text content of the tender document image file; S3, sorting out the text recognition results of the tender document according to the text block position information of the OCR recognition result, and obtaining the text content of the tender document; S4, extracting the title structure of the paragraph content of the bid document; S5, optimize the title paragraph using header and footer rules and title features.

2. The method for extracting the title structure of a tender document based on an OCR engine according to claim 1, characterized in that: In step S1, the ffmpeg tool is used to convert the tender document file in pdf format into an image file, and the image file is processed to obtain a clean image file; the specific method is: Open the command line interface and enter the command to convert the PDF file into image format; Remove stamps or interfering information: Use image processing algorithms to locate the content of the seal image on the base image; Optimize the seal text in the seal area according to the seal features and remove interference information; Use RGB different level processing tools to clean up the stamp content in the image and obtain clean image content.

3. The method for extracting the title structure of a tender document based on an OCR engine according to claim 1, characterized in that: In step S2, using OCR recognition technology to perform text content recognition on the tender document image file includes: Select a suitable OCR recognition tool or library to perform text recognition operations on image files and extract text information.

4. The method for extracting the title structure of a tender document based on an OCR engine according to claim 1, characterized in that: In step S3, the text recognition results of the tender documents are sorted according to the text block position information of the OCR recognition results, and the tender document text content is obtained, including: According to the location information of the text blocks, the tender text content is sorted and extracted to ensure accuracy and completeness.

5. The method for extracting the title structure of a tender document based on an OCR engine according to claim 1, characterized in that: In step S4, extracting the title structure of the paragraph content of the bid document includes: Through text processing, title information is extracted from text paragraphs and a title structure is established for subsequent processing and analysis.

6. The method for extracting the title structure of a tender document based on an OCR engine according to claim 1, characterized in that: In step S5, optimizing the title paragraph by using the header and footer rules and the title features includes optimizing the title paragraph according to the header and footer rules and the title features.

7. The method for extracting the title structure of a tender document based on an OCR engine according to claim 5, characterized in that: In step S4, paragraph title extraction and level classification are performed, and the specific method includes: Paragraph title discovery, using the directory location area to locate and extract title extraction; When there is no table of contents, the title is extracted based on the title features, which include the difference in character size between the upper and lower paragraphs, the paragraph position area, the character height, the difference in the number of characters between the previous and next paragraphs, and the paragraph number; Paragraph title level label classification and recognition, using SVM classification algorithm to classify paragraph title level, paragraph sequence level and paragraph vector analysis.

Images