OCR extraction optimization method based on layout analysis

Abstract

The invention discloses an OCR (Optical Character Recognition) extraction optimization method based on layout analysis, which relates to the technical field of information processing and comprises the following steps of: preprocessing image data, detecting image layout, identifying and distinguishing area sizes, performing OCR text extraction on each area, and reading and splicing texts in sequence. According to the method, the effects of follow-up OCR recognition and layout analysis can be remarkably improved by optimizing the pixels of the image, the layout detection and the OCR are combined, large and small regions are distinguished according to coordinates, related processing is carried out, information extraction of the complex typesetting image is achieved, the original text sequence and structure can be accurately kept, and the method is high in practicability and easy to popularize. And the accuracy of information extraction is ensured, a better detection effect can be ensured for irregular and fuzzy layout, and information repetition can be avoided.

Description

Technical Field

[0001] The present invention relates to the field of information processing technology, and in particular to an OCR extraction optimization method based on layout analysis. Background Art

[0002] In today's digital age, the digitization and automated processing of information have become key technologies for improving efficiency and service quality in various industries. The digitization and automated processing of information refers to the process of converting information into digital form and automatically processing it through a computer system. The purpose is to improve the efficiency, accuracy and reliability of information processing. Optical character recognition (OCR), as an important tool for converting text content in images into editable text, has been widely used in document digitization, automated data extraction and information retrieval.

[0003] At present, OCR technology often has some limitations in practical applications. It has poor detection effect on some irregular and blurred layouts, repeats the processing of complex areas, adopts a unified processing method for large and small areas without distinguishing them, which will lead to some duplication of information. Especially when processing pictures with complex layouts and formats, it is often impossible to accurately maintain the original text order and structure, thereby affecting the accuracy of information extraction. Therefore, the present invention proposes an OCR extraction optimization method based on layout analysis to solve the problems existing in the prior art. Summary of the invention

[0004] In view of the above problems, the purpose of the present invention is to propose an OCR extraction optimization method based on layout analysis, which solves some limitations of existing OCR technology in practical application, such as poor detection effect for some irregular and blurred layouts, repeated processing for complex areas, and unified processing for large and small areas without distinction, which will lead to some duplication of information.

[0005] In order to achieve the purpose of the present invention, the present invention is implemented by the following technical scheme: an OCR extraction optimization method based on layout analysis, comprising the following steps: Step 1: Optimize the pixels of the image data to be processed to adaptively adjust the image parameters; Step 2: Use the LayoutLMv series model to detect the image layout of the optimized image data, and identify the coordinates and types of different areas in the image; Step 3: For the region coordinates identified on each page of the image layout, traverse and check whether each region meets the inclusion condition. If the inclusion condition is met, the region is considered to be included in another region, that is, the region is a small region; Step 4: Perform OCR text extraction on the traversed and checked areas, and for the included area, subtract the text of the included area from the text of the included area; Step 5: Calculate the center points of each area after text extraction and sort them according to the coordinates, splice the text in order from top to bottom and from left to right, and generate formatted text output.

[0006] A further improvement is that in step 1, the image data is processed by pixel optimization using a programming method, and the specific steps are: first install Python and OpenCV libraries, then use OpenCV's convertScaleAbs function to adjust the image brightness, and then achieve contrast adjustment by changing the image's alpha value.

[0007] A further improvement is that in the step 2, the image data is preprocessed before image layout detection. The specific steps are: first use the image processing library to scale the image data to a size that is compatible with the LayoutLMv series model, then use the image processing library to normalize the scaled image data, and then convert the annotation information of the image data into a format that is compatible with the LayoutLMv series model.

[0008] Further improvement lies in: in the step 2, the specific steps of performing image layout detection on the image data are: first load the pre-trained LayoutLMv series model and processor, then use the processor to pre-process the image data and the annotation information corresponding to the image data, and then input the pre-processed data into the LayoutLMv series model for prediction, decode the prediction results, and obtain the coordinates and types of different areas.

[0009] A further improvement is that in step three, the area coordinates include four vertices and eight coordinate values, and the inclusion condition is that in the order of upper left-upper right-lower right-lower left, the sixth value and the eighth value are less than the preset value, and the remaining values ​​are greater than the preset value.

[0010] A further improvement is that in step 4, the image data is preprocessed before OCR text extraction. The specific steps are: first convert the color image into a grayscale image, then use the median filter method to remove noise points in the image, and then correct the tilt angle of the image.

[0011] A further improvement is that in step 5, when calculating the center point of each area, for each area, the average value of its x and y coordinates is calculated to obtain the coordinates of the center point.

[0012] A further improvement is that in step 5, when the texts are concatenated in sequence, line breaks are not added between paragraphs of the same type, but line breaks are added between paragraphs of different types.

[0013] The beneficial effects of the present invention are as follows: the present invention can significantly improve the effects of subsequent OCR recognition and page layout analysis by optimizing the pixel processing of the image, and combines layout detection with OCR to distinguish large and small areas according to coordinates and perform related processing, thereby realizing information extraction of images with complex typeset layouts, accurately maintaining the original text order and structure, and ensuring the accuracy of information extraction. It can also ensure better detection effects for irregular and blurred layouts, and can avoid information duplication. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0015] Figure 1 It is a schematic diagram of the flow of the OCR extraction optimization method based on layout analysis of the present invention. DETAILED DESCRIPTION

[0016] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0017] Optical character recognition (OCR) is a technology that converts printed or handwritten text into an editable text format. OCR systems usually include two main steps: first, text detection, which is used to locate and extract text areas from images; second, text recognition, which is used to convert the characters in these text areas into a computer-readable form. OCR technology is widely used in many fields, including but not limited to document management, automated data entry, postal services, license plate recognition, and healthcare. With the development of deep learning and artificial intelligence technology, the accuracy and efficiency of OCR are constantly improving, especially for the recognition of complex backgrounds, blurred text, and handwritten fonts. OCR technology is constantly being optimized; In today's digital age, the digitization and automated processing of information has become a key technology for improving efficiency and service quality in various industries. Optical character recognition (OCR), as an important tool for converting text content in images into editable text, has been widely used in document digitization, automated data extraction, information retrieval and other fields; However, existing OCR technology often has some limitations in practical applications. It has poor detection effects on some irregular and blurred layouts, repeated processing of complex areas, and a unified processing method for large and small areas without distinguishing them, which easily leads to some duplication of information.

[0018] For the problems existing in the above-mentioned prior art, see Figure 1 This embodiment provides an OCR extraction optimization method based on layout analysis, comprising the following steps: Step 1: Image data preprocessing Collect the image data to be processed, and perform pixel optimization processing on the image data to adaptively adjust the image parameters. In this embodiment, the exposure and contrast are adjusted. The clarity and other parameters in the image are adaptively adjusted according to the image type. Through the above pixel optimization processing of the image, the effect of subsequent OCR recognition and layout analysis can be significantly improved, especially in terms of exposure and contrast, so as to achieve better visual effects; This embodiment uses a programming method to perform pixel optimization processing. During the optimization processing, Python and OpenCV libraries are first installed, and then the convertScaleAbs function of OpenCV is used to adjust the image brightness to achieve exposure adjustment. Then, the contrast is adjusted by changing the alpha value of the image. An alpha value greater than 1 will increase the contrast, and a value less than 1 will reduce the contrast. After the adjustment is completed, the optimized image is saved; Step 2: Image layout detection Use the image processing library to scale the image data to a size that is compatible with the LayoutLMv series model in advance, then use the image processing library to normalize the scaled image data, and then convert the annotation information of the image data into a format that is compatible with the LayoutLMv series model. The secondary preprocessing of the image data can maintain the consistency of the input data, reduce the amount of calculation for subsequent data processing, and improve data processing efficiency; The LayoutLMv series model (LayoutXLM) is used to detect the image layout of the image data that has been preprocessed twice. The specific steps are as follows: first load the pre-trained LayoutLMv series model and processor, then use the processor to pre-process the image data and the annotation information corresponding to the image data, then input the pre-processed data into the LayoutLMv series model for prediction, and decode the prediction results to identify the coordinates and types of different areas in the image; Step 3: Identify and distinguish the area size The area coordinates in this embodiment include four vertices and eight coordinate values, and the four vertices and eight coordinate values ​​together define a rectangular area; First, for the region coordinates identified on each page of the image layout in step 2, traverse and check whether each region meets the inclusion condition, that is, in the order of upper left-upper right-lower right-lower left, the sixth and eighth values ​​are less than the preset value, and the remaining values ​​are greater than the preset value (the preset value is adaptively selected according to the actual data processing needs). If the inclusion condition is met, the region is considered to be included in another region, that is, the included region is the small region, and the region containing the small region is the large region; Step 4: Perform OCR text extraction on each area First, convert the color image after region recognition in step 3 into a grayscale image, which helps to reduce the complexity of subsequent processing, because the grayscale image has only one channel instead of three channels of the color image. Then, the median filter method is used to remove the noise points in the image, and then the tilt angle of the image is corrected to ensure that the text line is horizontal, which helps to improve the OCR recognition rate. Through the above operations, the image data is preprocessed three times to improve the accuracy of subsequent OCR; Perform OCR text extraction on the image area that has been preprocessed three times. For the included area, subtract the text of the included area from the text of the included area to distinguish between large and small areas and avoid repeated extraction. Step 5: Read and concatenate text in sequence For each area in step 4, calculate the average value of its x and y coordinates, get the coordinates of the center point, and sort them according to the coordinates. The coordinates of the center point are obtained by taking the average value of the minimum and maximum x and y coordinates of the area. Concatenate the text in order from top to bottom and from left to right. Do not add line breaks between paragraphs of the same type, but add line breaks between paragraphs of different types. Concatenate the extracted text into a complete string, generate formatted text output, and implement OCR extraction optimization based on layout analysis.

[0019] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. An OCR extraction optimization method based on layout analysis, characterized in that: The following steps are involved: Step 1: Optimize the pixels of the image data to be processed to adaptively adjust the image parameters; Step 2: Use the LayoutLMv series model to detect the image layout of the optimized image data, and identify the coordinates and types of different areas in the image; Step 3: For the region coordinates identified on each page of the image layout, traverse and check whether each region meets the inclusion condition. If the inclusion condition is met, the region is considered to be included in another region, that is, the region is a small region; Step 4: Perform OCR text extraction on the traversed and checked areas, and for the included area, subtract the text of the included area from the text of the included area; Step 5: Calculate the center points of each area after text extraction and sort them according to the coordinates, splice the text in order from top to bottom and from left to right, and generate formatted text output.

2. The OCR extraction optimization method based on layout analysis according to claim 1, characterized in that: In the step 1, the image data is processed by pixel optimization using a programming method, and the specific steps are: first install Python and OpenCV libraries, then use the convertScaleAbs function of OpenCV to adjust the image brightness, and then achieve contrast adjustment by changing the alpha value of the image.

3. The OCR extraction optimization method based on layout analysis according to claim 1, characterized in that: In the step 2, the image data is preprocessed before image layout detection. The specific steps are: first use the image processing library to scale the image data to a size that is compatible with the LayoutLMv series model, then use the image processing library to normalize the scaled image data, and then convert the annotation information of the image data into a format that is compatible with the LayoutLMv series model.

4. The OCR extraction optimization method based on layout analysis according to claim 1, characterized in that: In the step 2, the specific steps of performing image layout detection on the image data are: first load the pre-trained LayoutLMv series model and processor, then use the processor to pre-process the image data and the annotation information corresponding to the image data, and then input the pre-processed data into the LayoutLMv series model for prediction, decode the prediction results, and obtain the coordinates and types of different areas.

5. The OCR extraction optimization method based on layout analysis according to claim 1, characterized in that: In the step three, the area coordinates include four vertices and eight coordinate values, and the inclusion condition is: in the order of upper left-upper right-lower right-lower left, the sixth value and the eighth value are less than the preset value, and the remaining values ​​are greater than the preset value.

6. The OCR extraction optimization method based on layout analysis according to claim 1, characterized in that: In the step 4, the image data is preprocessed before OCR text extraction. The specific steps are: first convert the color image into a grayscale image, then use the median filter method to remove noise points in the image, and then correct the tilt angle of the image.

7. The OCR extraction optimization method based on layout analysis according to claim 1, characterized in that: In step 5, when calculating the center point of each region, for each region, the average value of its x and y coordinates is calculated to obtain the coordinates of the center point.

8. The OCR extraction optimization method based on layout analysis according to claim 1, characterized in that: In step 5, when the texts are concatenated in sequence, line breaks are not added between paragraphs of the same type, but line breaks are added between paragraphs of different types.

Images