Visual modeling-based official document information extraction method and system

Abstract

The invention relates to the technical field of information, in particular to an official document information extraction method and system based on visual modeling. The system comprises an official document image conversion module, a visual feature extraction module and a key information extraction module, and the official document image conversion module is used for converting an input official document into an image according to pages; the visual feature extraction module is used for extracting visual features in the converted official document image; the key information extraction module is used for identifying and extracting key information fields in the official document image. According to the official document information extraction method and system based on visual modeling, on the basis that visual feature modeling is carried out on official documents of different formats and types, simple, efficient and accurate information extraction is carried out according to visual features of extracted content fields.

Description

Technical Field

[0001] The present invention relates to the field of information technology, and particularly to a method and system for extracting official document information based on visual modeling. Background Art

[0002] With the continuous development of the information society, especially within organizations such as government agencies, enterprises, and institutions, the demand for official document management and information processing is increasing day by day. Traditional official document processing methods mostly rely on manual operations, which are not only inefficient but also easily limited by the subjectivity and omissions of manual processing. Especially when faced with a large amount of official document data, how to quickly and accurately extract key information has become a technical problem that亟待解决 (needs to be urgently solved).

[0003] Currently, existing extraction methods include those based on regular expressions and those based on natural language deep learning models; among them, the extraction method based on regular expressions identifies key information fields in official documents by matching specific text patterns. Regular expressions are a text character pattern-based search and extraction technology, widely used in tasks such as information extraction and text analysis. For structured or semi-structured text content (such as numbers, dates, units, etc. in official documents), the regular expression-based extraction method is relatively simple, but it has the following deficiencies: 1. Strong format dependence: Regular expressions usually rely on the fixed format and layout of the document, while official document formats often have various typesetting and format styles. Different units or departments may use different number formats, date formats, or title formats, resulting in poor applicability of regular expressions in different official documents. For example, the date in some official documents is represented as "October 1, 2024", while in others it may be "October 1st, 2024". Without regular rules for different formats, the extraction results will be incorrect.

[0004] 2. Poor flexibility: Regular expressions usually extract based on specific character patterns and do not have the ability to understand context, making it difficult to handle context changes in the content of official documents. When the key information in the document appears in different ways, regular expressions may not be able to handle it. For example, the issuer information may be presented as "Issuer: Zhang San" or "Issued by - Zhang San" in different official documents, and regular expressions are difficult to adjust dynamically to adapt to these changes.

[0005] 3. High maintenance cost: Due to changes in official document formats and content, the regular expression-based extraction method needs to continuously adjust and maintain rules according to new document formats, resulting in poor scalability. Especially when the document format changes significantly, the workload of rule maintenance and update is large.

[0006] Document extraction techniques based on Natural Language Processing (NLP) and deep learning primarily rely on training deep learning models (such as BERT) to identify and extract key information from documents. These models are typically trained on large corpora to learn the semantics and context of language, thereby understanding the content and structure of documents. However, they suffer from the following shortcomings: 1. Reliance on large amounts of labeled data: The performance of deep learning models typically depends on training with large amounts of labeled data. For document information extraction tasks, a large amount of labeled document data is needed to train the model to achieve good extraction results. However, the acquisition and preparation of labeled data often requires high labor costs and time investment, especially for documents with complex formats and diverse structures, which further increases the difficulty and cost of labeling.

[0007] 2. Insufficient Model Generalization Ability: Deep learning models perform well when processing highly structured and standardized text, but their generalization ability may be insufficient for official documents with significant format variations and semantic diversity. For example, some official documents may use relatively obscure terminology or complex sentence structures, leading to biases or errors in the model's understanding. Furthermore, the rules for extracting key information may differ between different types of official documents, requiring additional fine-tuning and optimization when dealing with different types of documents.

[0008] In summary, both existing extraction techniques have their own advantages, but their shortcomings are also quite obvious. Technique one is relatively universal and simple, but suffers from strong format dependence, poor flexibility, and high maintenance costs. Technique two uses a deep learning model for named entity extraction, achieving high accuracy within the document formats covered by the training data; however, training this model requires extensive data annotation.

[0009] Therefore, designing a method and system for extracting official document information based on visual modeling that can solve the above-mentioned technical problems is a technical issue that needs to be addressed. Summary of the Invention

[0010] To address the aforementioned problems, the present invention aims to provide a method and system for extracting official document information based on visual modeling. This method and system extracts information simply, efficiently, and accurately based on the visual characteristics of the extracted content fields, after performing visual feature modeling on official documents of different formats.

[0011] In a first aspect, the present invention provides a document information extraction system based on visual modeling, including a document image conversion module, a visual feature extraction module, and a key information extraction module; The document image conversion module is used to convert the input document into images page by page; The visual feature extraction module is used to extract visual features from the converted document image; The key information extraction module is used to identify and extract key information fields from official document images.

[0012] Furthermore, the key information extraction module designs rules to locate key information fields based on the document's layout and characteristics; it sets feature matching combinations for the main key information fields of official documents for information extraction from mainstream format official documents.

[0013] Furthermore, the key information extraction module also includes a sub-module for creating custom feature matching combinations for specified fields, which allows users to extend extraction capabilities.

[0014] Furthermore, the visual feature extraction module mainly extracts features including text color, line height, layout information, and vertical and horizontal spacing.

[0015] Secondly, the present invention provides a method for extracting official document information based on visual modeling, comprising the following steps: Step S1: The document image conversion module converts the input OFD, PDF, and CEB format documents into images page by page; Step S2: The visual feature extraction module extracts the visual features from the converted document image; Step S3: The key information extraction module uses the visual features extracted by the visual feature extraction module to identify and extract key information fields in the document by specifying feature matching combinations of different key information fields.

[0016] Furthermore, in step S2, the extraction of visual features is specifically as follows: Step S21: Use OCR technology to detect and recognize text region blocks, and combine the recognized and detected text blocks into rows to obtain a list of rows contained in each page, including block box information and text information; Step S22: Based on the representation result of the row list, use color analysis method to obtain the color information of each text row; Step S23: Calculate the vertical spacing of each line of text based on the block box information to obtain the line height feature and the vertical spacing feature between lines; Step S24: Determine the text layout by calculating the width of the text line area block and the left side of the text line in the X-axis direction.

[0017] Furthermore, in step S22, the color information of each text line is obtained as follows: Calculate the RGB value of each pixel and perform color clustering to determine the color information of the text in that line.

[0018] Furthermore, in step S24, the text layout is specifically as follows: based on text alignment rules, determine whether the line of text is left-aligned, right-aligned, centered, or indented.

[0019] The present invention has the following beneficial effects: This invention overcomes the limitations of traditional text-based extraction techniques through image processing and visual feature analysis, enabling efficient and accurate extraction of key information from official documents. Compared to existing technologies, this invention offers greater format adaptability, handling various document formats (such as OFD, PDF, CEB, scanned documents, etc.), effectively reducing the complexity of document information extraction. Simultaneously, this invention improves the accuracy and efficiency of document information extraction. By reducing manual intervention and lowering costs, it optimizes the document processing workflow, enhances the level of intelligence, and provides a more reliable solution for automated document processing in government, enterprise, and other fields. Attached Figure Description

[0020] Figure 1 This is a schematic diagram of Embodiment 1 of the present invention; Figure 2 This is a schematic diagram of Embodiment 2 of the present invention. Detailed Implementation

[0021] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments: Example 1 like Figure 1 As shown, this embodiment provides a document information extraction system based on visual modeling, such as... Figure 1 As shown, It includes a document image conversion module, a visual feature extraction module, and a key information extraction module; The document image conversion module is used to convert the input document into images page by page for subsequent visual feature extraction and information extraction. The visual feature extraction module is used to extract visual features from the converted document image, so as to identify key information fields in the document through the key information extraction module. The key information extraction module is used to identify and extract key information fields from official document images.

[0022] Furthermore, the key information extraction module designs rules to locate key information fields based on the document's layout and characteristics. For example, the characteristic combination of an official document header can be defined as follows: located on the first page of the document, located at the top of the page, with a line height greater than the average line height of the page, in red color, and with top and bottom margins greater than the average line height of the page. The characteristic combination of an official document number can be defined as follows: located after the beginning of the document, with a line height less than or equal to the average line height, and centered.

[0023] Set feature matching combinations for key information fields of official documents (such as: official header, issuing unit, document number, document title, addressee, signatory, etc.) for information extraction of mainstream format official documents.

[0024] Furthermore, the key information extraction module also includes a sub-module for creating custom feature matching combinations for specified fields, which allows users to extend extraction capabilities.

[0025] Furthermore, the visual feature extraction module mainly extracts features including text color, line height, layout information (indentation, alignment, whole line), and vertical and horizontal spacing.

[0026] Example 2 like Figure 2 As shown, this embodiment provides a method for extracting official document information based on visual modeling, including the following steps: Step S1: The document image conversion module converts the input OFD, PDF, and CEB format documents into images page by page; Step S2: The visual feature extraction module extracts the visual features from the converted document image; Step S3: The key information extraction module uses the visual features extracted by the visual feature extraction module to identify and extract key information fields in the official document (such as: red header, issuing unit, document number, document title, addressee, signatory, etc.) by specifying feature matching combinations of different key information fields.

[0027] Furthermore, in step S2, the extraction of visual features is specifically as follows: Step S21: Use OCR technology (such as edge detection, connected component analysis) to detect and recognize text region blocks, and combine the recognized and detected text blocks in rows to obtain a list of rows contained in each page, including block box information (4-point coordinates on the image) and text information. Step S22: Based on the representation result of the row list, use color analysis method to obtain the color information of each text row; Step S23: Calculate the vertical spacing of each line of text based on the block box information to obtain the line height feature and the vertical spacing feature between lines; Step S24: Determine the text layout by calculating the width of the text line area block and the left side of the text line in the X-axis direction.

[0028] Furthermore, in step S22, the color information of each text line is obtained as follows: Calculate the RGB value of each pixel and perform color clustering to determine the color information of the text in that line.

[0029] Furthermore, in step S24, the text layout is specifically as follows: based on text alignment rules, determine whether the line of text is left-aligned, right-aligned, centered, or indented.

[0030] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0031] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0032] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0033] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0034] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any other way. Any person skilled in the art may make changes or modifications to the above-disclosed technical content to create equivalent embodiments. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A document information extraction system based on visual modeling, characterized in that, It includes a document image conversion module, a visual feature extraction module, and a key information extraction module; The document image conversion module is used to convert the input document into images page by page; The visual feature extraction module is used to extract visual features from the converted document image; The key information extraction module is used to identify and extract key information fields from official document images.

2. The document information extraction system based on visual modeling according to claim 1, characterized in that, The key information extraction module designs rules to locate key information fields based on the layout and characteristics of the document; it sets feature matching combinations for the main key information fields of the official document for information extraction from mainstream format documents.

3. The document information extraction system based on visual modeling according to claim 2, characterized in that, The key information extraction module also includes: Create a custom feature matching combination submodule for specified fields, allowing users to extend extraction capabilities.

4. The document information extraction system based on visual modeling according to claim 1, characterized in that, The visual feature extraction module mainly extracts features including text color, line height, layout information, and vertical and horizontal spacing.

5. A method for extracting official document information based on visual modeling, characterized in that, The following steps are involved: Step S1: The document image conversion module converts the input OFD, PDF, and CEB format documents into images page by page; Step S2: The visual feature extraction module extracts the visual features from the converted document image; Step S3: The key information extraction module uses the visual features extracted by the visual feature extraction module to identify and extract key information fields in the document by specifying feature matching combinations of different key information fields.

6. The method for extracting official document information based on visual modeling according to claim 5, characterized in that, Step S2, the extraction of visual features, is as follows: Step S21: Use OCR technology to detect and recognize text region blocks, and combine the recognized and detected text blocks into rows to obtain a list of rows contained in each page, including block box information and text information; Step S22: Based on the representation result of the row list, use color analysis method to obtain the color information of each text row; Step S23: Calculate the vertical spacing of each line of text based on the block box information to obtain the line height feature and the vertical spacing feature between lines; Step S24: Determine the text layout by calculating the width of the text line area block and the left side of the text line in the X-axis direction.

7. The method for extracting official document information based on visual modeling according to claim 6, characterized in that, In step S22, the color information of each text line is obtained as follows: Calculate the RGB value of each pixel and perform color clustering to determine the color information of the text in that line.

8. The method for extracting official document information based on visual modeling according to claim 6, characterized in that, In step S24, the text is formatted as follows: Based on text alignment rules, determine whether the text in the line is left-aligned, right-aligned, centered, or indented.

Images