Document analysis method and device, electronic equipment and medium

Abstract

The embodiment of the invention provides a document analysis method and device, electronic equipment and a medium. The document analysis method comprises the steps that an obtained to-be-analyzed document is converted into a document image; performing layout analysis on the document image to obtain at least one layout unit; detecting each layout unit in the at least one layout unit, and determining element information; sorting the element information according to a reading sequence corresponding to the to-be-analyzed document to obtain typesetting information; typesetting the to-be-analyzed document according to the typesetting information; the analyzed document data can be obtained; by performing layout analysis and element detection on the document and sorting the element information according to the reading sequence, the document processing technology can adapt to complex document typesetting and accurately extract and process the document content, so that the accuracy, efficiency and reliability of document processing are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of document parsing, and in particular to a document parsing method, a document parsing device, an electronic device and a computer storage medium. Background Art

[0002] In the context of enterprise digital transformation and knowledge management, document parsing technology has become a bridge between raw data and intelligent applications. Related technologies mainly include rule engine-based methods and template matching-based methods. The rule engine uses predefined business rules and pattern recognition technology to extract document information, while template matching extracts specific information through predefined templates and is suitable for structured or semi-structured documents.

[0003] With the acceleration of digital transformation, document parsing technology plays an increasingly important role in information extraction and knowledge management. The development of deep learning, especially big model technology, has brought new opportunities for document parsing. However, the existing document parsing technology based on big models still has some shortcomings, such as inaccurate document parsing content; reading order errors in rule-based reading order sorting in document parsing, etc. Summary of the invention

[0004] In view of the above problems, embodiments of the present invention are proposed to provide a document parsing method, a document parsing device, an electronic device and a computer storage medium to overcome the above problems or at least partially solve the above problems.

[0005] In a first aspect of the present invention, an embodiment of the present invention discloses a document parsing method, comprising:

[0006] Convert the acquired document to be parsed into a document image;

[0007] Performing layout analysis on the document image to obtain at least one layout unit;

[0008] Detecting each layout unit in the at least one layout unit to determine element information;

[0009] Sorting the element information according to the reading order of the document to be parsed to obtain typesetting information;

[0010] The document to be parsed is typeset according to the typeset information.

[0011] Optionally, before the step of sorting the element information according to the reading order corresponding to the document to be parsed, the method further includes:

[0012] Performing formula recognition on the document image to determine the formula from the document image;

[0013] Sort the element information and the formula according to the reading order corresponding to the document to be parsed to obtain typesetting information;

[0014] The document to be parsed is typeset according to the typeset information.

[0015] Optionally, the step of detecting each layout unit in the at least one layout unit to determine element information includes:

[0016] When the layout unit includes text data, extracting text information from the text data;

[0017] Merging the text information to generate a text block;

[0018] The text block is determined to be element information.

[0019] Optionally, the step of detecting each layout unit in the at least one layout unit to determine element information includes:

[0020] When the layout unit includes an image element, cutting the layout unit to obtain a target image element;

[0021] The target image element is determined to be element information.

[0022] Optionally, the step of detecting each layout unit in the at least one layout unit to determine element information includes:

[0023] When the layout unit includes a table element, performing table recognition on the table element to determine table data information corresponding to the table element;

[0024] The table data information is determined to be element information.

[0025] Optionally, the step of performing table recognition on the table element to obtain table data information corresponding to the table element includes:

[0026] Detecting the table element by using a preset table detection model to determine the table text information and table image information contained in the table element;

[0027] Performing table structure detection on the table elements by using the table detection model to obtain a table structure;

[0028] The table elements are reconstructed according to the table text information, the table image information, the coordinates corresponding to the table text information, the coordinates corresponding to the table image information and the table structure to obtain the table data information corresponding to the table elements.

[0029] Optionally, the step of detecting the table element by using a preset table detection model to determine the table text information and table image information contained in the table element includes:

[0030] When the table element contains table image information, the table element is cropped to determine the table image information contained in the table element;

[0031] When the table element contains table text information, the table element is clipped to determine a target table element;

[0032] Optical character recognition is performed on the target table element to determine the table text information contained in the table element.

[0033] Optionally, before the step of performing layout analysis on the document image, the method further includes:

[0034] Label the collected training documents to obtain a training set;

[0035] The preset layout analysis model is trained using the training set.

[0036] Optionally, the step of labeling the collected documents to obtain a training set includes:

[0037] Perform formula annotation on the collected training documents to obtain a formula training set;

[0038] The layout analysis model is trained for formula detection using the formula training set.

[0039] In a second aspect of the present invention, an embodiment of the present invention discloses a document parsing device, comprising:

[0040] A preprocessing module, used for converting the acquired document to be parsed into a document image;

[0041] A layout analysis module, used for performing layout analysis on the document image to obtain at least one layout unit;

[0042] A detection and identification module, used to detect each layout unit in the at least one layout unit and determine element information;

[0043] A sorting module, used to sort the element information according to the reading order corresponding to the document to be parsed, to obtain typesetting information;

[0044] A typesetting module is used to typeset the document to be parsed according to the typesetting information.

[0045] In the third aspect of the present invention, an embodiment of the present invention further discloses an electronic device, comprising a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the computer program implements the steps of the document parsing method as described above when executed by the processor.

[0046] In a fourth aspect of the present invention, an embodiment of the present invention further discloses a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the document parsing method described above are implemented.

[0047] The embodiment of the present invention includes the following advantages: the embodiment of the present invention converts the acquired document to be parsed into a document image; performs layout analysis on the document image to obtain at least one layout unit; detects each layout unit in at least one layout unit to determine element information; sorts the element information according to the reading order corresponding to the document to be parsed to obtain typesetting information; typesets the document to be parsed according to the typesetting information; and obtains the parsed document data. By performing layout analysis on the document, it can adapt to complex document typesetting such as double-column and multi-column documents; by performing element detection on the document, it can adapt to complex documents including tables, images and mixed texts, and accurately extract document content; sorts the element information according to the reading order corresponding to the document to be parsed, ensuring the logic and coherence of the document content and avoiding errors in the reading order. By performing layout analysis and element detection on the document, and sorting the element information according to the reading order, the document processing technology can adapt to complex document typesetting, accurately extract and process document content, thereby improving the accuracy, efficiency and reliability of document processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0049] Figure 1 is a flowchart of steps of an embodiment of a document parsing method of the present invention;

[0050] Figure 2 is a schematic diagram of performing layout analysis in an embodiment of the present invention;

[0051] Figure 3 is a flowchart of steps of another document parsing method embodiment of the present invention;

[0052] Figure 4is a structural block diagram of an embodiment of a document parsing device;

[0053] Figure 5 is a structural block diagram of an electronic device provided by an embodiment of the present invention;

[0054] Figure 6 It is a structural block diagram of a storage medium provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0055] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0056] With the acceleration of digital transformation, document parsing technology plays an increasingly important role in information extraction and knowledge management. The development of deep learning, especially big model technology, has brought new opportunities for document parsing. However, the existing document parsing technology based on big models still has some shortcomings, such as inaccurate document parsing content; reading order errors in rule-based reading order sorting in document parsing, etc.

[0057] In view of the above problems, embodiments of the present invention are proposed to provide a document parsing method that overcomes the above problems or at least partially solves the above problems.

[0058] Reference Figure 1 , shows a flowchart of steps of an embodiment of a document parsing method of the present invention, which may specifically include:

[0059] Step 101, converting the acquired document to be parsed into a document image;

[0060] For the obtained document to be parsed, the document is first classified, and the format of the document needs to be determined. If the format of the document to be parsed is not a picture format, the document to be parsed is converted into a document image, and pre-processing such as image denoising is performed. Specifically, first, it is determined whether the document to be parsed is in a picture format. If the document to be parsed is not in a picture format, for example, in a PDF (Portable Document Format) format, the document to be parsed is converted into a document image.

[0061] Step 102, performing layout analysis on the document image to obtain at least one layout unit;

[0062] The document image is analyzed by a preset layout analysis model. In an embodiment of the present application, the layout analysis model can be constructed by using YOLOv10 (You Only Look Once, target detection algorithm). The YOLOv10 network optimizes the backbone network, neck network, NMS (Non-Maximum Suppression) problems and loss functions. These optimizations can effectively extract features and reduce the amount of calculation, while improving the detection quality.

[0063] Perform layout analysis on the document image to obtain at least one layout unit, referring to Figure 2 , the layout unit is to frame different units with a layout analysis frame, so that each unit is independent of other units. The layout analysis frame can be a BBox (Bounding Box).

[0064] Step 103, detecting each layout unit in the at least one layout unit to determine element information;

[0065] Each layout unit contains specific elements, including text, images, tables, etc.

[0066] Each layout unit is detected to determine the element information of each layout unit, that is, to determine whether the layout unit contains text, image or table.

[0067] Step 104, sorting the element information according to the reading order corresponding to the document to be parsed to obtain typesetting information;

[0068] The reading order of the document to be parsed is extracted, and the element information is sorted according to the reading order of the document to be parsed, that is, the typesetting information of the element information is obtained according to the reading order of the document to be parsed.

[0069] Step 105: typeset the document to be parsed according to the typesetting information.

[0070] The document to be parsed is typeset according to the typesetting information of the element information, and the parsing of the document to be parsed is completed.

[0071] The embodiment of the present invention converts the acquired document to be parsed into a document image; performs layout analysis on the document image to obtain at least one layout unit; detects each layout unit in the at least one layout unit to determine element information; sorts the element information according to the reading order corresponding to the document to be parsed to obtain typesetting information; typesets the document to be parsed according to the typesetting information; and obtains the parsed document data. By performing layout analysis on the document, it can adapt to complex document typesetting such as double-column and multi-column documents; by performing element detection on the document, it can adapt to complex documents including tables, images, and mixed texts, and accurately extract document content; sorts the element information according to the reading order corresponding to the document to be parsed, ensuring the logic and coherence of the document content and avoiding errors in the reading order. By performing layout analysis and element detection on the document, and sorting the element information according to the reading order, the document processing technology can adapt to complex document typesetting, accurately extract and process document content, thereby improving the accuracy, efficiency and reliability of document processing.

[0072] Reference Figure 3 , shows a flowchart of a document parsing method embodiment of the present invention, which may specifically include the following steps:

[0073] Step 201, converting the acquired document to be parsed into a document image;

[0074] For the obtained documents to be parsed, the documents need to be classified first. For the documents to be parsed, the document format needs to be determined. If the format of the document to be parsed is not a picture format, the document to be parsed is converted into a document image and the document image is preprocessed. Specifically, first, it is determined whether the document to be parsed is in a picture format. If the document to be parsed is not in a picture format, for example, in a Word format, the input text is converted into a picture format.

[0075] In an optional embodiment of the present invention, before the step of performing layout analysis on the document image, the method further includes:

[0076] Sub-step S2011, annotating the collected training documents to obtain a training set;

[0077] Download some data from the public datasets PubLayNet and DocLayNet, and collect various PDF documents or image data within the company, such as papers, corporate monthly magazines, vehicle manuals, body design specifications, resumes, standards and regulations, and other scanned documents as training documents. Then annotate the content of the training documents into categories such as text, pictures, tables, lists, table names, and figure names for training the layout analysis model. Before model training, data enhancement methods such as image denoising, sharpening, Gaussian filtering, and binarization were used to preprocess the data. The final trained model has the function of file layout detection.

[0078] PubLayNet is a large-scale document image dataset that provides detailed annotations of document layouts with bounding boxes and polygon segmentation, providing rich resources and tools for document intelligent processing and academic research. DocLayNet is a large-scale manually annotated document layout segmentation dataset that is suitable for a variety of application scenarios such as document automation processing, information extraction, document classification and retrieval, and machine learning model training.

[0079] Sub-step S2012, training the preset layout analysis model using the training set.

[0080] The layout analysis model is trained by downloading public datasets and collecting document data. Both datasets contain documents from different fields, such as science, finance, and law, which helps train the model to handle complex layout variations in the real world and promote the understanding of complex layout structures such as documents and papers. By collecting a variety of PDF documents or image data within the company, the trained model can better adapt to the company's specific document format and layout, improve the accuracy of recognition and processing, and through automated document analysis, it can be easier to organize and retrieve information in the company's knowledge base.

[0081] Before model training, the document images were preprocessed using data enhancement methods such as image denoising, sharpening, Gaussian filtering and binarization. Operations such as sharpening can improve the clarity of the data, and binarization can remove some watermarks to a certain extent. The trained model has layout detection capabilities and can show high robustness for blurred or watermarked data.

[0082] Step 202, performing layout analysis on the document image to obtain at least one layout unit;

[0083] In an embodiment of the present application, the preset layout analysis model can be a layout analysis model constructed using YOLOv10. The YOLOv10 network is selected mainly because this network optimizes the backbone network, the neck network, the NMS problem and the loss function. These optimizations can effectively extract features and reduce the amount of calculation, while improving the detection quality. The preset layout analysis model can also be other models, which are not specifically limited in this application.

[0084] Step 203, detecting each layout unit in the at least one layout unit to determine element information;

[0085] Each layout unit contains specific elements, including text, images, tables, etc.

[0086] Each layout unit is detected to determine the element information of each layout unit, that is, to determine whether the layout unit contains text, image or table.

[0087] In an optional embodiment of the present invention, the step of detecting each layout unit in the at least one layout unit to determine the element information includes:

[0088] Sub-step S11, when the layout unit includes text data, extracting text information from the text data;

[0089] In this application, the method of PaddleOCR is used for text detection and recognition. PaddleOCR is an open source OCR (Optical Character Recognition) tool with rich API interfaces. In this application, the OCR of PaddleOCR is modified and integrated into the document parsing system, and support for multiple languages ​​can be achieved through the configuration file.

[0090] PaddleOCR is used to detect and recognize text. When it is detected that a layout unit includes text data, the layout unit is cropped to remove the excess part and retain the part where the text data is located. The text data is then subjected to OCR recognition to extract the text information in the text data.

[0091] Sub-step S12, merging the text information to generate a text block;

[0092] The text information extracted from the text data is merged to generate text blocks. The text blocks retain the original text order of the document to be parsed.

[0093] Sub-step S13, determining that the text block is element information.

[0094] Identify the text block as element information and participate in subsequent sorting.

[0095] In an optional embodiment of the present invention, the step of detecting each layout unit in the at least one layout unit to determine the element information includes:

[0096] Sub-step S211, when the layout unit includes an image element, cutting the layout unit to obtain a target image element;

[0097] Sub-step S22, determining that the target image element is element information.

[0098] The target image elements are determined as element information and participate in subsequent sorting.

[0099] In an optional embodiment of the present invention, the step of detecting each layout unit in the at least one layout unit to determine the element information includes:

[0100] Sub-step S31, when the layout unit includes a table element, performing table recognition on the table element to determine the table data information corresponding to the table element; including:

[0101] Sub-step S311, detecting the table element by using a preset table detection model to determine the table text information and table image information contained in the table element;

[0102] Due to internal company data, we collected and annotated internal company table data while collecting public data sets. We fine-tuned PaddleOCR's original table recognition so that it can recognize charts or pictures in cells during the table recognition stage.

[0103] When the table element contains table image information, the table element is cropped to determine the table image information contained in the table element; and the table image information is saved;

[0104] When the table element contains table text information, the table element is clipped to determine the target table element;

[0105] Optical character recognition is performed on the target table element to determine the table text information contained in the table element.

[0106] Through cropping and recognition, the table image information and table text information contained in the table element are obtained.

[0107] Sub-step S312, performing table structure detection on the table elements through a table detection model to obtain a table structure;

[0108] The table structure includes the number of table rows and columns, and the position coordinates corresponding to the table image information and the table text information are represented by the number of rows and columns.

[0109] The preset table detection model can be a trained Unet model. The UNet model adopts a unique "U"-shaped structure, which consists of two parts: an encoder (downsampling path) and a decoder (upsampling path). The encoder is responsible for extracting the features of the input image, while the decoder restores the feature map to the size of the original input image through upsampling operations and gradually generates segmentation results. The key to UNet is the introduction of skip connections in the decoder, which connects the feature map in the encoder with the corresponding feature map in the decoder, helping the decoder to better utilize feature information at different levels, thereby improving the accuracy of image segmentation and detail retention. The working principle of UNet is based on the encoder (gradually reducing the size of the image through convolutional layers and pooling layers while extracting features) and the decoder (gradually restoring the image size through transposed convolutional layers or upsampling, and merging features from the encoder to generate the final segmented image).

[0110] By collecting and labeling the internal tabular data of the enterprise, the Unet model is trained so that the Unet model can adapt to the internal tabular data of the enterprise.

[0111] Sub-step S313, reconstructing the table elements according to the table text information, the table image information, the coordinates corresponding to the table text information, the coordinates corresponding to the table image information and the table structure, to obtain the table data information corresponding to the table elements.

[0112] The table elements are detected by the preset table detection model, and the table text information, table image information, coordinates corresponding to the table text information, coordinates corresponding to the table image information and table structure contained in the table elements are obtained. Based on these data, the table elements are reconstructed to obtain the table data information corresponding to the table elements. The obtained table data information is saved in HTML, Latex or Excel.

[0113] Sub-step S32, determining that the table data information is element information.

[0114] Determine that the table data information is element information and participate in subsequent sorting.

[0115] In an optional embodiment of the present invention, before the step of sorting the element information according to the reading order corresponding to the document to be parsed, the step further includes:

[0116] Step 204, performing formula recognition on the document image, and determining a formula from the document image;

[0117] Determining formulas from document images is mainly divided into two parts: formula detection and formula recognition. Formula detection is a detection model built based on YOLOv10 on the basis of layout analysis.

[0118] In an optional embodiment of the present invention, the step of labeling the collected documents to obtain a training set includes:

[0119] Perform formula annotation on the collected training documents to obtain a formula training set;

[0120] The layout analysis model is trained for formula detection using the formula training set.

[0121] In formula detection, various documents with formulas are collected and annotated as formula training sets, and then the formula training sets are used for model training. The final model has good robustness for formula detection.

[0122] Formula recognition uses UniMERNet, an algorithm designed for formula recognition in real-world scenarios. UniMERNet is a deep learning model designed for mathematical expression recognition. It can convert mathematical expressions in images into LaTeX code and is applicable to a wide range of real-world scenarios. Using the formula training set, we fine-tuned the original UniMERNet and used the formula training set to train the model so that the model can adapt to the formula data within the enterprise. The results show that this method has good performance in complex and long formulas, handwritten formulas, and noisy screenshot formulas, and can also adapt well to the formula data within the enterprise.

[0123] Determine the table data information as element information and participate in subsequent sorting.

[0124] Step 205, sorting the element information and the formula according to the reading order corresponding to the document to be parsed to obtain typesetting information;

[0125] Reading order restoration based on document parsing requires restoring the reading order in different paragraphs or different layout analysis results. Therefore, in the reading order extraction stage, the layout analysis results and OCR results are used to extract the reading order.

[0126] In the above steps, the document image is analyzed for layout, at least one layout unit is obtained, the element information in the layout unit is determined, the element information includes text data, image elements and table elements, the text data in the layout unit is OCRed and the text data is combined into a text block. The text blocks, table elements and image elements are sorted according to the reading order of the original document, and the BBox of each text block, table element and image element is matched one by one, thereby forming the reading order of the entire document.

[0127] Extraction of the original document reading order and sorting according to the original document reading order are achieved through a preset sorting model.

[0128] The sorting model is built based on the seq2seq model, which mainly consists of a text encoder, a layout encoder, a joint encoder, and a reading order predictor. The encoder part uses LayoutLM as the basis. LayoutLM is a pre-trained language model designed for document layout analysis, which can simultaneously process text content and its position information in the document. In the encoding stage, the encoder is responsible for encoding the input sequence, that is, the text content in the document and its layout position. Specifically, the encoder packages the input sequence and the target sequence into a continuous input sequence, and designs a self-attention mask to control the visibility between tokens. LayoutReader allows tokens in the source sequence to pay attention to each other, while preventing tokens in the target sequence from paying attention to the right context. In the decoding stage, the decoder is used to predict the index in the source sequence to generate the correct reading sequence. Since the source and target sequences are reordered sequences, the prediction candidates can be limited to the source sequence. The decoder uses probability calculation to determine the position of each word in the target sequence, which involves the calculation of the input embedding and hidden state of the source sequence.

[0129] During the sorting model training process, the training set is divided into a training set and a test set at a ratio of 9:1, and then the training set is input into the model for training. Through the training of the sorting model, the sorting model can process complex documents and identify the reading order of complex documents.

[0130] Step 206: typeset the document to be parsed according to the typesetting information.

[0131] According to the typesetting information, the document to be parsed is typeset, the parsing of the document to be parsed is completed, and the output is json or markdown.

[0132] The embodiment of the present invention converts the acquired document to be parsed into a document image; performs layout analysis on the document image to obtain at least one layout unit; detects each layout unit in the at least one layout unit to determine element information; sorts the element information according to the reading order corresponding to the document to be parsed to obtain typesetting information; typesets the document to be parsed according to the typesetting information; and obtains the parsed document data. By performing layout analysis on the document, it can adapt to complex document typesetting such as double-column and multi-column documents; by performing element detection on the document, it can adapt to complex documents including tables, images, and mixed texts, and accurately extract document content; sorts the element information according to the reading order corresponding to the document to be parsed, ensuring the logic and coherence of the document content and avoiding errors in the reading order. By performing layout analysis and element detection on the document, and sorting the element information according to the reading order, the document processing technology can adapt to complex document typesetting, accurately extract and process document content, thereby improving the accuracy, efficiency and reliability of document processing.

[0133] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0134] Reference Figure 4 , shows a structural block diagram of an embodiment of a document parsing device of the present invention, which may specifically include:

[0135] A preprocessing module 401 is used to convert the acquired document to be parsed into a document image;

[0136] A layout analysis module 402, configured to perform layout analysis on the document image to obtain at least one layout unit;

[0137] A detection and identification module 403, configured to detect each layout unit in the at least one layout unit and determine element information;

[0138] A sorting module 404 is used to sort the element information according to the reading order of the document to be parsed to obtain typesetting information;

[0139] The typesetting module 405 is used to typeset the document to be parsed according to the typesetting information.

[0140] In an optional embodiment of the present invention, the device further comprises:

[0141] The formula recognition module is used to perform formula recognition on the document image and determine the formula from the document image.

[0142] In an optional embodiment of the present invention, the detection and identification module 403 includes:

[0143] A text data detection submodule, for extracting text information from the text data when the layout unit includes text data;

[0144] In an optional embodiment of the present invention, the text data detection submodule includes:

[0145] The fusion unit is used to fuse the text information to generate a text block.

[0146] In an optional embodiment of the present invention, the detection and identification module 403 includes:

[0147] The image element detection submodule is used to crop the layout unit to obtain the target image element when the layout unit includes the image element.

[0148] In an optional embodiment of the present invention, the detection and identification module 403 includes:

[0149] A table element identification submodule, used for, when the layout unit includes a table element, performing table identification on the table element and determining table data information corresponding to the table element;

[0150] In an optional embodiment of the present invention, the table element identification submodule includes:

[0151] A table element detection unit, used to detect the table element by using a preset table detection model, and determine the table text information and table image information contained in the table element;

[0152] A table result detection unit, used for performing table structure detection on the table elements through the table detection model to obtain a table structure;

[0153] A table reconstruction unit is used to reconstruct the table elements according to the table text information, the table image information, the coordinates corresponding to the table text information, the coordinates corresponding to the table image information and the table structure to obtain the table data information corresponding to the table elements.

[0154] In an optional embodiment of the present invention, the table element detection unit includes:

[0155] A table image information detection subunit, used for, when the table element contains table image information, cutting the table element and determining the table image information contained in the table element;

[0156] A table text information detection subunit, used for cutting the table element when the table element contains table text information, and determining a target table element;

[0157] The optical character recognition subunit is used to perform optical character recognition on the target table element to determine the table text information contained in the table element.

[0158] In an optional embodiment of the present invention, the device further comprises:

[0159] The annotation module is used to annotate the collected training documents to obtain a training set;

[0160] The training module is used to train the preset layout analysis model through the training set.

[0161] In an optional embodiment of the present invention, the marking module further includes:

[0162] The formula annotation submodule is used to annotate the collected training documents with formulas to obtain a formula training set;

[0163] The formula training submodule is used to perform formula detection training on the layout analysis model through the formula training set.

[0164] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0165] Reference Figure 5 , an embodiment of the present invention further provides an electronic device, including:

[0166] A processor 501 and a storage medium 502, wherein the storage medium 502 stores a computer program executable by the processor 501, and when the electronic device is running, the processor 501 executes the computer program to perform the document parsing method as described in any one of the embodiments of the present invention.

[0167] The memory 501 may include a random access memory (RAM) or a non-volatile memory, such as at least one disk memory. Optionally, the memory 501 may also be at least one storage device located away from the aforementioned processor.

[0168] Reference Figure 6The embodiment of the present invention further provides a computer-readable storage medium 601, on which a computer program is stored. When the computer program is executed by a processor, the document parsing method as described in any one of the embodiments of the present invention is executed.

[0169] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0170] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0171] Those skilled in the art will readily appreciate other embodiments of the invention after considering the specification and practicing the invention disclosed herein. The present invention is intended to cover any variations, uses or adaptations of the present invention that follow the general principles of the present invention and include common knowledge or customary techniques in the art that are not disclosed in this disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present invention are indicated by the following claims.

[0172] It should be understood that the present invention is not limited to the exact construction that has been described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

[0173] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

[0174] The document parsing method, the document parsing device, the electronic device and the computer-readable storage medium provided by the present invention are introduced in detail above. The principle and implementation mode of the present invention are explained by using specific examples in this article. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation mode and the scope of application. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A document parsing method, characterized in that: The method comprises: Convert the acquired document to be parsed into a document image; Performing layout analysis on the document image to obtain at least one layout unit; Detecting each layout unit in the at least one layout unit to determine element information; Sorting the element information according to the reading order of the document to be parsed to obtain typesetting information; The document to be parsed is typeset according to the typeset information.

2. The document parsing method according to claim 1, characterized in that: Before the step of sorting the element information according to the reading order corresponding to the document to be parsed, the method further includes: Performing formula recognition on the document image to determine the formula from the document image; Sort the element information and the formula according to the reading order corresponding to the document to be parsed to obtain typesetting information; The document to be parsed is typeset according to the typeset information.

3. The document parsing method according to claim 2, characterized in that: The step of detecting each layout unit in the at least one layout unit to determine element information comprises: When the layout unit includes text data, extracting text information from the text data; Merging the text information to generate a text block; The text block is determined to be element information.

4. The document parsing method according to claim 2, characterized in that: The step of detecting each layout unit in the at least one layout unit to determine element information comprises: When the layout unit includes an image element, cutting the layout unit to obtain a target image element; The target image element is determined to be element information.

5. The document parsing method according to claim 2, characterized in that: The step of detecting each layout unit in the at least one layout unit to determine element information comprises: When the layout unit includes a table element, performing table recognition on the table element to determine table data information corresponding to the table element; The table data information is determined to be element information.

6. The document parsing method according to claim 5, characterized in that: The step of performing table recognition on the table element to obtain table data information corresponding to the table element includes: Detecting the table element by using a preset table detection model to determine the table text information and table image information contained in the table element; Performing table structure detection on the table elements by using the table detection model to obtain a table structure; The table elements are reconstructed according to the table text information, the table image information, the coordinates corresponding to the table text information, the coordinates corresponding to the table image information and the table structure to obtain the table data information corresponding to the table elements.

7. The document parsing method according to claim 6, wherein the step of detecting the table element by using a preset table detection model to determine the table text information and table image information contained in the table element comprises: When the table element contains table image information, the table element is cropped to determine the table image information contained in the table element; When the table element contains table text information, the table element is clipped to determine a target table element; Optical character recognition is performed on the target table element to determine the table text information contained in the table element.

8. The document parsing method according to claim 2, characterized in that: Before the step of performing layout analysis on the document image, the method further comprises: Label the collected training documents to obtain a training set; The preset layout analysis model is trained using the training set.

9. The document parsing method according to claim 8, characterized in that: The step of labeling the collected documents to obtain a training set includes: Perform formula annotation on the collected training documents to obtain a formula training set; The layout analysis model is trained for formula detection using the formula training set.

10. A document parsing device, characterized in that: include: A preprocessing module, used for converting the acquired document to be parsed into a document image; A layout analysis module, used for performing layout analysis on the document image to obtain at least one layout unit; A detection and identification module, used to detect each layout unit in the at least one layout unit and determine element information; A sorting module, used to sort the element information according to the reading order corresponding to the document to be parsed, to obtain typesetting information; A typesetting module is used to typeset the document to be parsed according to the typesetting information.

11. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the document parsing method as described in any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that: When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the document parsing method as described in any one of claims 1 to 9.

Images