Document content extraction method, device and equipment and computer readable storage medium

Abstract

The invention relates to a document content extraction method, device and equipment and a computer readable storage medium. A to-be-extracted target document is obtained, the target document is classified, the document type of the target document is determined, the document type of the target document comprises a portable document format type, a character type, a picture type, an office file type and a hypertext markup language type, and the target document is extracted based on the document type of the target document. And extracting the document content of the target document by adopting an extraction mode corresponding to the document type to obtain a target format document. Compared with the prior art, the document extraction speed can be increased, documents in any format can be compatible, all documents can be converted into the target format, all the documents can be compatible, the character attribute information, the category information, the position information and the background information of any document are reserved, and the document extraction efficiency is improved. And the loss of original information of the document is reduced.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a document content extraction method, device, equipment and computer-readable storage medium. Background Art

[0002] In any robot service or artificial intelligence service, there is a need to utilize the information in existing unstructured documents, such as pdf, doc or docx, as well as information in text and pictures. When artificial intelligence uses these unstructured data, it needs to first extract the text and structure information from these data.

[0003] However, the formats of these data are diverse. For example, the PDF format includes scanned PDF, screenshot mixed PDF, and text PDF. Doc is similar to docx, and there are other marked documents such as Hypertext Markup Language (HTML) and Lightweight Markup Language (markdown) files, which brings some challenges to the extraction of document content.

[0004] In the related art, the following methods are usually used to extract document content: 1) convert the original files into images, and then perform optical character recognition (OCR) on the images; 2) use the visual big model to convert the original files into images and then into markdown; 3) parse various format files independently into text txt format. However, the above methods have some disadvantages. The disadvantage of method 1) is that it relies too much on OCR, which not only reduces the speed of extracting some data that was originally text-based, but also has extremely high requirements for OCR. For example, for some complex words and uncommon formula tables, OCR will lose information and also discard some non-text image information; the disadvantage of method 2) is that the calculation amount is too large and the speed is too slow, and it cannot express complex layout data, such as posters; the disadvantage of method 3) is that too much information is lost, and only some formatted text parts can be extracted, and visual information such as titles are lost, and complex layout data such as multi-column data cannot be extracted. In order to lower the threshold for intelligent services to use these document information, it is urgent to provide an extraction method to reduce these document information to a unified form of data, improve the speed of document extraction, and reduce the loss of original document information. Summary of the invention

[0005] In order to solve the above technical problems, the present disclosure provides a document content extraction method, device, equipment and computer-readable storage medium, which can improve the speed of document extraction and reduce the loss of original document information.

[0006] In a first aspect, an embodiment of the present disclosure provides a method for extracting document content, the method comprising:

[0007] Get the target document to be extracted;

[0008] Classifying the target document to determine the document type of the target document, wherein the document type of the target document includes a portable document format type, a text type, a picture type, an office file type, and a hypertext markup language type;

[0009] Based on the document type of the target document, extracting the document content of the target document using an extraction method corresponding to the document type to obtain a document in a target format;

[0010] The target format document includes text attribute information, category information, location information, and background information of each region.

[0011] In a second aspect, an embodiment of the present disclosure provides a document content extraction device, the device comprising:

[0012] An acquisition module is used to acquire the target document to be extracted;

[0013] A classification module, used to classify the target document and determine the document type of the target document, wherein the document type of the target document includes a portable document format type, a text type, a picture type, an office file type, and a hypertext markup language type;

[0014] An obtaining module, used for extracting the document content of the target document based on the document type of the target document by adopting an extraction method corresponding to the document type to obtain a document in a target format;

[0015] The target format document includes text attribute information, category information, location information, and background information of each region.

[0016] In a third aspect, an embodiment of the present disclosure provides an electronic device, including:

[0017] Memory;

[0018] Processor; and

[0019] Computer programs;

[0020] The computer program is stored in the memory and is configured to be executed by the processor to implement the method as described in the first aspect.

[0021] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to implement the method as described in the first aspect.

[0022] In a fifth aspect, an embodiment of the present disclosure further provides a computer program product, which includes a computer program or instructions, and when the computer program or instructions are executed by a processor, the method described in the first aspect is implemented.

[0023] The document content extraction method, device, equipment and computer-readable storage medium provided by the embodiments of the present disclosure obtain the target document to be extracted, classify the target document, determine the document type of the target document, and the document type of the target document includes a portable document format type, a text type, a picture type, an office file type, and a hypertext markup language type. Based on the document type of the target document, the document content of the target document is extracted using an extraction method corresponding to the document type to obtain a target format document. Compared with the prior art, the embodiments of the present disclosure can increase the speed of document extraction by extracting the document content of the target document using an extraction method corresponding to the document type, and can be compatible with documents of any format. Any document can be converted into a target format, so that it is compatible with all documents, and retains the text attribute information, category information, location information, and background information of any document, thereby reducing the loss of original document information. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0025] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0026] Figure 1 A flow chart of a document content extraction method provided in an embodiment of the present disclosure;

[0027] Figure 2 A schematic diagram of a complete process of extracting document content provided by an embodiment of the present disclosure;

[0028] Figure 3 A flow chart of a document content extraction method provided by another embodiment of the present disclosure;

[0029] Figure 4 A flow chart of a document content extraction method provided by another embodiment of the present disclosure;

[0030] Figure 5 A schematic diagram of dividing a page of a target document into regions provided in an embodiment of the present disclosure;

[0031] Figure 6 A schematic diagram of the structure of a document content extraction device provided in an embodiment of the present disclosure;

[0032] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0033] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0034] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.

[0035] In any robot service or artificial intelligence service, there is a need to utilize the information in existing unstructured documents, such as pdf, doc or docx, as well as information in text and pictures. When artificial intelligence uses these unstructured data, it needs to first extract the text and structure information from these data.

[0036] However, the formats of these data are diverse. For example, the PDF format includes scanned PDF, screenshot mixed PDF, and text PDF. Doc is similar to docx, and there are other marked documents such as Hypertext Markup Language (HTML) and Lightweight Markup Language (markdown) files, which brings some challenges to the extraction of document content.

[0037] In the related art, the following methods are usually used to extract document content: 1) convert the original files into images, and then perform optical character recognition (OCR) on the images; 2) use the visual big model to convert the original files into images and then into markdown; 3) parse various format files independently into text txt format. However, the above methods have some disadvantages. The disadvantage of method 1) is that it relies too much on OCR, which not only reduces the speed of extracting some data that was originally text-based, but also has extremely high requirements for OCR. For example, for some complex words and uncommon formula tables, OCR will lose information and also discard some non-text image information; the disadvantage of method 2) is that the calculation amount is too large and the speed is too slow, and it cannot express complex layout data, such as posters; the disadvantage of method 3) is that too much information is lost, and only some formatted text parts can be extracted, and visual information such as titles are lost, and complex layout data such as multi-column data cannot be extracted. In order to lower the threshold for intelligent services to use these document information, it is urgent to provide an extraction method to reduce these document information to a unified form of data, improve the speed of document extraction, and reduce the loss of original document information.

[0038] To address this problem, an embodiment of the present disclosure provides a document content extraction method, which is described below in conjunction with a specific embodiment.

[0039] Figure 1 A flow chart of a document content extraction method provided in an embodiment of the present disclosure. The executor of the method is an electronic device. The electronic device may be a portable mobile device such as a tablet computer or a laptop computer; or a fixed device such as a personal computer or a server, wherein the server may be a single server or a server cluster, and the server cluster may be a distributed cluster or a centralized cluster. The method may be applied to a scenario where document content is extracted. It is understandable that the document content extraction method provided in an embodiment of the present disclosure may also be applied in other scenarios.

[0040] Below Figure 1 The document content extraction method shown is introduced, and the method can be applied to electronic devices. The method includes the following specific steps:

[0041] S101, obtaining a target document to be extracted.

[0042] In this step, the electronic device acquires the target document to be extracted. For example, the user imports the target document to be extracted into the electronic device, and the electronic device acquires the target document to be extracted.

[0043] S102: Classify the target document to determine the document type of the target document, where the document type of the target document includes a portable document format type, a text type, a picture type, an office file type, and a hypertext markup language type.

[0044] In this step, if Figure 2 As shown, the electronic device will classify the input target document and determine the document type of the target document. Optionally, the document type of the target document includes a portable document format type, a text type, a picture type, an office file type, a hypertext markup language type, and may also include other types, which are not specifically limited here.

[0045] S103: Based on the document type of the target document, extract the document content of the target document using an extraction method corresponding to the document type to obtain a document in a target format.

[0046] In this step, after determining the document type of the target document, the electronic device will extract the document content of the target document according to the document type of the target document using an extraction method corresponding to the document type to obtain a document in a target format.

[0047] The target format document includes text attribute information, category information, location information, and background information of each region.

[0048] Optionally, the text attribute information includes text, font, font size, color, etc., the category information includes table, figure, code, formula, nested structure, etc., the position information includes pixel position or coordinate range. The background information includes background style (such as background image, background code, color), context layout, etc., which are expressed in vector form.

[0049] Optionally, the target format is KingSoft Multimodal Document (KMD).

[0050] In some optional embodiments, S103 includes but is not limited to S1031, S1032, and S1033:

[0051] S1031, if the document type of the target document is an office document type or a hypertext markup language type, converting the target document into a portable document format type document;

[0052] like Figure 2 As shown, if the document type of the target document is an office file type (ie, Office class) or a hypertext markup language type (ie, Html class), the electronic device will convert the target document into a portable document format PDF type document.

[0053] S1032: If the conversion is successful, extract the document content of the target document using an extraction method corresponding to the portable document format type document to obtain a target format document;

[0054] like Figure 2 As shown, if the PDF conversion is successful, the electronic device will extract the document content of the target document using an extraction method corresponding to the portable document format type document to obtain a target format document.

[0055] S1033: If the conversion fails, perform text extraction based on a text extraction method corresponding to the document type of the target document to obtain a document in a target format.

[0056] like Figure 2 As shown, if the PDF conversion fails, the electronic device will perform text extraction based on the text extraction method corresponding to the document type of the target document to obtain the target format document. Specifically, for office document types, the text extraction engine corresponding to the office document type is used to extract text to obtain the target format document; for hypertext markup language types, the text extraction engine corresponding to the hypertext markup language type is used to extract text to obtain the target format document.

[0057] In other optional implementations, based on the document type of the target document, the document content of the target document is extracted using an extraction method corresponding to the document type to obtain a target format document, including: if the document type of the target document is a text type, the target document is output as a target format document.

[0058] In this embodiment, if the document type of the target document is a text type, the electronic device will convert the target document into a target format document for output, so that the content of the original file data can be used to the maximum extent, without too much secondary processing, and the valid part of the original file format data is directly output, thereby improving the speed of document extraction.

[0059] The disclosed embodiment obtains the target document to be extracted, classifies the target document, determines the document type of the target document, and the document type of the target document includes a portable document format type, a text type, a picture type, an office file type, and a hypertext markup language type. Based on the document type of the target document, the document content of the target document is extracted using an extraction method corresponding to the document type to obtain a target format document. Compared with the prior art, the disclosed embodiment can increase the speed of document extraction by extracting the document content of the target document using an extraction method corresponding to the document type, is compatible with documents of any format, and any document can be converted into a target format, making it compatible with all documents, and retains the text attribute information, category information, location information, and background information of any document, thereby reducing the loss of original document information.

[0060] On the basis of the above embodiment, based on the document type of the target document, the document content of the target document is extracted using an extraction method corresponding to the document type to obtain the target format document. The method also includes: correcting the target format document and outputting the corrected target format document.

[0061] In this embodiment, the electronic device will perform correction processing on the target format document and output the corrected target format document, so that the extracted document content is closer to the target document and has higher accuracy.

[0062] In some embodiments, the correction processing method includes at least one of the following: removing empty areas in each area, adjusting the position of each area in the page, extracting the background image of the target document, merging areas of vertical text, normalizing the encoding format, calculating headers and footers, and nesting multiple areas corresponding to the same table.

[0063] Optionally, 1) remove the empty areas in each area, that is, remove the empty text area span, such as the space area. 2) Adjust the position of each area in the page, that is, adjust the position of each area span through text calculation and font calculation, so as to obtain better display and position information. 3) Extract the background image of the target document, that is, obtain other element information except text and pictures, and convert it into a picture, so that the rendering effect is more consistent with the original effect. 4) Merge the areas of vertical text, that is, merge the span, from 1D to 2D, to facilitate the subsequent conversion of text. 5) Normalize the encoding format, that is, convert some empty characters, special characters, font encoding, etc. 6) Calculate the header and footer, that is, calculate the page footer corresponding to each page, laying a good foundation for subsequent processing. 7) Perform nested processing on multiple areas corresponding to the same table, that is, perform table span processing. According to the table recognition result, multiple spans are summarized into corresponding table cells, and cells are summarized into corresponding table rows. That is, multiple areas of the same table cell are summarized into corresponding positions, and multiple table cells of the same table row are summarized into corresponding positions.

[0064] The disclosed embodiment corrects the target format document and outputs a corrected target format document, so that the extracted document content is closer to the target document and more accurate, and retains the text attribute information, category information, location information, and background information of any document, thereby reducing the loss of original document information.

[0065] Figure 3 A flowchart of a document content extraction method provided by another embodiment of the present disclosure is shown in FIG. Figure 3 As shown, the method includes the following steps:

[0066] S201: Obtain a target document to be extracted.

[0067] Specifically, the implementation process and principle of S201 and S101 are the same, and will not be repeated here.

[0068] S202: Classify the target document to determine the document type of the target document, where the document type of the target document includes a portable document format type, a text type, a picture type, an office file type, and a hypertext markup language type.

[0069] Specifically, the implementation process and principle of S202 and S102 are the same, and will not be repeated here.

[0070] S203: If the document type of the target document is a portable document format type, render the target document to render each page of the target document.

[0071] In this step, if Figure 2 As shown, if the document type of the target document is a portable document format type (ie, PDF), the electronic device will perform basic PDF rendering on the target document to render each page of the target document.

[0072] S204: For each page of the target document, divide the page into multiple areas.

[0073] In this step, for each page of the target document, Figure 5 As shown, the electronic device divides the page into multiple areas, namely spans, and the content selected by a rectangular frame is a region.

[0074] S205 . For any area of ​​the page, detect whether the area has embedded words.

[0075] In this step, the electronic device detects whether there is an embedded word in any area of ​​each page.

[0076] In some embodiments, if the region has embedded characters, OCR recognition is performed on the region.

[0077] S206: If there is no embedded word in the region, detect whether the region is image data.

[0078] like Figure 2 As shown, if there is no embedded word in the area, a large image detection is performed, that is, whether the area is image data.

[0079] In some embodiments, if the area is image data, that is, no text exists, the image data is directly output as the extraction result of the area.

[0080] S207: If the region is not image data, convert the region into a text-type image region.

[0081] In this step, if Figure 2 As shown, if the area is not image data, that is, text exists, the electronic device converts the area into a text-type image area.

[0082] S208: Perform text detection and text recognition on the text image type area to obtain a document content extraction result of the area.

[0083] Furthermore, the electronic device performs text detection and text recognition on the text-type image area to obtain a document content extraction result of the area.

[0084] S209: Merge the document content extraction results of each region to obtain text attribute information, category information, and position information of each region in the target format document.

[0085] like Figure 2 As shown, the electronic device will merge the results of each area. Specifically, the electronic device will merge the document content extraction results of each area based on the location information and logical relationship of each area. Optionally, the text attribute information includes text, font, font size, color, etc., the category information includes tables, figures, codes, formulas, nested structures, etc., and the location information includes pixel position or coordinate range.

[0086] S210, for each page of the target document, convert the page into a picture type page, extract the picture code of the picture type page, and then intercept the extracted background information of the page by region to obtain the background information of each region of the target format document.

[0087] like Figure 2 As shown, for each page of the target document, the electronic device converts each page of the target document from PDF to an image type page, performs image encoding CNN extraction on the image type page, extracts the background information of each page, and then intercepts the extracted background information of each page by region to obtain the background information of each region of the target format document. Optionally, the background information includes background style (such as background image, background encoding, color), context layout, etc., which are expressed in vector form.

[0088] S211, encoding the text attribute information, category information, location information of each region in the target format document and the background information of each region in correspondence with each other, and outputting the target format document.

[0089] like Figure 2 As shown, the electronic device encodes the text attribute information, category information, location information and background information of each area in the target format document according to the corresponding location area, and outputs the target format document, namely, the KMD format.

[0090] The disclosed embodiment obtains the target document to be extracted, classifies the target document, determines the document type of the target document, and the document type of the target document includes a portable document format type, a text type, a picture type, an office file type, and a hypertext markup language type. If the document type of the target document is a portable document format type, the target document is rendered to render each page of the target document. Then, for each page of the target document, the page is divided into multiple areas. For any area of ​​the page, it is detected whether the area has embedded words. If the area has no embedded words, it is detected whether the area is image data. Further, if the area is not image data, the area is converted into a text-type picture type area, and text detection and text recognition are performed on the text-type picture type area to obtain the document content extraction result of the area. The document content extraction results of each area are merged to obtain the text attribute information, category information, and location information of each area in the target format document. Then, for each page of the target document, the page is converted into a picture type page, the picture type page is subjected to picture coding extraction, and the background information of the extracted page is intercepted by region to obtain the background information of each region of the target format document, and the text attribute information, category information, position information of each region in the target format document and the background information of each region are coded and corresponded, and the target format document is output. Through this method, the speed of document extraction can be improved, and it can be compatible with documents of any format. No matter what document, it can be converted into the target format, so that it can be compatible with all documents, and the text attribute information, category information, position information, and background information of any document are retained, reducing the loss of original document information.

[0091] Figure 4 A flowchart of a document content extraction method provided by another embodiment of the present disclosure is shown in FIG. Figure 4 As shown, the method includes the following steps:

[0092] S301: Obtain a target document to be extracted.

[0093] Specifically, the implementation process and principle of S301 and S101 are the same, and will not be repeated here.

[0094] S302: Classify the target document to determine the document type of the target document, where the document type of the target document includes a portable document format type, a text type, a picture type, an office file type, and a hypertext markup language type.

[0095] Specifically, the implementation process and principle of S302 and S102 are the same, and will not be repeated here.

[0096] S303: If the document type of the target document is a picture type, each picture is divided into a region.

[0097] In this step, if Figure 2 As shown, if the document type of the target document is a picture type, the electronic device divides each picture into an area, namely, a span.

[0098] S304: Detect whether the area is a recognizable text image.

[0099] In this step, if Figure 2 As shown, the electronic device detects whether the image in the area is a recognizable text image.

[0100] S305: If the region is a recognizable text image, perform graphic correction processing on the region to obtain a corrected text image region.

[0101] In this step, if Figure 2 As shown, if the area is a recognizable text image, that is, a picture with text, the electronic device performs graphic correction processing on the area to obtain a corrected text image area. Optionally, the correction processing includes tilt adjustment, clarity, contrast, brightness, lighting adjustment, etc.

[0102] In some embodiments, if the region is an unrecognizable text-type image, that is, an image without any text, the image is directly output as the extraction result of the region.

[0103] S306: Perform text detection and text recognition on the corrected text image area to obtain a document content extraction result for the area.

[0104] In this step, if Figure 2 As shown, the electronic device performs text detection and text recognition on the corrected text image area to obtain a document content extraction result of the area.

[0105] S307: Merge the document content extraction results of each region to obtain text attribute information, category information, and position information of each region in the target format document.

[0106] In this step, if Figure 2 As shown, the electronic device will merge the results of each area. Specifically, the electronic device will merge the document content extraction results of each area based on the location information and logical relationship of each area. Optionally, the text attribute information includes text, font, font size, color, etc., the category information includes tables, figures, codes, formulas, nested structures, etc., and the location information includes pixel position or coordinate range.

[0107] S308 , for any area of ​​the target document, perform image encoding extraction on the area to obtain background information of each area of ​​the target format document.

[0108] In this step, if Figure 2 As shown, for any area of ​​the target document, the electronic device performs image encoding CNN extraction on the area to obtain background information of each area of ​​the target format document. Optionally, the background information includes background style (such as background image, background encoding, color), context layout, etc., expressed in vector form.

[0109] S309: Encode the text attribute information, category information, location information of each region in the target format document and the background information of each region in a corresponding manner, and output the target format document.

[0110] In this step, if Figure 2 As shown, the electronic device encodes the text attribute information, category information, location information and background information of each area in the target format document according to the corresponding location area, and outputs the target format document, namely, the KMD format.

[0111] The disclosed embodiment obtains the target document to be extracted, classifies the target document, and determines the document type of the target document. The document type of the target document includes a portable document format type, a text type, a picture type, an office file type, and a hypertext markup language type. If the document type of the target document is a picture type, each picture is divided into a region. Then, it is detected whether the region is a recognizable text-type picture. If the region is a recognizable text-type picture, the region is subjected to graphic correction processing to obtain a corrected text-type picture region. Further, the corrected text-type picture region is subjected to text detection and text recognition to obtain a document content extraction result of the region, and the document content extraction results of each region are merged to obtain text attribute information, category information, and position information of each region in the target format document. Then, for any region of the target document, the region is subjected to image coding extraction to obtain background information of each region of the target format document, and the text attribute information, category information, position information of each region in the target format document and the background information of each region are coded and corresponded to the position, and the target format document is output. Compared with the prior art, the embodiments of the present disclosure can improve the speed of document extraction and is compatible with documents of any format. Any document can be converted into a target format, making it compatible with all documents and retaining the text attribute information, category information, location information, and background information of any document, thereby reducing the loss of original document information.

[0112] Figure 6The document content extraction device provided in the embodiment of the present disclosure is a schematic diagram of the structure of the document content extraction device. The document content extraction device may be the electronic device described in the above embodiment, or the document content extraction device may be a component or assembly in the electronic device. The document content extraction device provided in the embodiment of the present disclosure may execute the processing flow provided in the document content extraction method embodiment, such as Figure 6 As shown, the document content extraction device 50 includes: an acquisition module 51, a classification module 52, and a obtaining module 53; wherein the acquisition module 51 is used to obtain the target document to be extracted; the classification module 52 is used to classify the target document and determine the document type of the target document, and the document type of the target document includes a portable document format type, a text type, a picture type, an office file type, and a hypertext markup language type; the obtaining module 53 is used to extract the document content of the target document based on the document type of the target document by adopting an extraction method corresponding to the document type to obtain a target format document; wherein the target format document includes text attribute information, category information, location information, and background information of each area.

[0113] Optionally, the obtaining module 53 extracts the document content of the target document based on the document type of the target document by adopting an extraction method corresponding to the document type, and when obtaining the target format document, is specifically used for: if the document type of the target document is a portable document format type, rendering the target document to render each page of the target document; for each page of the target document, dividing the page into multiple areas; for any area of ​​the page, detecting whether the area has embedded words; if the area has no embedded words, detecting whether the area is image data; if the area is not image data, converting the area into a text-type picture type area; Perform text detection and text recognition on a word-image type area to obtain a document content extraction result for the area; merge the document content extraction results of each area to obtain text attribute information, category information, and position information for each area in a target format document; for each page of the target document, convert the page into a picture type page, perform image encoding extraction on the picture type page, and then extract the background information of the extracted page by area to obtain background information for each area in the target format document; encode the text attribute information, category information, position information of each area in the target format document and the background information of each area to correspond to the encoding position, and output the target format document.

[0114] Optionally, the obtaining module 53 extracts the document content of the target document based on the document type of the target document by adopting an extraction method corresponding to the document type, and when obtaining the target format document, it is specifically used for: if the document type of the target document is a picture type, each picture is divided into an area; whether the area is a recognizable text picture; if the area is a recognizable text picture, the area is subjected to graphic correction processing to obtain a corrected text picture area; text detection and text recognition are performed on the corrected text picture area to obtain the document content extraction result of the area; the document content extraction results of each area are merged to obtain the text attribute information, category information, and position information of each area in the target format document; for any area of ​​the target document, the area is subjected to image coding extraction to obtain the background information of each area of ​​the target format document; the text attribute information, category information, position information of each area in the target format document and the background information of each area are encoded in position correspondence, and the target format document is output.

[0115] Optionally, the obtaining module 53 extracts the document content of the target document based on the document type of the target document by adopting an extraction method corresponding to the document type to obtain the target format document, and is specifically used for: if the document type of the target document is an office document type or a hypertext markup language type, converting the target document into a portable document format type document; if the conversion is successful, extracting the document content of the target document by adopting an extraction method corresponding to the portable document format type document to obtain the target format document; if the conversion fails, performing text extraction based on a text extraction method corresponding to the document type of the target document to obtain the target format document.

[0116] Optionally, the obtaining module 53 extracts the document content of the target document based on the document type of the target document by adopting an extraction method corresponding to the document type to obtain the target format document. Specifically, if the document type of the target document is a text type, the target document is output as a target format document.

[0117] Optionally, based on the document type of the target document, the document content of the target document is extracted using an extraction method corresponding to the document type to obtain the target format document. The document content extraction device 50 also includes: an output module 54; the output module 54 is used to correct the target format document and output the corrected target format document.

[0118] Optionally, the correction processing method includes at least one of the following: removing empty areas in each area, adjusting the position of each area in the page, extracting the background image of the target document, merging areas of vertical text, normalizing the encoding format, calculating headers and footers, and nesting multiple areas corresponding to the same table.

[0119] Figure 6 The document content extraction device of the illustrated embodiment can be used to execute the technical solution of the above-mentioned method embodiment, and its implementation principle and technical effect are similar and will not be described in detail here.

[0120] Figure 7 Schematic diagram of the structure of an electronic device in the embodiment of the present disclosure. Figure 7 , which shows a structural schematic diagram of an electronic device 600 suitable for implementing the embodiments of the present disclosure. Figure 7 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0121] like Figure 7 As shown, the electronic device 600 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage device 608 into a random access memory (RAM) 603 to implement the document content extraction method of the embodiment described in the present disclosure. In the RAM 603, various programs and data required for the operation of the electronic device 600 are also stored. The processing device 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0122] Typically, the following devices may be connected to the I / O interface 605: an input device 606 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 608 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 609. The communication device 609 may allow the electronic device 600 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 7 The electronic device 600 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0123] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains a program code for executing the method shown in the flowchart, thereby implementing the document content extraction method as described above. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 609, or installed from the storage device 608, or installed from the ROM 602. When the computer program is executed by the processing device 601, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0124] It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0125] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0126] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0127] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device:

[0128] Get the target document to be extracted;

[0129] Classifying the target document to determine the document type of the target document, wherein the document type of the target document includes a portable document format type, a text type, a picture type, an office file type, and a hypertext markup language type;

[0130] Based on the document type of the target document, extracting the document content of the target document using an extraction method corresponding to the document type to obtain a document in a target format;

[0131] The target format document includes text attribute information, category information, location information, and background information of each region.

[0132] Optionally, when the above one or more programs are executed by the electronic device, the electronic device may also execute other steps described in the above embodiments.

[0133] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including, but not limited to, object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0134] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0135] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware, wherein the name of a unit does not, in some cases, limit the unit itself.

[0136] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0137] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0138] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other to form a technical solution.

[0139] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0140] Although the subject matter has been described in language specific to structural features and / or methodological logical actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims.

Claims

1. A document content extraction method, characterized in that: The method comprises: Get the target document to be extracted; Classifying the target document to determine the document type of the target document, wherein the document type of the target document includes a portable document format type, a text type, a picture type, an office file type, and a hypertext markup language type; Based on the document type of the target document, extracting the document content of the target document using an extraction method corresponding to the document type to obtain a document in a target format; The target format document includes text attribute information, category information, location information, and background information of each region.

2. The method according to claim 1, characterized in that: The step of extracting the document content of the target document based on the document type of the target document by using an extraction method corresponding to the document type to obtain a document in a target format includes: If the document type of the target document is a portable document format type, rendering the target document to render each page of the target document; For each page of the target document, dividing the page into a plurality of regions; For any area of ​​the page, detecting whether the area has an embedded word; If there is no embedded word in the region, detecting whether the region is image data; If the area is not image data, converting the area into a text-type image area; Performing text detection and text recognition on the text-type image area to obtain a document content extraction result of the area; Merge the document content extraction results of each region to obtain text attribute information, category information, and location information of each region in the target format document; For each page of the target document, convert the page into a picture type page, extract the picture code of the picture type page, and then intercept the extracted background information of the page by region to obtain the background information of each region of the target format document; The text attribute information, category information, location information of each area in the target format document and the background information of each area are encoded and corresponded to each other, and the target format document is output.

3. The method according to claim 1, characterized in that: The step of extracting the document content of the target document based on the document type of the target document by using an extraction method corresponding to the document type to obtain a document in a target format includes: If the document type of the target document is a picture type, each picture is divided into a region; Detecting whether the area is a recognizable text image; If the region is a recognizable text image, then the region is subjected to graphic correction processing to obtain a corrected text image region; Performing text detection and text recognition on the corrected text image area to obtain a document content extraction result of the area; Merge the document content extraction results of each region to obtain text attribute information, category information, and location information of each region in the target format document; For any area of ​​the target document, image encoding and extraction are performed on the area to obtain background information of each area of ​​the target format document; The text attribute information, category information, location information of each area in the target format document and the background information of each area are encoded and corresponded to each other, and the target format document is output.

4. The method according to claim 1, characterized in that: The step of extracting the document content of the target document based on the document type of the target document by using an extraction method corresponding to the document type to obtain a document in a target format includes: If the document type of the target document is an office document type or a hypertext markup language type, converting the target document into a portable document format type document; If the conversion is successful, extracting the document content of the target document using an extraction method corresponding to the portable document format type document to obtain a target format document; If the conversion fails, text extraction is performed based on a text extraction method corresponding to the document type of the target document to obtain a document in a target format.

5. The method according to claim 1, characterized in that The step of extracting the document content of the target document based on the document type of the target document by using an extraction method corresponding to the document type to obtain a document in a target format includes: If the document type of the target document is a text type, the target document is output as a target format document.

6. The method according to claim 1, characterized in that After extracting the document content of the target document based on the document type of the target document and adopting an extraction method corresponding to the document type to obtain the target format document, the method further includes: The target format document is corrected and a corrected target format document is output.

7. The method according to claim 6, characterized in that The correction processing method includes at least one of the following: Remove empty areas in each area, adjust the position of each area on the page, extract the background image of the target document, merge the areas of vertical text, normalize the encoding format, calculate the header and footer, and nest multiple areas corresponding to the same table.

8. A document content extraction device, characterized in that: include: An acquisition module is used to acquire the target document to be extracted; A classification module, used to classify the target document and determine the document type of the target document, wherein the document type of the target document includes a portable document format type, a text type, a picture type, an office file type, and a hypertext markup language type; An obtaining module, used for extracting the document content of the target document based on the document type of the target document by adopting an extraction method corresponding to the document type to obtain a document in a target format; The target format document includes text attribute information, category information, location information, and background information of each region.

9. An electronic device, characterized in that: include: Memory; processor; as well as Computer programs; The computer program is stored in the memory and is configured to be executed by the processor to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images